Fetal heart standard section detection method based on multiple tasks

Abstract

The invention relates to the technical field of medical ultrasonic image processing, in particular to a fetal heart standard section detection method based on multiple tasks, which comprises the following steps: acquiring a to-be-detected fetal heart section image, inputting a target detection model, outputting types, quantity and positions of anatomical structures in the to-be-detected fetal heart section image, and classifying the to-be-detected fetal heart section image based on a classification algorithm. And obtaining a judgment and classification result of the fetal heart section image to be detected. On the basis of a target detection model path fusion feature pyramid, a self-adaptive spatial feature fusion module containing sampling operation and small-size feature map fusion branches and an interleaving path are added, and a path interleaving pyramid network based on the self-adaptive spatial feature fusion module is constructed as a neck network of a target detection model. And meanwhile, a detail fusion enhancement module introducing an optimized feature fusion strategy is utilized, so that the recognition precision of a key anatomical structure in the fetal heart ultrasonic image and the accuracy of section judgment and classification results are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical ultrasonic image processing, and in particular to a multi-task based fetal heart standard section detection method. Background Art

[0002] Congenital heart disease is a common congenital malformation disease with an incidence rate of about 4‰. It is one of the main causes of death in newborns and infants. The disability and long-term medical needs caused by congenital heart disease are a heavy burden on individuals, families and society. Early detection, early diagnosis, early intervention and early treatment of congenital heart disease can reduce infant mortality, reduce disability and help improve the quality of the newborn population. Therefore, prenatal screening for congenital heart disease is a crucial link.

[0003] Ultrasound testing is safe, non-invasive, economical, convenient, and can monitor maternal changes and fetal growth and development throughout the pregnancy cycle. It has become the most widely used imaging test method in obstetric examinations. Since its application in the assessment of fetal heart structure and function, ultrasound testing has effectively improved the detection rate of congenital heart disease and played an important role in the diagnosis and treatment of fetal heart disease; however, the fetal heart is small in size, complex in structure, has a high beating rate, and is easily affected by the position. In addition, due to the complex interaction between ultrasound and maternal / fetal biological tissues, fetal heart ultrasound images have problems such as low contrast, signal attenuation, speckle noise, and artifacts. It is still very challenging to obtain high-quality and high-standard fetal ultrasound images, which is very dependent on the experience of physicians.

[0004] With the development of image processing technology and the continuous innovation of high-quality imaging equipment, auxiliary technology based on traditional machine learning has been applied to cardiac ultrasound image processing; however, since traditional machine learning methods mainly obtain image features through a large amount of image preprocessing, the selection of features requires expert cognition and the artificially set features are susceptible to interference, and have poor adaptability and practicality, the traditional image detection methods are applied to fetal ultrasound image analysis with low accuracy for fetal hearts that are small in size, complex in structure, high in beating rate, and whose position changes with body position.

[0005] In recent years, deep learning has become a research hotspot in the field of machine learning. The development of deep learning provides a new idea for solving the problem of fetal ultrasound image analysis accuracy. Faced with fetal ultrasound images with general image quality and constantly changing heart positions, it is difficult to design feature operators that can effectively characterize cardiac sections or internal anatomical structures using traditional methods. Compared with traditional image classification methods, deep learning methods avoid the problem of artificially setting and selecting target features in the process of detection and classification; deep learning learns a large amount of training data so that the network learns the most appropriate weights from the data to extract the most effective features; through the design of multi-layer networks, the deep network obtains deep semantic information; end-to-end training is used to merge the two steps of feature extraction and fusion. Therefore, applying deep learning technology based on convolutional neural networks to the recognition and classification of fetal ultrasound images can effectively improve the recognition accuracy; however, when faced with changes in perspective, scale, and shape changes caused by cardiac contraction and relaxation, the deep learning network has poor adaptability and insufficient generalization ability. These factors lead to challenges in solving problems in the field of fetal ultrasound. Therefore, it has very important research significance. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problems of low contrast, signal attenuation, speckle noise and artifacts in fetal heart ultrasound images in the prior art; insufficient adaptability and generalization ability when facing changes in viewing angle, scale and shape, and difficulty in simultaneously improving the accuracy of anatomical structure recognition and realizing multiple section classification and standard section discrimination.

[0007] In order to solve the above technical problems, the present invention provides a multi-task based fetal heart standard section detection method, comprising: Obtain the fetal heart section image to be detected The initial feature map of each scale is input into the neck network of the target detection model, including: The first The initial feature map of each scale is input into the top-down pyramid network. A preliminary fusion layer is used to fuse the low-scale feature map after upsampling and two-dimensional convolution operation with the adaptive fusion feature map of multiple pre-selected intermediate-scale initial feature maps output by the adaptive spatial feature fusion module, and output the second to third The initial fusion feature map of the scale; using the The initial fusion layer After upsampling and two-dimensional convolution operations, the preliminary fused feature map of the first scale is fused with the initial feature map of the first scale to output the preliminary fused feature map of the first scale; The initial fusion feature map of the first scale is input into the bottom-up pyramid network. An enhanced fusion layer is used to fuse the high-scale feature map after the convolution operation and the adaptive fusion feature map of multiple pre-selected intermediate-scale preliminary fusion feature maps output by the adaptive spatial feature fusion module, and output the 2nd to 3rd scales. The enhanced fusion feature map of the scale; using the The enhanced fusion layer After convolution operation on the enhanced fusion feature map of the first scale, The initial feature maps of scales are fused and output Enhanced fusion feature map of each scale; The preliminary fusion feature map of the first scale and the second to third scales The enhanced fusion feature maps of each scale are input into the head network of the target detection model to determine the type, number and position of the anatomical structure in the fetal heart section image to be detected, and then the judgment and classification results of the fetal heart section image to be detected are obtained based on the classification algorithm.

[0008] Preferably, the neck network comprises: a plurality of adaptive spatial feature fusion modules, a top-down pyramid network and a bottom-up pyramid network; The top-down pyramid network contains A preliminary fusion layer; each preliminary fusion layer includes an upsampling module, a two-dimensional convolution module, and a detail fusion enhancement module connected in sequence; The bottom-up pyramid network contains Enhanced fusion layers; each enhanced fusion layer includes a neck convolution module and a detail fusion enhancement module connected in sequence; The adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale initial feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the preliminary fusion layer corresponding to the adaptive spatial feature fusion module; it is also used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale preliminary fusion feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the enhanced fusion layer corresponding to the adaptive spatial fusion feature module; The multiple pre-selected intermediate scale initial feature maps represent the second scale to the first scale. The initial feature map of the scale; the multiple pre-selected intermediate scale preliminary fusion feature maps represent the initial feature map of the scale from the second scale to the first scale. Preliminary fusion feature map of different scales.

[0009] Preferably, the adaptive spatial feature fusion module includes a feature adjustment unit, a feature fusion unit and a feature screening unit; The feature adjustment unit is composed of a sampling operation, two convolution operations with convolution kernels of 1×1, and a splicing operation; The feature fusion unit is composed of a convolution operation with a convolution kernel of 1×1, an output channel number of 2, and an activation function of Softmax; The feature screening unit is composed of a convolution block with a convolution kernel of 1×1 and a plurality of residual blocks connected in series; each residual block is composed of a 3×3 convolution and a jump connection.

[0010] Preferably, inputting two to-be-processed feature maps of different scales and different numbers of channels into the adaptive spatial feature fusion module comprises: Input two feature maps to be processed with different scales and different numbers of channels into the feature adjustment unit, perform sampling operation and then convolution operation on the two feature maps to be processed respectively, adjust the scales of the two feature maps to be processed to the target scale, adjust the number of channels to the same number of channels, and then splice them to output the spliced ​​feature map; The concatenated feature map is input into the feature fusion unit, and the weights corresponding to the two feature maps to be processed are obtained through convolution operation. The two feature maps to be processed and their corresponding weights are weighted summed to output a weighted fusion feature map. The weighted fusion feature map is input into the feature screening unit, and the number of channels of the weighted fusion feature map is adjusted to the target number of channels through the convolution block and the residual block, and an adaptive fusion feature map of the target scale and the target number of channels is output; Among them, the output scale and output channel number of the preliminary fusion layer or the enhanced fusion layer of each adaptive spatial feature fusion module are respectively used as the target scale and target channel number of each adaptive spatial feature fusion module.

[0011] Preferably, the detail fusion enhancement module includes a channel attention mechanism unit and a feature focusing unit; The channel attention mechanism unit is composed of a splicing operation, an average pooling operation, a maximum pooling operation, multiple groups of convolution operations, a ReLU activation function, and a Sigmoid activation function; The feature focusing unit is composed of a convolution operation, a parallel convolution column and a Sigmoid activation function; there are two convolution columns in the parallel convolution column, one convolution column first performs an average pooling and then performs two convolution operations, and the other convolution column directly performs two convolution operations.

[0012] Preferably, inputting two feature maps to be fused of the same scale and the same number of channels into the detail fusion enhancement module comprises: Input two feature maps to be fused of the same scale and the same number of channels into the channel attention mechanism unit, and perform a splicing operation on the two feature maps to be fused to obtain a spliced ​​fused feature map; first perform average pooling and maximum pooling operations on the spliced ​​fused feature map, respectively, and then perform multiple groups of convolution and ReLU activation function processing respectively, and finally perform addition processing to obtain the processed spliced ​​fused feature map, and then use the Sigmoid activation function to obtain the channel weights corresponding to the two feature maps to be fused; perform element-by-element multiplication of the two feature maps to be fused and their corresponding channel weights to obtain two preliminary enhanced feature maps; sum the two preliminary enhanced feature maps, and output the channel information enhanced feature map; The channel information enhanced feature map is input into the feature focusing unit. After the convolution operation and the parallel convolution column, the results obtained by the parallel convolution column are multiplied, and then the aggregation weights corresponding to the two preliminary enhanced feature maps are obtained through the Sigmoid activation function. The two preliminary enhanced feature maps and their corresponding aggregation weights are weighted summed to obtain the output feature map of the detail fusion enhancement module.

[0013] Preferably, the front The preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ; in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current preliminary fusion layer; Represents the feature map output by the two-dimensional convolution module of the current preliminary fusion layer; the output scale and number of output channels of each preliminary fusion layer are the same as the output scale and number of output channels of the two-dimensional convolution module of the preliminary fusion layer; No. The first scale preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ; in, Indicates The initial feature map of scales; Indicates The feature map output by the two-dimensional convolution module of the preliminary fusion layer; forward Enhanced fusion feature map output by the detail fusion enhancement module of the enhanced fusion layer The expression is: ; in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current enhanced fusion layer; Represents the feature map output by the neck convolution module of the current enhanced fusion layer; the output scale and number of output channels of each enhanced fusion layer are the same as the output scale and number of output channels of the neck convolution module of the enhanced fusion layer; No. The output of the detail fusion enhancement module of the enhanced fusion layer is Scale enhanced fusion feature map The expression is: ; in, Indicates The initial feature map of scales; Indicates The feature map output by the neck convolution module of the enhanced fusion layer; Represents the channel attention mechanism; Represents feature aggregation; Represents a concatenation operation.

[0014] Preferably, the step of acquiring the fetal heart section image to be detected is: The initial feature maps of each scale include: From the fetal heart ultrasound image, obtain the fetal heart section image to be detected, input it into the backbone network of the target detection model, and output the fetal heart section image to be detected. The initial feature map of scales; The backbone network includes multiple backbone convolution modules, multiple C2f modules and SPPF modules; Using each C2f module, the feature map output by the previous trunk convolution module is split, spliced ​​and fused to obtain the first to the The C2f module outputs feature maps of scales; The C2f module outputs feature maps of each scale as the first to the second output of the backbone network. The initial feature map of scales; Using the SPPF module, The C2f module outputs the feature map of the scale for pooling operation and outputs the backbone network The initial feature map of the scale.

[0015] Preferably, it also includes: Selecting multiple standard section images of different types from multiple fetal heart ultrasound images; wherein the standard section image types include four-chamber heart section images, three-vessel section images, three-vessel tracheal section images, left ventricular outflow tract section images, and right ventricular outflow tract section images; Perform translation, rotation, and zoom-in / out preprocessing operations on each standard section image to obtain multiple preprocessed section images corresponding to each standard section image; construct an extended data sample set based on all standard section images and their corresponding multiple preprocessed section images; According to the standard section requirements, the roLabelImg script is used to use a rotating rectangular box to annotate various anatomical structures of each section image in the extended data sample set, and the vertex coordinates and anatomical structure types of each rotating rectangular box corresponding to each section image are recorded through a label file; the anatomical structure types include: left ventricle, left atrium, right ventricle, right atrium, aorta, aortic arch, descending aorta, superior vena cava, pulmonary artery, ductus arteriosus, right ventricular outflow tract, trachea, and spine; Based on all the cross-sectional images in the expanded data sample set and their corresponding label files, a fetal heart ultrasound image dataset is constructed; According to a preset ratio, the fetal heart ultrasound image dataset is divided into a training set and a test set for training and testing the target detection model and the classification algorithm; wherein, based on the classification algorithm, the judgment result obtained is to judge whether the fetal heart section image to be detected is a standard section image; based on the classification algorithm, the classification result obtained is to determine the standard section image type corresponding to the fetal heart section image to be detected.

[0016] Preferably, based on the target detection model and the classification algorithm, a multi-task based fetal heart standard section detection model is constructed, and the training process of the multi-task based fetal heart standard section detection model includes: Based on the training set, the Adam back-propagation algorithm is used to iteratively optimize the multi-task fetal heart standard section detection model until the total loss function converges, and a trained multi-task fetal heart standard section detection model is obtained; The total loss function consists of classification loss, intersection-over-union loss, and distribution focus loss, and its expression is: ; in, represents the total loss function; represents the classification loss; Represents the weight coefficient corresponding to the classification loss; represents the intersection-and-combination loss; Represents the weight coefficient corresponding to the intersection-over-union loss; represents the distribution focal loss; Represents the weight coefficient corresponding to the distribution focus; The expression of the classification loss is: ; The expression of the intersection-over-union loss is: ; The expression of the distribution focus loss is: ; in, represents the total number of samples; Representation sample One-hot tags, Representation sample The probability of predicting the positive class; is the Gaussian probability intersection and union ratio; is the true coordinate value of the sample; and are two adjacent values ​​of the bounding box position coordinates predicted by the model; and is the weight corresponding to two adjacent values ​​of the bounding box position coordinates predicted by the model.

[0017] The above technical solution of the present invention has the following beneficial effects compared with the prior art: The multi-task-based fetal heart standard section detection method described in the present invention constructs a fetal heart ultrasound image dataset through data expansion and precise labeling, which covers rich samples and accurate labeling information, provides sufficient and high-quality data for model training, enables the model to learn more comprehensive and accurate features, and lays a solid foundation for subsequent high-precision detection and classification.

[0018] The present invention discloses a multi-task based fetal heart standard section detection method, which is based on a path fusion feature pyramid network. By introducing an adaptive spatial feature fusion module including a sampling operation, a path interleaving feature pyramid structure based on the adaptive spatial feature fusion module is constructed. The structure is used as the neck network of the YOLOv8-OBB target detection model, and a small-size feature map fusion branch is added to enhance feature diversity and improve target detection capability. Feature maps of different scales are adjusted, fused and screened by the adaptive spatial feature fusion module, which effectively integrates information of different scales, so that the model can better capture the anatomical structure of the fetal heart with large size differences, and enhance multi-scale target detection capability and generalization capability.

[0019] The present invention discloses a multi-task-based fetal heart standard section detection method, which extracts the global information and maximum information of the feature map in parallel through the channel attention mechanism unit in the detail fusion enhancement module, and obtains the key channel information, thereby enhancing the sensitivity to important features; obtains the prominent semantic information and local information in the feature map through the feature focusing unit in the detail fusion enhancement module; and enables the model to focus more on key features through the optimized feature fusion strategy of the detail fusion enhancement module, thereby reducing the feature redundancy caused by problems such as low contrast of fetal heart ultrasound images, signal attenuation, speckle noise and artifacts, thereby improving the accuracy and performance of the detection of key anatomical structures of the fetal heart.

[0020] The present invention discloses a multi-task based fetal heart standard section detection method, which adds a classification algorithm based on the improved YOLOv8-OBB target detection. The model not only achieves high-precision classification based on the type, quantity and position of the identified anatomical structures, but also provides rich anatomical information, accurately determines whether the fetal heart section image to be detected is a standard section, and classifies five fetal heart ultrasound standard sections, providing strong support for prenatal diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 It is a flow chart of a multi-task based fetal heart standard section detection method provided by the present invention; Figure 2 It is the structural diagram of the detail fusion enhancement module; Figure 3 It is a structural diagram of a path interleaving feature pyramid and an adaptive spatial feature fusion module based on an adaptive spatial feature fusion module; wherein, Figure 3 The dotted box on the left is the path interleaving feature pyramid structure based on the adaptive spatial feature fusion module; Figure 3 The right side shows the adaptive spatial feature fusion module structure; Figure 4 This is a structural diagram of the multi-task based fetal heart standard section detection model; Upsample represents the upsampling module; Conv2d represents the two-dimensional convolution module; ASFF represents the adaptive spatial feature fusion module; DFE represents the detail fusion enhancement module; OBB represents the detection head; Figure 5 It is a schematic diagram of the anatomical structure identification and classification results of the standard four-chamber view of the fetal heart; Figure 6 It is a schematic diagram of the anatomical structure identification and classification results of the standard three-vessel section of the fetal heart; Figure 7It is a schematic diagram of the anatomical structure identification and classification results of the standard three-vessel tracheal section of the fetal heart; Figure 8 It is a schematic diagram of the anatomical structure identification and classification results of the standard left ventricular outflow tract section of the fetal heart; Fig. 9 It is a schematic diagram of the anatomical structure identification and classification results of the standard right ventricular outflow tract section of the fetal heart. DETAILED DESCRIPTION

[0022] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0023] Embodiment 1 Reference Figure 1 As shown, Figure 1 The present invention provides a flowchart of a multi-task based fetal heart standard section detection method; specifically comprising: S1: Obtain a cross-sectional image of the fetal heart to be detected from the fetal heart ultrasound image.

[0024] S2: Using the target detection model, the types, numbers and locations of anatomical structures in the fetal heart section image to be detected are obtained, including: S21: Input the fetal heart section image to be detected into the backbone network of the target detection model, and output the fetal heart section image to be detected. The backbone network includes a plurality of backbone convolution modules, a plurality of C2f modules and an SPPF module; Using each C2f module, the feature map output by the previous trunk convolution module is split, spliced ​​and fused to obtain the first to the The C2f module outputs feature maps of scales; The C2f module outputs feature maps of each scale as the first to the second output of the backbone network. The initial feature map of scales; Using the SPPF module, The C2f module outputs the feature map of the scale for pooling operation and outputs the backbone network The initial feature map of the scale.

[0025] S22: The initial feature maps of each scale are input into the neck network of the target detection model, including: S221: The neck network includes multiple adaptive spatial feature fusion modules, a top-down pyramid network and a bottom-up pyramid network; The top-down pyramid network contains preliminary fusion layers; each preliminary fusion layer includes an upsampling module, a two-dimensional convolution module and a detail fusion enhancement module; The bottom-up pyramid network contains There are three enhanced fusion layers; each enhanced fusion layer includes a neck convolution module and a detail fusion enhancement module.

[0026] S222: The detail fusion enhancement module includes a channel attention mechanism unit and a feature focusing unit; in a specific embodiment of the present invention, the structure of the detail fusion enhancement module is as follows Figure 2 As shown; The channel attention mechanism unit is composed of a splicing operation, an average pooling operation, a maximum pooling operation, multiple groups of convolution operations, a ReLU activation function, and a Sigmoid activation function; The feature focusing unit is composed of a convolution operation, a parallel convolution column and a Sigmoid activation function; there are two convolution columns in the parallel convolution column, one convolution column first performs an average pooling and then performs two convolution operations, and the other convolution column directly performs two convolution operations.

[0027] Two feature maps to be fused with the same scale and the same number of channels are input into the detail fusion enhancement module, including: Input two feature maps to be fused of the same scale and the same number of channels into the channel attention mechanism unit, and perform a splicing operation on the two feature maps to be fused to obtain a spliced ​​fused feature map; first perform average pooling and maximum pooling operations on the spliced ​​fused feature map, respectively, and then perform multiple groups of convolution and ReLU activation function processing respectively, and finally perform addition processing to obtain the processed spliced ​​fused feature map, and then use the Sigmoid activation function to obtain the channel weights corresponding to the two feature maps to be fused; perform element-by-element multiplication of the two feature maps to be fused and their corresponding channel weights to obtain two preliminary enhanced feature maps; sum the two preliminary enhanced feature maps, and output the channel information enhanced feature map; The channel information enhanced feature map is input into the feature focusing unit. After the convolution operation and the parallel convolution column, the results obtained by the parallel convolution column are multiplied, and then the aggregation weights corresponding to the two preliminary enhanced feature maps are obtained through the Sigmoid activation function. The two preliminary enhanced feature maps and their corresponding aggregation weights are weighted summed to obtain the output feature map of the detail fusion enhancement module.

[0028] Before use The detail fusion enhancement module of each preliminary fusion layer in the preliminary fusion layer fuses the feature map output by the two-dimensional convolution module of the preliminary fusion layer and the adaptive fusion feature map output by the adaptive spatial fusion feature module with the same output scale and output channel number of the preliminary fusion layer. The preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ; in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current preliminary fusion layer; Represents the feature map output by the two-dimensional convolution module of the current preliminary fusion layer; the output scale and number of output channels of each preliminary fusion layer are the same as the output scale and number of output channels of the two-dimensional convolution module of the preliminary fusion layer; Use the The detail fusion enhancement module of the preliminary fusion layer fuses the output of the backbone network. The initial feature map of the scale and the The feature map output by the two-dimensional convolution module of the preliminary fusion layer, then The first scale preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ; in, Represents the output of the backbone network The initial feature map of scales; Indicates The feature map output by the two-dimensional convolution module of the preliminary fusion layer; Before use The detail fusion enhancement module of each enhanced fusion layer in the enhanced fusion layer fuses the feature map output by the neck convolution module of the enhanced fusion layer and the adaptive fusion feature map output by the adaptive spatial fusion feature module with the same output scale and output channel number of the enhanced fusion layer. Enhanced fusion feature map output by the detail fusion enhancement module of the enhanced fusion layer The expression is: ; in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current enhanced fusion layer; Represents the feature map output by the neck convolution module of the current enhanced fusion layer; the output scale and number of output channels of each enhanced fusion layer are the same as the output scale and number of output channels of the neck convolution module of the enhanced fusion layer; Use the The detail fusion enhancement module of the enhanced fusion layer fuses the output of the backbone network. The initial feature map of the scale and the The feature map output by the neck convolution module of the enhanced fusion layer, then The output of the detail fusion enhancement module of the enhanced fusion layer is Scale enhanced fusion feature map The expression is: ; in, Represents the output of the backbone network The initial feature map of scales; Indicates The feature map output by the neck convolution module of the enhanced fusion layer; Represents the channel attention mechanism; Represents feature aggregation; Represents a concatenation operation.

[0029] S223: The adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale initial feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the preliminary fusion layer corresponding to the adaptive spatial feature fusion module; and is also used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale preliminary fusion feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the enhanced fusion layer corresponding to the adaptive spatial fusion feature module; The multiple pre-selected intermediate scale initial feature maps represent the second scale to the The initial feature map of the scale; the multiple pre-selected intermediate scale preliminary fusion feature maps represent the initial feature map of the scale from the second scale to the first scale. Preliminary fusion feature map of different scales.

[0030] The adaptive spatial feature fusion module includes a feature adjustment unit, a feature fusion unit and a feature screening unit; in a specific embodiment of the present invention, the structure of the adaptive spatial feature fusion module is as follows: Figure 3 The network structure is shown in the dashed box on the left; The feature adjustment unit is composed of a sampling operation, two convolution operations with convolution kernels of 1×1, and a splicing operation; The feature fusion unit is composed of a convolution operation with a convolution kernel of 1×1, an output channel number of 2, and an activation function of Softmax; The feature screening unit is composed of a convolution block with a convolution kernel of 1×1 and a plurality of residual blocks connected in series; each residual block is composed of a 3×3 convolution and a jump connection.

[0031] Two feature maps to be processed with different scales and different numbers of channels are input into the adaptive spatial feature fusion module, including: Input two feature maps to be processed with different scales and different numbers of channels into the feature adjustment unit, perform sampling operation and then convolution operation on the two feature maps to be processed respectively, adjust the scales of the two feature maps to be processed to the target scale, adjust the number of channels to the same number of channels, and then splice them to output the spliced ​​feature map; The concatenated feature map is input into the feature fusion unit, and the weights corresponding to the two feature maps to be processed are obtained through convolution operation. The two feature maps to be processed and their corresponding weights are weighted summed to output a weighted fusion feature map. The weighted fusion feature map is input into the feature screening unit, and the number of channels of the weighted fusion feature map is adjusted to the target number of channels through the convolution block and the residual block, and an adaptive fusion feature map of the target scale and the target number of channels is output; Among them, the output scale and output channel number of the preliminary fusion layer or the enhanced fusion layer of each adaptive spatial feature fusion module are respectively used as the target scale and target channel number of each adaptive spatial feature fusion module.

[0032] In a specific implementation of the present invention, the adaptive spatial feature fusion module is used to The scale and The initial feature map of each scale is resized, feature fused, and the number of channels is adjusted; it is also used to adjust the scale of the first feature map output by the top-down pyramid network. The scale and The initial fusion feature map of each scale is used for scale adjustment, feature fusion and channel number adjustment operations; among them, , ; The output of the backbone network The scale and The initial feature map of each scale is used as the input of the adaptive spatial feature fusion module, and its output adaptive fusion feature map The expression is: ; ; ; ; in, Represents the output of the backbone network The initial feature map of scales; Represents the output of the backbone network The initial feature map of scales; Indicates The feature map after sampling and convolution operation of the initial feature map of the scale; Indicates The feature map after sampling and convolution operation of the initial feature map of the scale; The top-down pyramid network output The scale and The initial fusion feature map of each scale is used as the input of the adaptive spatial feature fusion module, and its output adaptive fusion feature map The expression is: ; ; ; ; in, Represents the output of the top-down pyramid network. Preliminary fusion feature maps of scales; Represents the output of the top-down pyramid network. Preliminary fusion feature maps of various scales; Indicates The feature map after sampling and convolution operation of the preliminary fusion feature map of each scale; Indicates The feature map after sampling and convolution operation of the preliminary fusion feature map of each scale; represents a feature adjustment unit; represents the feature fusion unit; represents a feature screening unit; Indicates sampling operation; Indicates a convolution operation with a convolution kernel of 1×1 and an output channel number of 2; Represents a splicing operation; Indicates a convolution operation with a convolution kernel of 1×1, 2 output channels, and a softmax activation function; Indicates addition according to the channel dimension; Indicates a convolution operation with a convolution kernel of 1×1 and an unchanged number of output channels; express A residual block.

[0033] S224: The initial feature map of each scale is input into the top-down pyramid network. A preliminary fusion layer is used to fuse the low-scale feature map after upsampling and two-dimensional convolution operation with the adaptive fusion feature map of multiple pre-selected intermediate-scale initial feature maps output by the adaptive spatial feature fusion module, and output the second to third The initial fusion feature map of the scale; using the The initial fusion layer After upsampling and two-dimensional convolution operations, the preliminary fused feature map of the first scale is fused with the initial feature map of the first scale to output the preliminary fused feature map of the first scale; The initial fusion feature map of the first scale is input into the bottom-up pyramid network. An enhanced fusion layer is used to fuse the high-scale feature map after the convolution operation and the adaptive fusion feature map of multiple pre-selected intermediate-scale preliminary fusion feature maps output by the adaptive spatial feature fusion module, and output the 2nd to 3rd scales. The enhanced fusion feature map of the scale; using the The enhanced fusion layer After convolution operation on the enhanced fusion feature map of the first scale, The initial feature maps of scales are fused and output Enhanced fusion feature maps of different scales.

[0034] S225: Combine the preliminary fusion feature map of the first scale and the second to third scales The enhanced fusion feature map of the scale is used as the output of the neck network fusion feature maps of objects at different scales.

[0035] S23: Output the neck network The target fusion feature map of each scale is input into the head network of the target detection model to determine the type, quantity and position of the anatomical structure in the fetal heart section image to be detected; wherein the head network includes The detection heads of each scale are used to determine the type, quantity and location of anatomical structures in the fusion feature map of each scale target.

[0036] S3: Based on the type, number and position of the anatomical structures in the fetal heart section image to be detected, a classification algorithm is used to obtain a judgment and classification result of the fetal heart section image to be detected; wherein, based on the classification algorithm, the judgment result obtained is to determine whether the fetal heart section image to be detected is a standard section image; based on the classification algorithm, the classification result obtained is to determine the type of standard section image corresponding to the fetal heart section image to be detected.

[0037] Based on the target detection model and the classification algorithm, a multi-task fetal heart standard section detection model is constructed; the process of obtaining a fetal heart ultrasound image data set based on the multi-task fetal heart standard section detection model includes: Selecting multiple standard section images of different types from multiple fetal heart ultrasound images; wherein the standard section image types include four-chamber heart section images, three-vessel section images, three-vessel tracheal section images, left ventricular outflow tract section images, and right ventricular outflow tract section images; Performing translation, rotation, and zooming preprocessing operations on each standard section image to expand the fetal heart section image data sample, and obtaining multiple preprocessed section images corresponding to each standard section image; constructing an expanded data sample set based on all standard section images and their corresponding multiple preprocessed section images; According to the standard section requirements, the roLabelImg script is used to use a rotating rectangular box to annotate various anatomical structures of each section image in the extended data sample set, and the vertex coordinates and anatomical structure types of each rotating rectangular box corresponding to each section image are recorded through a TXT tag file; the anatomical structure types include: left ventricle, left atrium, right ventricle, right atrium, aorta, aortic arch, descending aorta, superior vena cava, pulmonary artery, ductus arteriosus, right ventricular outflow tract, trachea, and spine; Based on all the cross-sectional images in the expanded data sample set and their corresponding label files, a fetal heart ultrasound image dataset is constructed; According to a preset ratio, the fetal heart ultrasound image dataset is divided into a training set and a test set for training and testing the target detection model and classification algorithm; wherein the preset ratio is 7:3, that is, the training set accounts for 70% of the fetal heart ultrasound image dataset, and the test set accounts for 30% of the fetal heart ultrasound image dataset.

[0038] The training process of the multi-task based fetal heart standard section detection model includes: Based on the training set, the Adam back-propagation algorithm is used to iteratively optimize the multi-task fetal heart standard section detection model until the total loss function converges, and a trained multi-task fetal heart standard section detection model is obtained; The total loss function includes classification loss, intersection-over-union loss, and distribution focus loss, and its expression is: ; in, represents the total loss function; represents the classification loss; Represents the weight coefficient corresponding to the classification loss; represents the intersection-and-combination loss; Represents the weight coefficient corresponding to the intersection-over-union loss; represents the distribution focal loss; Represents the weight coefficient corresponding to the distribution focus; The expression of the classification loss is: ; The expression of the intersection-over-union loss is: ; The expression of the distribution focus loss is: ; in, represents the total number of samples; Representation sample One-hot tags, Representation sample The probability of predicting the positive class; represents the Gaussian probability intersection-over-combination ratio; Represents the true coordinate value of the sample; and are two adjacent values ​​of the bounding box position coordinates predicted by the model; and is the weight corresponding to two adjacent values ​​of the bounding box position coordinates predicted by the model.

[0039] Reference Figure 3 The network structure on the right side, the target detection model used in the present invention is to reconstruct the neck network of the YOLOv8-OBB rotation target detection model to obtain an improved target detection network; the neck network reconstruction is specifically manifested as: on the basis of the path fusion feature pyramid network (PAN) in the YOLOv8-OBB rotation target detection model, combined with the adaptive spatial feature fusion module (ASFF) that increases sampling operations and introduces small-size feature map fusion branches, a path interleaved pyramid network (Pw-FPN) based on the adaptive spatial feature fusion module is constructed as the neck network of the target detection model, and the feature map before each round of cross-level fusion of the path fusion feature pyramid network is input into the ASFF by adding an interleaved path, and then cross-level connection is performed, that is, the initial feature maps of different scales output by the bottom-up backbone network or the preliminary fused feature maps of different scales output by the top-down feature pyramid network in the path fusion feature pyramid network are input into the ASFF by adding an interleaved path, and then cross-level connection is performed; the output feature map representation of the ASFF fused output feature map in the Pw-FPN includes: Each interleaved path between the backbone network and the top-down feature pyramid consists of two C2f modules in the backbone network, an adaptive spatial feature fusion module, and a preliminary fusion layer; at this time, the output feature map of the process The expression is: ; in, Represents the output of the backbone network The initial feature map of scales; Represents the output of the backbone network The initial feature map of scales; Represents the feature map output by the two-dimensional convolution module of the current preliminary fusion layer; Represents the adaptive fusion feature map output by the current adaptive spatial feature fusion module; represents a fusion operation; Each interleaved path between the top-down feature pyramid and the bottom-up feature pyramid consists of two detail fusion enhancement modules in the top-down feature pyramid, an adaptive spatial feature fusion module, and an enhanced fusion layer; at this time, the output feature map of the process The expression is: ; in, Represents the output of the top-down pyramid network. Preliminary fusion feature maps of various scales; Represents the output of the top-down pyramid network. Preliminary fusion feature maps of various scales; Represents the feature map output by the neck convolution module of the current enhanced fusion layer; Represents the adaptive fusion feature map output by the current adaptive spatial feature fusion module; Output feature map of Pw-FPN without ASFF fusion The representation includes: The preliminary fusion feature map output by the first preliminary fusion layer and the The enhanced fusion feature maps output by the enhanced fusion layer are expressed as: and ; in, Represents the output of the backbone network The initial feature map of scales; Indicates The feature map output by the two-dimensional convolution module of the preliminary fusion layer; Represents the output of the backbone network The initial feature map of scales; Indicates The feature map output by the neck convolution module of the enhanced fusion layer.

[0040] Embodiment 2 Reference Figure 4 As shown, in a specific embodiment of the present invention, a network structure of a fetal heart standard section detection model based on multi-task is provided; wherein the number of network layers of the target detection model is set to four layers; specifically including: The backbone network includes a first backbone convolution module, a second backbone convolution module, a first C2f module, a third backbone convolution module, a second C2f module, a fourth backbone convolution module, a third C2f module, a fifth backbone convolution module, a fourth C2f module and an SPPF module connected in sequence; The neck network includes a first upsampling module, a first two-dimensional convolution module, a first detail fusion enhancement module, a second upsampling module, a second two-dimensional convolution module, a second detail fusion enhancement module, a third upsampling module, a third two-dimensional convolution module, a third detail fusion enhancement module, a first neck convolution module, a fourth detail fusion enhancement module, a second neck convolution module, a fifth detail fusion enhancement module, a third neck convolution module and a sixth detail fusion enhancement module, which are connected in sequence; and also includes a first adaptive spatial feature fusion module, a second adaptive spatial feature fusion module, a third adaptive spatial feature fusion module and a fourth adaptive spatial feature fusion module; The input of the first adaptive spatial feature fusion module is connected to the output of the second C2f module and the output of the third C2f module, and the output of the first adaptive spatial feature fusion module is connected to the input of the first detail fusion enhancement module; The input of the second adaptive spatial feature fusion module is connected to the output of the second C2f module and the output of the third C2f module, and the output of the second adaptive spatial feature fusion module is connected to the input of the second detail fusion enhancement module; The input of the third adaptive spatial feature fusion module is connected to the output of the first detail fusion enhancement module and the output of the second detail fusion enhancement module, and the output of the third adaptive spatial feature fusion module is connected to the input of the fourth detail fusion enhancement module; The input of the fourth adaptive spatial feature fusion module is connected to the output of the first detail fusion enhancement module and the output of the second detail fusion enhancement module, and the output of the fourth adaptive spatial feature fusion module is connected to the fifth detail fusion enhancement module; Among them, the input of the first upsampling module is connected to the output of the SPPF module; the input of the third detail fusion enhancement module is connected to the output of the first C2f module; the input of the sixth detail fusion enhancement module is connected to the output of the SPPF module; Among them, the first preliminary fusion layer includes a first upsampling module, a first two-dimensional convolution module and a first detail fusion enhancement module; the second preliminary fusion layer includes a second upsampling module, a second two-dimensional convolution module and a second detail fusion enhancement module; the third preliminary fusion layer includes a third upsampling module, a third two-dimensional convolution module and a third detail fusion enhancement module; the first enhanced fusion layer includes a first neck convolution module and a fourth detail fusion enhancement module; the second enhanced fusion layer includes a second neck convolution module and a fifth detail fusion enhancement module; the third enhanced fusion layer includes a third neck convolution module and a sixth detail fusion enhancement module; The head network includes detection heads of four scales connected in sequence; wherein the input of the detection head of the first scale is connected to the output of the third detail fusion enhancement module; the inputs of the detection heads of the second scale to the fourth scale are connected to the outputs of the fourth to the sixth detail fusion enhancement modules in sequence respectively.

[0041] Input the fetal heart section image to be detected into the backbone network of the target detection model, and output the initial feature maps of four scales; wherein the outputs of the first C2f module, the second C2f module and the third C2f module are used as the initial feature maps of the first to third scales output by the backbone network; and the output of the SPPF module is used as the initial feature map of the fourth scale output by the backbone network; The four-scale initial feature maps output by the backbone network are input into the neck network of the target detection model, and the target fusion feature maps of four scales are output, including: The first upsampling module and the first two-dimensional convolution module are used to sequentially perform upsampling and two-dimensional convolution operations on the initial feature map of the fourth scale, and then the initial feature map of the third scale output by the first adaptive spatial feature fusion module is fused through the first detail fusion enhancement module to obtain a preliminary fused feature map of the third scale. The first adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on the initial feature maps of the second scale and the third scale output by the backbone network, so as to obtain an adaptive fused feature map of the third scale output by the first adaptive spatial feature fusion module; at this time, the target scale and target channel number of the first adaptive spatial feature fusion module are respectively the output scale and output channel number of the first preliminary fusion layer, that is, the output scale and output channel number of the first two-dimensional convolution module; The second upsampling module and the second two-dimensional convolution module are used to sequentially perform upsampling and two-dimensional convolution operations on the preliminary fused feature map of the third scale, and then the preliminary fused feature map of the second scale output by the second adaptive spatial feature fusion module is fused through the second detail fusion enhancement module to obtain a preliminary fused feature map of the second scale. The second adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on the initial feature maps of the second scale and the third scale output by the backbone network, so as to obtain an adaptive fused feature map of the second scale output by the second adaptive spatial feature fusion module; at this time, the target scale and target channel number of the second adaptive spatial feature fusion module are respectively the output scale and output channel number of the second preliminary fusion layer, that is, the output scale and output channel number of the second two-dimensional convolution module; The preliminary fused feature map of the second scale is upsampled and convolved in turn by the third upsampling module and the third two-dimensional convolution module, and then fused with the initial feature map of the first scale output by the backbone network through the third detail fusion enhancement module to obtain the preliminary fused feature map of the first scale. After the convolution operation is performed on the preliminary fusion feature map of the first scale by using the first neck convolution module, the adaptive fusion feature map of the second scale output by the third adaptive spatial feature fusion module is fused through the fourth detail fusion enhancement module to obtain an enhanced fusion feature map of the second scale; The third adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on the preliminary fusion feature map of the second scale and the preliminary fusion feature map of the third scale, so as to obtain an adaptive fusion feature map of the second scale output by the third adaptive spatial feature fusion module; at this time, the target scale and target channel number of the third adaptive spatial feature fusion module are respectively the output scale and output channel number of the first enhanced fusion layer, that is, the output scale and output channel number of the first neck convolution module; After the second neck convolution module performs a convolution operation on the enhanced fusion feature map of the second scale, the enhanced fusion feature map of the third scale output by the fourth adaptive spatial feature fusion module is fused through the fifth detail fusion enhancement module to obtain an enhanced fusion feature map of the third scale; The fourth adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on the preliminary fusion feature map of the second scale and the preliminary fusion feature map of the third scale, so as to obtain an adaptive fusion feature map of the third scale output by the fourth adaptive spatial feature fusion module; at this time, the target scale and target channel number of the fourth adaptive spatial feature fusion module are respectively the output scale and output channel number of the second enhanced fusion layer, that is, the output scale and output channel number of the second neck convolution module; After the third neck convolution module performs a convolution operation on the enhanced fusion feature map of the third scale, it is fused with the initial feature map of the fourth scale output by the backbone network through the sixth detail fusion enhancement module to obtain an enhanced fusion feature map of the fourth scale; The preliminary fusion feature map of the first scale and the enhanced fusion feature maps of the second to fourth scales are used as the target fusion feature maps of the four scales output by the neck network; The target fusion feature maps of the four scales output by the neck network are input into the head network of the target detection model to determine the type, quantity and position of the anatomical structure in the fetal heart section image to be detected; wherein, the input of the detection head of the first scale is the target fusion feature map of the first scale; the input of the detection head of the second scale is the target fusion feature map of the second scale; the input of the detection head of the third scale is the target fusion feature map of the third scale; and the input of the detection head of the fourth scale is the target fusion feature map of the fourth scale; Based on the type, number and position of the anatomical structure in the fetal heart section image to be detected, a classification algorithm is used to obtain the discrimination and classification results of the fetal heart section image to be detected.

[0042] In a specific embodiment of the present invention, the fetal heart ultrasound image to be detected is input into a trained multi-task based fetal heart standard section detection model for detection, and the type, number and position of the anatomical structure in the fetal heart section image to be detected are detected. According to the type, number and position of the anatomical structure in the detected fetal heart section image to be detected, it is determined whether it is a standard section, and the standard four-chamber heart section, three-vessel section, three-vessel tracheal section, left ventricular outflow tract section and right ventricular outflow tract section are classified therefrom to obtain a final result image with a rotating rectangular frame and a TXT tag file, and the detection time during the detection process and the model parameters of the fetal heart standard section detection model are recorded at the same time; wherein the final result image contains multiple detection result images, each detection result corresponds to the anatomical structure recognition and classification result of a standard section image, such as Figure 5 , Figure 6 , Figure 7 , Figure 8 and Fig. 9 shown.

[0043] In summary, the present invention provides a multi-task-based fetal heart standard section detection method, which constructs a fetal heart ultrasound image dataset for model training and verification; due to the large size difference between different types of anatomical structures in fetal heart section images, a path interleaved feature pyramid (Pw-FPN) with a fusion branch of a small-size feature map is designed, and an innovative design is made for the connection method of the neck network, that is, an interleaved path is added on the basis of the path fusion feature pyramid (PAN), an adaptive spatial feature fusion module (ASFF) is introduced, and the neck network of the YOLOv8-OBB rotation target detection model is reconstructed, thereby enhancing the multi-scale target detection capability of the model. , which improves the recognition accuracy of anatomical structures in fetal heart ultrasound images; due to the problems of low contrast, signal attenuation, speckle noise and artifacts in fetal heart ultrasound images, in order to reduce the impact of noise on the detection results, a detail fusion enhancement module (DFE) with optimized feature fusion strategy is designed, so that the model can focus more on key features and reduce feature redundancy, thereby further improving the accuracy and performance of fetal heart key anatomical structure detection; based on the improved YOLOv8-OBB rotation target detection model and classification algorithm, a multi-task fetal heart standard section detection model is constructed, which can provide rich anatomical information while achieving high-precision classification, and realize the discrimination and classification of standard sections.

[0044] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A multi-task based fetal heart standard section detection method, characterized in that: include: Obtain the fetal heart section image to be detected The initial feature map of each scale is input into the neck network of the target detection model, including: The first The initial feature map of each scale is input into the top-down pyramid network. A preliminary fusion layer is used to fuse the low-scale feature map after upsampling and two-dimensional convolution operation with the adaptive fusion feature map of multiple pre-selected intermediate-scale initial feature maps output by the adaptive spatial feature fusion module, and output the second to third The initial fusion feature map of the scale; using the The initial fusion layer After upsampling and two-dimensional convolution operations, the preliminary fused feature map of the first scale is fused with the initial feature map of the first scale to output the preliminary fused feature map of the first scale; The initial fusion feature map of the first scale is input into the bottom-up pyramid network. An enhanced fusion layer is used to fuse the high-scale feature map after the convolution operation and the adaptive fusion feature map of multiple pre-selected intermediate-scale preliminary fusion feature maps output by the adaptive spatial feature fusion module, and output the 2nd to 3rd scales. The enhanced fusion feature map of the scale; using the The enhanced fusion layer After convolution operation on the enhanced fusion feature map of the first scale, The initial feature maps of scales are fused and output Enhanced fusion feature map of scales; The preliminary fusion feature map of the first scale and the second to third scales The enhanced fusion feature maps of each scale are input into the head network of the target detection model to determine the type, number and position of the anatomical structure in the fetal heart section image to be detected, and then the judgment and classification results of the fetal heart section image to be detected are obtained based on the classification algorithm.

2. A multi-task based fetal heart standard section detection method according to claim 1, characterized in that: The neck network includes: multiple adaptive spatial feature fusion modules, a top-down pyramid network and a bottom-up pyramid network; The top-down pyramid network contains A preliminary fusion layer; each preliminary fusion layer includes an upsampling module, a two-dimensional convolution module, and a detail fusion enhancement module connected in sequence; The bottom-up pyramid network contains Enhanced fusion layers; each enhanced fusion layer includes a neck convolution module and a detail fusion enhancement module connected in sequence; The adaptive spatial feature fusion module is used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale initial feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the preliminary fusion layer corresponding to the adaptive spatial feature fusion module; it is also used to perform scale adjustment, feature fusion and channel number adjustment operations on multiple pre-selected intermediate scale preliminary fusion feature maps, and output an adaptive fusion feature map with the same output scale and output channel number as the enhanced fusion layer corresponding to the adaptive spatial fusion feature module; The multiple pre-selected intermediate scale initial feature maps represent the second scale to the The initial feature map of the scale; the multiple pre-selected intermediate scale preliminary fusion feature maps represent the initial feature map of the scale from the second scale to the first scale. Preliminary fusion feature map of different scales.

3. A multi-task based fetal heart standard section detection method according to claim 2, characterized in that: The adaptive spatial feature fusion module includes a feature adjustment unit, a feature fusion unit and a feature screening unit; The feature adjustment unit is composed of a sampling operation, two convolution operations with convolution kernels of 1×1, and a splicing operation; The feature fusion unit is composed of a convolution operation with a convolution kernel of 1×1, an output channel number of 2, and an activation function of Softmax; The feature screening unit is composed of a convolution block with a convolution kernel of 1×1 and a plurality of residual blocks connected in series; each residual block is composed of a 3×3 convolution and a jump connection.

4. A multi-task based fetal heart standard section detection method according to claim 3, characterized in that: Inputting two feature maps to be processed with different scales and different numbers of channels into the adaptive spatial feature fusion module includes: Input two feature maps to be processed with different scales and different numbers of channels into the feature adjustment unit, perform sampling operation and then convolution operation on the two feature maps to be processed respectively, adjust the scales of the two feature maps to be processed to the target scale, adjust the number of channels to the same number of channels, and then splice them to output the spliced ​​feature map; The concatenated feature map is input into the feature fusion unit, and the weights corresponding to the two feature maps to be processed are obtained through convolution operation. The two feature maps to be processed and their corresponding weights are weighted summed to output a weighted fusion feature map. The weighted fusion feature map is input into the feature screening unit, and the number of channels of the weighted fusion feature map is adjusted to the target number of channels through the convolution block and the residual block, and an adaptive fusion feature map of the target scale and the target number of channels is output; Among them, the output scale and output channel number of the preliminary fusion layer or the enhanced fusion layer of each adaptive spatial feature fusion module are respectively used as the target scale and target channel number of each adaptive spatial feature fusion module.

5. The multi-task based fetal heart standard section detection method according to claim 2, characterized in that: The detail fusion enhancement module includes a channel attention mechanism unit and a feature focusing unit; The channel attention mechanism unit is composed of a splicing operation, an average pooling operation, a maximum pooling operation, multiple groups of convolution operations, a ReLU activation function, and a Sigmoid activation function; The feature focusing unit is composed of a convolution operation, a parallel convolution column and a Sigmoid activation function; there are two convolution columns in the parallel convolution column, one convolution column first performs an average pooling and then performs two convolution operations, and the other convolution column directly performs two convolution operations.

6. A multi-task based fetal heart standard section detection method according to claim 5, characterized in that: Inputting two feature maps to be fused with the same scale and the same number of channels into the detail fusion enhancement module includes: Input two feature maps to be fused of the same scale and the same number of channels into the channel attention mechanism unit, and perform a splicing operation on the two feature maps to be fused to obtain a spliced ​​fused feature map; first perform average pooling and maximum pooling operations on the spliced ​​fused feature map, respectively, and then perform multiple groups of convolution and ReLU activation function processing respectively, and finally perform addition processing to obtain the processed spliced ​​fused feature map, and then use the Sigmoid activation function to obtain the channel weights corresponding to the two feature maps to be fused; perform element-by-element multiplication of the two feature maps to be fused and their corresponding channel weights to obtain two preliminary enhanced feature maps; sum the two preliminary enhanced feature maps, and output the channel information enhanced feature map; The channel information enhanced feature map is input into the feature focusing unit. After the convolution operation and the parallel convolution column, the results obtained by the parallel convolution column are multiplied, and then the aggregation weights corresponding to the two preliminary enhanced feature maps are obtained through the Sigmoid activation function. The two preliminary enhanced feature maps and their corresponding aggregation weights are weighted summed to obtain the output feature map of the detail fusion enhancement module.

7. A multi-task based fetal heart standard section detection method according to claim 5, characterized in that: forward The preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ； in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current preliminary fusion layer; Represents the feature map output by the two-dimensional convolution module of the current preliminary fusion layer; the output scale and number of output channels of each preliminary fusion layer are the same as the output scale and number of output channels of the two-dimensional convolution module of the preliminary fusion layer; No. The first scale preliminary fusion feature map output by the detail fusion enhancement module of the preliminary fusion layer The expression is: ； in, Indicates The initial feature map of scales; Indicates The feature map output by the two-dimensional convolution module of the preliminary fusion layer; forward Enhanced fusion feature map output by the detail fusion enhancement module of the enhanced fusion layer The expression is: ； in, Represents the adaptive fusion feature map output by the adaptive spatial feature fusion module with the same output scale and output channel number as the current enhanced fusion layer; Represents the feature map output by the neck convolution module of the current enhanced fusion layer; the output scale and number of output channels of each enhanced fusion layer are the same as the output scale and number of output channels of the neck convolution module of the enhanced fusion layer; No. The output of the detail fusion enhancement module of the enhanced fusion layer is Scale enhanced fusion feature map The expression is: ； in, Indicates The initial feature map of scales; Indicates The feature map output by the neck convolution module of the enhanced fusion layer; Represents the channel attention mechanism; Represents feature aggregation; Represents a concatenation operation.

8. The multi-task based fetal heart standard section detection method according to claim 1, characterized in that: The step of obtaining the heart section image of the fetus to be detected is: The initial feature maps of each scale include: From the fetal heart ultrasound image, obtain the fetal heart section image to be detected, input it into the backbone network of the target detection model, and output the fetal heart section image to be detected. The initial feature map of scales; The backbone network includes multiple backbone convolution modules, multiple C2f modules and SPPF modules; Using each C2f module, the feature map output by the previous trunk convolution module is split, spliced ​​and fused to obtain the first to the The C2f module outputs feature maps of scales; The C2f module outputs feature maps of each scale as the first to the second output of the backbone network. The initial feature map of scales; Using the SPPF module, The C2f module outputs the feature map of the scale for pooling operation and outputs the backbone network The initial feature map of the scale.

9. The method for detecting standard sections of a fetal heart based on multiple tasks according to claim 1, characterized in that: Also includes: Selecting multiple standard section images of different types from multiple fetal heart ultrasound images; wherein the standard section image types include four-chamber heart section images, three-vessel section images, three-vessel tracheal section images, left ventricular outflow tract section images, and right ventricular outflow tract section images; Perform translation, rotation, and zoom-in / out preprocessing operations on each standard section image to obtain multiple preprocessed section images corresponding to each standard section image; construct an extended data sample set based on all standard section images and their corresponding multiple preprocessed section images; According to the standard section requirements, the roLabelImg script is used to use a rotating rectangular box to annotate various anatomical structures of each section image in the extended data sample set, and the vertex coordinates and anatomical structure types of each rotating rectangular box corresponding to each section image are recorded through a label file; the anatomical structure types include: left ventricle, left atrium, right ventricle, right atrium, aorta, aortic arch, descending aorta, superior vena cava, pulmonary artery, ductus arteriosus, right ventricular outflow tract, trachea, and spine; Based on all the cross-sectional images in the expanded data sample set and their corresponding label files, a fetal heart ultrasound image dataset is constructed; According to a preset ratio, the fetal heart ultrasound image dataset is divided into a training set and a test set for training and testing the target detection model and the classification algorithm; wherein, based on the classification algorithm, the judgment result obtained is to judge whether the fetal heart section image to be detected is a standard section image; based on the classification algorithm, the classification result obtained is to determine the standard section image type corresponding to the fetal heart section image to be detected.

10. A multi-task based fetal heart standard section detection method according to claim 9, characterized in that: Based on the target detection model and the classification algorithm, a multi-task based fetal heart standard section detection model is constructed. The training process of the multi-task based fetal heart standard section detection model includes: Based on the training set, the Adam back-propagation algorithm is used to iteratively optimize the multi-task fetal heart standard section detection model until the total loss function converges, and a trained multi-task fetal heart standard section detection model is obtained; The total loss function consists of classification loss, intersection-over-union loss, and distribution focus loss, and its expression is: ； in, represents the total loss function; represents the classification loss; Represents the weight coefficient corresponding to the classification loss; represents the intersection-and-combination loss; Represents the weight coefficient corresponding to the intersection-over-union loss; represents the distribution focal loss; Represents the weight coefficient corresponding to the distribution focus; The expression of the classification loss is: ； The expression of the intersection-over-union loss is: ； The expression of the distribution focus loss is: ； in, represents the total number of samples; Representation sample One-hot tags, Representation sample The probability of predicting the positive class; is the Gaussian probability intersection and union ratio; is the true coordinate value of the sample; and are two adjacent values ​​of the bounding box position coordinates predicted by the model; and is the weight corresponding to two adjacent values ​​of the bounding box position coordinates predicted by the model.

Images