Fetal craniocerebral standard section detection method and system

Abstract

The invention relates to a fetal craniocerebral standard section detection method and system, and the method comprises the steps: S1, obtaining a fetal craniocerebral ultrasonic image; s2, a fetal brain standard section detection network model is constructed based on a YOLOv7 model, the fetal brain standard section detection network model comprises a backbone network, a neck network and a head network which are connected in sequence, the backbone network and the neck network are improved, the backbone network is used for extracting image features, and the head network is used for extracting the image features; the neck network is used for carrying out feature fusion on the image features extracted by the backbone network, and the head network is used for detecting the fusion features obtained by the neck network; and S3, identifying whether the fetal craniocerebral ultrasonic image is a standard section image or not through the fetal craniocerebral standard section detection network model. According to the invention, the standard section image of the fetal brain can be effectively detected.

Description

Technical Field

[0001] The present invention relates to the technical field of medical ultrasonic image processing, and in particular to a method and system for detecting standard sections of a fetal cranium. Background Art

[0002] As a non-invasive, radiation-free, convenient and dynamic observation method, ultrasound has become an indispensable tool for clinical diagnosis and is widely used in prenatal screening around the world. In current clinical diagnosis, medical ultrasound has been widely used in various professional departments, such as electrocardiogram, breast ultrasound, abdominal ultrasound, transrectal ultrasound, cardiovascular ultrasound and prenatal diagnostic ultrasound, especially in obstetrics and gynecology. In prenatal testing in obstetrics and gynecology, ultrasound doctors use ultrasound imaging technology to obtain the corresponding standard sections of the fetal brain, evaluate the fetal development, screen for fetal malformations, and ensure the healthy growth of the fetus.

[0003] However, obtaining standard fetal cranial sections is highly dependent on the clinical experience and level of ultrasound doctors. Ultrasound doctors need to repeatedly screen and select standard sections. As time goes by and the workload increases, the efficiency and accuracy will gradually decrease, and it is extremely challenging for novice doctors. In addition, manual detection is highly subjective, resulting in large differences in the degree of standardization of fetal cranial ultrasound images, which seriously affects the results of fetal ultrasound detection and diagnosis.

[0004] With the rapid development of deep learning, deep convolutional neural networks have made it possible to accurately detect fetal cranial sections. More and more researchers are using deep learning technology to solve the problem of fetal cranial ultrasound image detection. The detection models proposed by many researchers can accurately identify key anatomical structures from fetal cranial ultrasound images, effectively improving the accuracy of recognition. However, fetal cranial ultrasound images have multiple sections with similar internal structures and multiple key anatomical structures. However, there are still certain difficulties in effectively identifying multiple different key anatomical structures.

[0005] Therefore, accurately and effectively detecting the three standard sections from fetal cranial brain images is an urgent problem to be solved. Summary of the invention

[0006] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the standard cross-sectional images of the fetal brain cannot be accurately and effectively detected in the prior art.

[0007] In order to solve the above technical problems, the present invention provides a method for detecting standard sections of fetal cranial brain, comprising:

[0008] Step S1: Acquire a fetal cranial ultrasound image;

[0009] Step S2: constructing a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network;

[0010] An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy;

[0011] Step S3: identifying whether the fetal cranial ultrasound image is a standard section image by using the fetal cranial standard section detection network model, wherein the standard section images include a standard thalamus section image, a standard lateral ventricle section image, and a standard cerebellum section image;

[0012] Among them, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure. The first unique anatomical structure includes the cerebral peduncle and the choroid plexus, the second unique anatomical structure includes the posterior peduncle of the lateral ventricle and the choroid plexus, the third unique anatomical structure includes the cerebellum and the posterior cranial fossa cistern, and the common anatomical structures include the thalamus, the falx cerebri, the cavum septum pellucidum, the lateral fissure, and the skull halo.

[0013] In one embodiment of the present invention, the backbone network includes a first CBS module, a second CBS module, a third CBS module, a fourth CBS module, a first AK-ELAN module, a first MP-1 module, a second AK-ELAN module, a second MP-1 module, a third AK-ELAN module, a third MP-1 module, a fourth AK-ELAN module, and an EMA module connected in sequence;

[0014] The first AK-ELAN module, the second AK-ELAN module, the third AK-ELAN module and the fourth AK-ELAN module each include a first branch and a second branch, the first branch includes a fifth CBS module, the second branch includes a sixth CBS module, a seventh CBS module, a first AKConv, an eighth CBS module and a second AKConv connected in sequence, the fifth CBS module and the second AKConv are connected to the first Concat layer, and the output of the sixth CBS module and the output of the first AKConv are also connected to the first Concat layer;

[0015] The first MP-1 module, the second MP-1 module, and the third MP-1 module all include a third branch and a fourth branch, the third branch includes a first MaxPool layer and a ninth CBS module connected in sequence, the fourth branch includes a tenth CBS module and an eleventh CBS module connected in sequence, and the ninth CBS module and the eleventh CBS module are both connected to the second Concat layer.

[0016] In one embodiment of the present invention, the neck network includes a SPPCSPC module, a twelfth CBS module, a first UpSample layer, a third Concat layer, a first ELAN-2 module, a thirteenth CBS module, a second UpSample layer, a fourth Concat layer, a second ELAN-2 module, a first MP-2 module, a fifth Concat layer, a third ELAN-2 module, a second MP-2 module, a sixth Concat layer, and a fourth ELAN-2 module connected in sequence;

[0017] The fourth ELAN-2 module is also connected to the twelfth CBS module to implement a recursive feature pyramid RFP operation;

[0018] The SPPCSPC module is also connected to the sixth Concat layer;

[0019] The third AK-ELAN module is also connected to the third Concat layer through the fourteenth CBS module, and the output of the third ELAN-2 module is also connected to the input of the third Concat layer to implement a recursive feature pyramid RFP operation;

[0020] The second AK-ELAN module is also connected to the fourth Concat layer through the fifteenth CBS module, and the output of the second ELAN-2 module is also connected to the input of the fourth Concat layer to implement a recursive feature pyramid RFP operation;

[0021] The first ELAN-2 module and the second ELAN-2 module each include a fifth branch and a sixth branch, the fifth branch includes a sixteenth CBS module, the sixth branch includes a seventeenth CBS module, an eighteenth CBS module, a nineteenth CBS module, a twentieth CBS module and a twenty-first CBS module connected in sequence, the sixteenth CBS module and the twenty-first CBS module are connected to a seventh Concat layer, and the output of the seventeenth CBS module and the output of the nineteenth CBS module are also connected to the seventh Concat layer;

[0022] The first MP-2 module and the second MP-2 module both include a seventh branch and an eighth branch, the seventh branch includes a second MaxPool layer and a twenty-second CBS module connected in sequence, the eighth branch includes a twenty-third CBS module and a twenty-fourth CBS module connected in sequence, and the twenty-second CBS module and the twenty-fourth CBS module are both connected to the eighth Concat layer;

[0023] The SPPCSPC module includes a twenty-fifth CBS module, a twenty-sixth CBS module, and a twenty-seventh CBS module connected in sequence, the twenty-seventh CBS module is connected to the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer respectively, the outputs of the twenty-seventh CBS module, the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer are connected to the twenty-eighth CBS module and the twenty-ninth CBS module through the ninth Concat layer, the output of the twenty-fifth CBS module passes through the thirtieth CBS module, and then passes through the tenth Concat layer together with the output of the twenty-ninth CBS module and is connected to the thirty-first CBS module;

[0024] The second ELAN-2 module, the third ELAN-2 module and the fourth ELAN-2 module input the first output and the second output into the Fusion module for feature fusion at each iteration of the neck network, and the Fusion module performs feature fusion through 1*1 convolution and activation function.

[0025] In one embodiment of the present invention, the first to thirty-first CBS modules are represented as follows:

[0026] CBS(x)=Relu(BatchNorm(Conv(x)))

[0027] Among them, x represents the input feature, Conv(·) represents the convolution symbol, BatchNorm(·) represents batch normalization, and Relu(·) represents the Relu activation function.

[0028] In one embodiment of the present invention, the EMA module includes feature grouping, parallel sub-networks and cross-space learning, wherein:

[0029] The feature grouping is specifically as follows: the input feature is divided into G sub-features according to the channel dimension direction to learn different feature information and strengthen the feature representation of the region of interest in each sub-feature;

[0030] The parallel subnetwork is specifically composed of two 1×1 branches and one 3×3 branch; two global average pooling operations are used in the 1×1 branch to encode the channels along the X and Y directions respectively, to capture the long-range dependencies across channels, and to obtain channel information and directional position information; in the 3×3 branch, 3×3 convolutions are used to perform cross-channel interaction of local features to enrich the feature space;

[0031] The cross-space learning specifically includes: fusing global position information with spatial information of different scales to improve the feature extraction capability of the network.

[0032] In one embodiment of the present invention, step S2 further includes training the fetal cranial standard section detection network model using a total loss function, and judging whether the training of the fetal cranial standard section detection network model is completed according to whether the total loss function converges, wherein the total loss function is expressed as:

[0033] L total =λ1L box +λ2L obj +λ3L cls +λ4L cls_plane

[0034] Among them, L box represents the bounding box localization loss, L obj represents the target confidence loss, L cls represents the classification loss, L cls_plane represents the standard section target category loss function, and λ1, λ2, λ3, and λ4 represent different weight coefficients.

[0035] In one embodiment of the present invention, the standard section target category loss function L cls_plane It is expressed as:

[0036]

[0037] Where n represents the number of images, N k Indicates the number of identifications of the first, second, and third unique anatomical structures, N Fk represents the number of errors in identifying the first, second, and third unique anatomical structures, N represents the number of all anatomical structures identified, and N Ftrepresents the number of all anatomical errors, and α and β represent different weight coefficients.

[0038] In order to solve the above technical problems, the present invention provides a fetal cranial standard section detection system, comprising:

[0039] Acquisition module: used to obtain fetal cranial ultrasound images;

[0040] Construction module: used to construct a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network;

[0041] An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy;

[0042] Detection module: used for identifying whether the fetal cranial ultrasound image is a standard section image through the fetal cranial standard section detection network model, wherein the standard section image includes a standard thalamus section image, a standard lateral ventricle section image and a standard cerebellum section image;

[0043] Among them, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure. The first unique anatomical structure includes the cerebral peduncle and the choroid plexus, the second unique anatomical structure includes the posterior peduncle of the lateral ventricle and the choroid plexus, the third unique anatomical structure includes the cerebellum and the posterior cranial fossa cistern, and the common anatomical structures include the thalamus, the falx cerebri, the cavum septum pellucidum, the lateral fissure, and the skull halo.

[0044] To solve the above technical problems, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned fetal cranial standard section detection method when executing the computer program.

[0045] In order to solve the above technical problems, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting standard sections of the fetal skull and brain are implemented.

[0046] The above technical solution of the present invention has the following advantages compared with the prior art:

[0047] The present invention is based on the YOLOv7 model, introduces and designs AK-ELAN module and EMA module in the backbone network, and adopts recursive feature pyramid RFP network instead of feature pyramid network FPN in the neck network module; specifically, AK-ELAN combines the convolution kernel AKConv with arbitrary sampling shape and arbitrary parameters, so that the backbone network pays attention to the target features with changing size and shape; the EMA module consists of three parts: feature grouping, parallel sub-network and cross-space learning, and improves the feature extraction ability of the network by learning the global position information of the target features and spatial information of different scales and effectively fusing them; RFP is built on FPN, and further improves the recognition accuracy of key anatomical structures by merging additional feedback connections from the FPN layer into the top-down input features;

[0048] The present invention also constructs a standard section target category loss function to improve the effectiveness of the fetal cranial standard section detection network model training, and further improve the recognition accuracy of unique anatomical structures and common anatomical structures;

[0049] The present invention effectively improves the recognition accuracy of key anatomical structures in fetal cranial ultrasound images, can assist doctors in performing effective work, and is suitable for large-scale promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 It is a structural diagram of a fetal cranial standard section detection network model constructed based on the YOLOv7 model in an embodiment of the present invention;

[0053] Figure 3 : is a schematic diagram of the structure of the AK-ELAN module in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the structure of the ELAN-2 module in an embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the structure of the SPPCSPC module in an embodiment of the present invention;

[0056] Figure 6 It is a schematic diagram of the structure of the MP-1 module and the MP-2 module in an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of the anatomical structure recognition and classification results of the standard fetal thalamus section in an embodiment of the present invention;

[0058] Figure 8 is a schematic diagram of the anatomical structure recognition and classification results of a standard fetal lateral ventricle section in an embodiment of the present invention;

[0059] Fig. 9 It is a schematic diagram of the anatomical structure recognition and classification results of the standard fetal cerebellum section in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0061] Embodiment 1

[0062] Reference Figure 1 As shown, the present invention relates to a method for detecting standard sections of fetal cranial brain, comprising:

[0063] Step S1: Acquire a fetal cranial ultrasound image;

[0064] Step S2: constructing a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network;

[0065] An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy;

[0066] Step S3: Identify whether the fetal cranial ultrasound image is a standard section image through the fetal cranial standard section detection network model, and the standard section images include a standard thalamus section image, a standard lateral ventricle section image, and a standard cerebellum section image; wherein, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure, the first unique anatomical structure includes a cerebral peduncle and a choroid plexus, the second unique anatomical structure includes a posterior peduncle of the lateral ventricle and a choroid plexus, the third unique anatomical structure includes a cerebellum and a posterior cranial fossa, and the common anatomical structure includes a thalamus, a falx cerebri, a cavum septum pellucidum, a sylvian fissure, and a skull halo.

[0067] Furthermore, the step S1 is specifically as follows: a large number of fetal cranial ultrasound images are acquired, and data preprocessing is performed, and annotated according to the requirements of each standard section image to generate a fetal cranial ultrasound image data set, as follows:

[0068] A large number of fetal cranial ultrasound images are obtained, and standard fetal thalamus, lateral ventricle and cerebellum section images are selected from them. After each standard section image is cropped to the same size, pre-processing such as translation, rotation and deformation is performed to expand the fetal cranial image data sample.

[0069] For further information, see Figure 2 , the backbone network includes a first CBS module, a second CBS module, a third CBS module, a fourth CBS module, a first AK-ELAN module, a first MP-1 module, a second AK-ELAN module, a second MP-1 module, a third AK-ELAN module, a third MP-1 module, a fourth AK-ELAN module and an EMA module connected in sequence;

[0070] See also Figure 3 , the first AK-ELAN module, the second AK-ELAN module, the third AK-ELAN module and the fourth AK-ELAN module all include a first branch and a second branch, the first branch includes a fifth CBS module, the second branch includes a sixth CBS module, a seventh CBS module, a first AKConv, an eighth CBS module and a second AKConv connected in sequence, the fifth CBS module and the second AKConv are connected to the first Concat layer, and the output of the sixth CBS module and the output of the first AKConv are also connected to the first Concat layer;

[0071] See also Figure 6The first MP-1 module, the second MP-1 module, and the third MP-1 module all include a third branch and a fourth branch, the third branch includes a first MaxPool layer and a ninth CBS module connected in sequence, the fourth branch includes a tenth CBS module and an eleventh CBS module connected in sequence, and the ninth CBS module and the eleventh CBS module are both connected to the second Concat layer.

[0072] Further, the neck network includes a SPPCSPC module, a twelfth CBS module, a first UpSample layer, a third Concat layer, a first ELAN-2 module, a thirteenth CBS module, a second UpSample layer, a fourth Concat layer, a second ELAN-2 module, a first MP-2 module, a fifth Concat layer, a third ELAN-2 module, a second MP-2 module, a sixth Concat layer, and a fourth ELAN-2 module, which are connected in sequence;

[0073] The fourth ELAN-2 module is also connected to the twelfth CBS module to implement a recursive feature pyramid RFP operation;

[0074] The SPPCSPC module is also connected to the sixth Concat layer;

[0075] The third AK-ELAN module is also connected to the third Concat layer through the fourteenth CBS module, and the output of the third ELAN-2 module is also connected to the input of the third Concat layer to implement a recursive feature pyramid RFP operation;

[0076] The second AK-ELAN module is also connected to the fourth Concat layer through the fifteenth CBS module, and the output of the second ELAN-2 module is also connected to the input of the fourth Concat layer to implement a recursive feature pyramid RFP operation.

[0077] See also Figure 4 , the first ELAN-2 module and the second ELAN-2 module both include a fifth branch and a sixth branch, the fifth branch includes a sixteenth CBS module, the sixth branch includes a seventeenth CBS module, an eighteenth CBS module, a nineteenth CBS module, a twentieth CBS module and a twenty-first CBS module connected in sequence, the sixteenth CBS module and the twenty-first CBS module are connected to the seventh Concat layer, and the output of the seventeenth CBS module and the output of the nineteenth CBS module are also connected to the seventh Concat layer.

[0078] See also Figure 6The first MP-2 module and the second MP-2 module both include a seventh branch and an eighth branch, the seventh branch includes a second MaxPool layer and a twenty-second CBS module connected in sequence, the eighth branch includes a twenty-third CBS module and a twenty-fourth CBS module connected in sequence, and the twenty-second CBS module and the twenty-fourth CBS module are both connected to the eighth Concat layer.

[0079] It should be noted that although the MP-2 module and the MP-1 module in this embodiment have the same network structure, their parameters are different, so this embodiment describes the MP-2 module and the MP-1 module separately.

[0080] See also Figure 5 The SPPCSPC module includes a twenty-fifth CBS module, a twenty-sixth CBS module, and a twenty-seventh CBS module connected in sequence. The twenty-seventh CBS module is connected to the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer respectively. The outputs of the twenty-seventh CBS module, the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer are connected to the twenty-eighth CBS module and the twenty-ninth CBS module through the ninth Concat layer. The output of the twenty-fifth CBS module passes through the thirtieth CBS module, and then passes through the tenth Concat layer together with the output of the twenty-ninth CBS module and is connected to the thirty-first CBS module.

[0081] The second ELAN-2 module, the third ELAN-2 module, and the fourth ELAN-2 module input the first output and the second output into the Fusion module at each iteration of the neck network ( Figure 2 The Fusion module performs feature fusion by 1*1 convolution and activation function. It should be noted that: Figure 2 The solid lines of the outputs of the second ELAN-2 module, the third ELAN-2 module and the fourth ELAN-2 module connected to the Fusion module represent the first output of each ELAN-2 module, and the dotted lines represent the second output of each ELAN-2 module.

[0082] The features output by the Fusion modules corresponding to the second ELAN-2 module, the third ELAN-2 module and the fourth ELAN-2 module are all input into the three detection heads (detection head 1, detection head 2, detection head 3) of the head network.

[0083] Further, the first CBS module to the thirty-first CBS module are represented as:

[0084] CBS(x)=Relu(BatchNorm(Conv(x)))

[0085] Among them, x represents the input feature, Conv(·) represents the convolution symbol, BatchNorm(·) represents batch normalization, and Relu(·) represents the Relu activation function.

[0086] Furthermore, the EMA module is an efficient multi-scale attention mechanism capable of cross-space learning, which consists of three parts: feature grouping, parallel sub-networks, and cross-space learning;

[0087] The feature grouping refers to dividing the input feature into G sub-features in the channel dimension direction to learn different feature information and strengthen the feature representation of the region of interest in each sub-feature;

[0088] The parallel subnetwork is composed of two 1×1 branches and one 3×3 branch. In the 1×1 branch, two global average pooling operations are used to encode the channels along the X and Y directions respectively, to capture the long-range dependencies across channels, and to obtain channel information and directional position information. In the 3×3 branch, 3×3 convolution is used to perform cross-channel interaction of local features to enrich the feature space.

[0089] The cross-space learning effectively integrates global position information and spatial information of different scales, so that the network not only considers the long-range dependency of target features when extracting features, but also obtains rich spatial feature information at the same time, thereby improving the feature extraction capability of the network.

[0090] Furthermore, the recursive feature pyramid network RFP is built on the feature pyramid network FPN. By incorporating additional feedback connections from the FPN layer into the top-down input features, the output features of RFP are expressed as:

[0091] f i =F i (f i+1 , x i ),x i =B i (x i-1 ,R i (f i ))

[0092] Among them, x i represents the features of the i-th layer, F i represents the i-th bottom-up FPN operation, B i represents the i-th stage of the top-down backbone network, R i Represents features that are recursively fed back into the top-down input features.

[0093] Furthermore, the head network is consistent with the head network of the YOLOv7 model, and the key anatomical structures in the fetal brain are predicted by feature maps of 20×20 (corresponding to detection head 1), 40×40 (corresponding to detection head 2) and 80×80 (corresponding to detection head 3). Among them, the 20×20 feature map is used to detect large-scale key anatomical structures in the fetal brain. The 40×40 feature map is used to detect medium-sized targets. Compared with the 20×20 and 40×40 feature maps, the 80×80 feature map is larger, but the perception range is smaller, and it contains more position information, so it is more suitable for detecting smaller key anatomical structures in the fetal brain. Since the head network belongs to the prior art, it will not be repeated in this embodiment.

[0094] Furthermore, step S2 also includes training the fetal cranial standard section detection network model using a total loss function, and judging whether the training of the fetal cranial standard section detection network model is completed according to whether the total loss function converges. If the total loss function converges, the training is completed, wherein the total loss function is expressed as:

[0095] L tota =λ1L box +λ2L obj +λ3L cls +λ4L cls_plane

[0096] Among them, L box represents the bounding box positioning loss (this loss measures the difference between the bounding box predicted by the model and the true bounding box, and CIoU loss can be used); L obj represents the target confidence loss (this loss measures the accuracy of the model's confidence prediction of whether the bounding box contains the target object, usually using binary cross entropy loss); L cls represents classification loss (classification loss measures the accuracy of the model in classifying the object, usually using binary cross entropy loss); L cls_plane represents the standard section target category loss function, and λ1, λ2, λ3, and λ4 represent different weight coefficients.

[0097] Since the bounding box positioning loss, target confidence loss and classification loss are all existing technologies, they are not described in detail in this embodiment.

[0098] Furthermore, when training the fetal cranial standard section detection network model, this embodiment uses the Adam back-propagation algorithm based on the training set to iteratively optimize the fetal cranial standard section detection network model until the total loss function converges, thereby obtaining the trained fetal cranial standard section detection network model.

[0099] Furthermore, the standard section target category loss function L cls_plane It is expressed as:

[0100]

[0101] Where n represents the number of images, N k Indicates the number of identifications of the first, second, and third unique anatomical structures, N Fk represents the number of errors in identifying the first, second, and third unique anatomical structures, N represents the number of all anatomical structures identified, and N Ft represents the number of all anatomical errors, and α and β represent different weight coefficients.

[0102] The standard section target category loss function L constructed in this embodiment cls_plane It can enable the fetal cranial standard section detection network model to achieve more accurate recognition of standard thalamus section images, standard lateral ventricle section images, and standard cerebellum section images.

[0103] Figure 7 The results of the identification and classification of the anatomical structure of the standard fetal thalamus section are shown. Figure 7 The area where the number 1 is located represents the falx cerebri. Figure 7 The area where the number 2 is located represents the thalamus. Figure 7 The area where the number 3 is located represents the cerebral foot. Figure 7 The area where the number 4 is located represents the cavum septum pellucidum. Figure 7 The area where the number 5 is located represents the Sylvian fissure.

[0104] Figure 8 The results of the anatomical structure recognition and classification of the standard fetal lateral ventricle section are shown. Figure 8 The area where the number 1 is located represents the falx cerebri. Figure 8 The area where the number 2 is located represents the thalamus. Figure 8 The area where the number 3 is located represents the choroid plexus. Figure 8 The area where the number 4 is located represents the cavum septum pellucidum. Figure 8 The area where the number 5 is located represents the lateral fissure of the brain. Figure 8 The area where the number 6 is located represents the posterior peduncle of the lateral ventricle.

[0105] Fig. 9 The results of the identification and classification of the anatomical structure of the standard fetal cerebellum section are shown. Fig. 9 The area where the number 1 is located represents the falx cerebri. Fig. 9 The area where the number 2 is located represents the thalamus. Fig. 9 The area where the number 3 is located represents the posterior cranial fossa. Fig. 9 The area where the number 4 is located represents the cavum septum pellucidum. Fig. 9 The area where the number 5 is located represents the lateral fissure of the brain. Fig. 9The area where the number 6 is located represents the cerebellum.

[0106] Embodiment 2

[0107] This embodiment provides a fetal cranial standard section detection system, including:

[0108] Construction module: used to construct a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network;

[0109] An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy;

[0110] Detection module: used for identifying whether the fetal cranial ultrasound image is a standard section image through the fetal cranial standard section detection network model, wherein the standard section image includes a standard thalamus section image, a standard lateral ventricle section image and a standard cerebellum section image;

[0111] Among them, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure. The first unique anatomical structure includes the cerebral peduncle and the choroid plexus, the second unique anatomical structure includes the posterior peduncle of the lateral ventricle and the choroid plexus, the third unique anatomical structure includes the cerebellum and the posterior cranial fossa cistern, and the common anatomical structures include the thalamus, the falx cerebri, the cavum septum pellucidum, the lateral fissure, and the skull halo.

[0112] Embodiment 3

[0113] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for detecting standard sections of the fetal skull and brain described in Embodiment 1 are implemented.

[0114] Embodiment 4

[0115] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for detecting standard sections of the fetal cranium and brain described in Embodiment 1 are implemented.

[0116] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0121] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for detecting standard sections of fetal brain, characterized in that: include: Step S1: Acquire a fetal cranial ultrasound image; Step S2: constructing a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network; An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy; Step S3: identifying whether the fetal cranial ultrasound image is a standard section image by using the fetal cranial standard section detection network model, wherein the standard section images include a standard thalamus section image, a standard lateral ventricle section image, and a standard cerebellum section image; Among them, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure. The first unique anatomical structure includes the cerebral peduncle and the choroid plexus, the second unique anatomical structure includes the posterior peduncle of the lateral ventricle and the choroid plexus, the third unique anatomical structure includes the cerebellum and the posterior cranial fossa cistern, and the common anatomical structures include the thalamus, the falx cerebri, the cavum septum pellucidum, the lateral fissure, and the skull halo.

2. The method for detecting standard sections of the fetal brain according to claim 1, characterized in that: The backbone network includes a first CBS module, a second CBS module, a third CBS module, a fourth CBS module, a first AK-ELAN module, a first MP-1 module, a second AK-ELAN module, a second MP-1 module, a third AK-ELAN module, a third MP-1 module, a fourth AK-ELAN module and an EMA module connected in sequence; The first AK-ELAN module, the second AK-ELAN module, the third AK-ELAN module and the fourth AK-ELAN module each include a first branch and a second branch, the first branch includes a fifth CBS module, the second branch includes a sixth CBS module, a seventh CBS module, a first AKConv, an eighth CBS module and a second AKConv connected in sequence, the fifth CBS module and the second AKConv are connected to the first Concat layer, and the output of the sixth CBS module and the output of the first AKConv are also connected to the first Concat layer; The first MP-1 module, the second MP-1 module, and the third MP-1 module all include a third branch and a fourth branch, the third branch includes a first MaxPool layer and a ninth CBS module connected in sequence, the fourth branch includes a tenth CBS module and an eleventh CBS module connected in sequence, and the ninth CBS module and the eleventh CBS module are both connected to the second Concat layer.

3. The method for detecting standard sections of the fetal brain according to claim 2, characterized in that: The neck network includes a SPPCSPC module, a twelfth CBS module, a first UpSample layer, a third Concat layer, a first ELAN-2 module, a thirteenth CBS module, a second UpSample layer, a fourth Concat layer, a second ELAN-2 module, a first MP-2 module, a fifth Concat layer, a third ELAN-2 module, a second MP-2 module, a sixth Concat layer, and a fourth ELAN-2 module connected in sequence; The fourth ELAN-2 module is also connected to the twelfth CBS module to implement a recursive feature pyramid RFP operation; The SPPCSPC module is also connected to the sixth Concat layer; The third AK-ELAN module is also connected to the third Concat layer through the fourteenth CBS module, and the output of the third ELAN-2 module is also connected to the input of the third Concat layer to implement a recursive feature pyramid RFP operation; The second AK-ELAN module is also connected to the fourth Concat layer through the fifteenth CBS module, and the output of the second ELAN-2 module is also connected to the input of the fourth Concat layer to implement a recursive feature pyramid RFP operation; The first ELAN-2 module and the second ELAN-2 module each include a fifth branch and a sixth branch, the fifth branch includes a sixteenth CBS module, the sixth branch includes a seventeenth CBS module, an eighteenth CBS module, a nineteenth CBS module, a twentieth CBS module and a twenty-first CBS module connected in sequence, the sixteenth CBS module and the twenty-first CBS module are connected to a seventh Concat layer, and the output of the seventeenth CBS module and the output of the nineteenth CBS module are also connected to the seventh Concat layer; The first MP-2 module and the second MP-2 module both include a seventh branch and an eighth branch, the seventh branch includes a second MaxPool layer and a twenty-second CBS module connected in sequence, the eighth branch includes a twenty-third CBS module and a twenty-fourth CBS module connected in sequence, and the twenty-second CBS module and the twenty-fourth CBS module are both connected to the eighth Concat layer; The SPPCSPC module includes a twenty-fifth CBS module, a twenty-sixth CBS module, and a twenty-seventh CBS module connected in sequence, the twenty-seventh CBS module is connected to the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer respectively, the outputs of the twenty-seventh CBS module, the third MaxPool layer, the fourth MaxPool layer, and the fifth MaxPool layer are connected to the twenty-eighth CBS module and the twenty-ninth CBS module through the ninth Concat layer, the output of the twenty-fifth CBS module passes through the thirtieth CBS module, and then passes through the tenth Concat layer together with the output of the twenty-ninth CBS module and is connected to the thirty-first CBS module; The second ELAN-2 module, the third ELAN-2 module and the fourth ELAN-2 module input the first output and the second output into the Fusion module for feature fusion at each iteration of the neck network, and the Fusion module performs feature fusion through 1*1 convolution and activation function.

4. The method for detecting standard sections of the fetal brain according to claim 3, characterized in that: The first to thirty-first CBS modules are represented as follows: CBS(x)=Relu(BatchNorm(Conv(x))) Among them, x represents the input feature, Conv(·) represents the convolution symbol, BatchNorm(·) represents batch normalization, and Relu(·) represents the Relu activation function.

5. The method for detecting standard sections of the fetal brain according to claim 3, characterized in that: The EMA module includes feature grouping, parallel sub-networks and cross-space learning, where: The feature grouping is specifically as follows: the input feature is divided into G sub-features according to the channel dimension direction to learn different feature information and strengthen the feature representation of the region of interest in each sub-feature; The parallel subnetwork is specifically composed of two 1×1 branches and one 3×3 branch; two global average pooling operations are used in the 1×1 branch to encode the channels along the X and Y directions respectively, to capture the long-range dependencies across channels, and to obtain channel information and directional position information; in the 3×3 branch, 3×3 convolutions are used to perform cross-channel interaction of local features to enrich the feature space; The cross-space learning specifically includes: fusing global position information with spatial information of different scales to improve the feature extraction capability of the network.

6. The method for detecting standard sections of the fetal brain according to claim 1, characterized in that: The step S2 also includes training the fetal cranial standard section detection network model using a total loss function, and judging whether the training of the fetal cranial standard section detection network model is completed according to whether the total loss function converges, wherein the total loss function is expressed as: L total =λ1L box +λ2L obj +λ3L cls +λ4L cls_plane Among them, L box represents the bounding box localization loss, L obj represents the target confidence loss, L cls represents the classification loss, L cls_plane represents the standard section target category loss function, and λ1, λ2, λ3, and λ4 represent different weight coefficients.

7. The method for detecting standard sections of the fetal brain according to claim 6, characterized in that: The standard section target category loss function L cls_plane It is expressed as: Where n represents the number of images, N k Indicates the number of identifications of the first, second, and third unique anatomical structures, N Fk represents the number of errors in identifying the first, second, and third unique anatomical structures, N represents the number of all anatomical structures identified, and N Ft represents the number of all anatomical errors, and α and β represent different weight coefficients.

8. A fetal cranial standard section detection system, characterized by: include: Acquisition module: used to obtain fetal cranial ultrasound images; Construction module: used to construct a fetal cranial standard section detection network model based on the YOLOv7 model, wherein the fetal cranial standard section detection network model includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract image features, the neck network is used to perform feature fusion on the image features extracted by the backbone network, and the head network is used to detect the fusion features obtained by the neck network; An AK-ELAN module and an EMA module are introduced into the backbone network; the AK-ELAN module is used to make the backbone network pay attention to target features whose size and shape are constantly changing; the EMA module is used to improve the feature extraction capability of the backbone network by learning and fusing the global position information of the target features and spatial information of different scales; a recursive feature pyramid RFP is introduced into the neck network to improve the target recognition accuracy; Detection module: used for identifying whether the fetal cranial ultrasound image is a standard section image through the fetal cranial standard section detection network model, wherein the standard section image includes a standard thalamus section image, a standard lateral ventricle section image and a standard cerebellum section image; Among them, the standard thalamus section image includes a first unique anatomical structure and a common anatomical structure, the standard lateral ventricle section image includes a second unique anatomical structure and a common anatomical structure, and the standard cerebellum section image includes a third unique anatomical structure and a common anatomical structure. The first unique anatomical structure includes the cerebral peduncle and the choroid plexus, the second unique anatomical structure includes the posterior peduncle of the lateral ventricle and the choroid plexus, the third unique anatomical structure includes the cerebellum and the posterior cranial fossa cistern, and the common anatomical structures include the thalamus, the falx cerebri, the cavum septum pellucidum, the lateral fissure, and the skull halo.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for detecting standard sections of the fetal skull and brain as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting standard sections of the fetal skull and brain as described in any one of claims 1 to 7 are implemented.

Images