Method and device for detecting small sample of fetal anatomical structure based on ultrasound images

Abstract

The present application relates to a method, apparatus, computer device, storage medium and computer program product for detecting a small sample of fetal anatomical structure based on ultrasound images, wherein the method includes: obtaining an initial deep learning model and a standard ultrasound image of a fetus; performing feature extraction to obtain a feature map; obtaining a first classification score based on the feature map and through a DAEFLM module and a first detection head in the initial deep learning model; obtaining a fused feature map containing multi-topological information based on the feature map and through a TKRM module; obtaining a second classification score and a detection frame based on the fused feature map and through a second detection head in the initial deep learning model; performing weighted summation on the first classification score and the second classification score; constructing a standard Faster-RCNN loss function based on the classification result corresponding to the target classification score and the standard ultrasound image, and the detection frame; adjusting the model parameters of the initial deep learning model; and performing small sample detection of fetal anatomical structure according to the target deep learning model.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for detecting a small sample of fetal anatomical structure based on ultrasound images. Background Art

[0002] Ultrasound is a widely used imaging modality for the diagnosis, screening, and treatment of many diseases due to its portability, low cost, and non-invasiveness. Over the years, ultrasound has been proven to be the preferred screening method during pregnancy and is commonly used to assess fetal growth and development, as well as to monitor pregnancy and evaluate clinical suspicions. Analysis of ultrasound anatomical structures is one of the most important parts of fetal screening because it provides direct evidence of possible fetal malformations, impaired placental positioning, and risk of preterm birth. Fetal multi-anatomical structure detection is a common task in ultrasound anatomical structure analysis. By simultaneously detecting multiple anatomical structures, a clearer understanding of the associations and effects between different anatomical structures can be obtained, providing more comprehensive information to help doctors make more accurate diagnoses. In addition, multi-anatomical structure detection can also assist downstream tasks such as standard position detection.

[0003] In recent years, deep learning methods have demonstrated outstanding performance in fetal ultrasound multi-anatomical structure detection tasks. They significantly outperform traditional methods in both detection speed and accuracy, and have found widespread application in analyzing anatomical structures such as the heart and brain. However, deep learning-based object detection methods still have limitations. The accuracy and precision of these models rely heavily on a large amount of training data, and the scarcity of labeled data is a challenge for these methods. In some cases, due to ethical and privacy regulations, large amounts of data are inaccessible, and samples for rare cases such as rare diseases are difficult to obtain. In these situations, deep learning models cannot provide satisfactory detection results. Few-shot learning aims to improve the accuracy and generalization ability of models for identifying new categories by training them on abundant data from basic categories and less data from new categories, providing a solution to the data scarcity issue of deep learning models. Few-shot learning is widely used in medical image analysis research, such as tumor segmentation, disease classification, and image registration. Numerous studies have demonstrated that few-shot learning can address the problem of limited training data in medical images in most cases. While extensive research has been conducted on few-shot learning in medical images, research on few-shot object detection (FSOD) remains underexplored. Generally speaking, ultrasound images have low contrast and high noise, and it is difficult to achieve high detection performance by identifying and locating anatomical structures based solely on visual features.

[0004] It can be seen that the traditional small sample detection of fetal anatomical structure based on ultrasound images has the defect of poor accuracy. Summary of the Invention

[0005] Based on this, it is necessary to provide an accurate method, device, computer equipment, computer-readable storage medium and computer program product for detecting small samples of fetal anatomical structures based on ultrasound images to address the above technical problems.

[0006] In a first aspect, the present application provides a method for detecting a small sample of fetal anatomical structure based on ultrasound images. The method comprises:

[0007] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features are extracted from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, to which a TKRM module (topological knowledge reasoning module) and a DAEFLM module (discrimination enhancement feature learning module) are added;

[0008] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0009] Obtaining a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0010] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0011] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and using the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0012] A small sample of fetal anatomical structures is detected based on the target deep learning model.

[0013] In one embodiment, obtaining a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; obtaining a fused feature map containing multi-topology information based on the feature map and through the TKRM module includes:

[0014] Extracting a candidate frame of the feature map;

[0015] Obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model;

[0016] Obtaining a first classification score according to the feature map, the candidate bounding box, and the initial classification result, and through the DAEFLM module and the first detection head;

[0017] A fused feature map containing multi-topology information is obtained according to the feature map and the initial classification result and through the TKRM module.

[0018] In one embodiment, extracting a candidate frame of the feature map includes:

[0019] The RPN network is used to extract candidate boxes on the obtained feature map.

[0020] In one embodiment, obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model includes:

[0021] Using the ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate region corresponding to the candidate box to the feature map, so as to pool the feature map into a regional feature map of uniform size;

[0022] Sending the regional feature map to a fully connected layer to obtain a feature map of fixed size;

[0023] The fixed-size feature map is input into the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the coordinates of the key structure positions.

[0024] In one embodiment, obtaining a fused feature map containing multi-topology information based on the feature map and the initial classification result through the TKRM module includes:

[0025] The fixed-size feature map, the initial classification result, and the key structure position coordinates are input into the TKRM module to obtain a fused feature map containing multi-topology information.

[0026] In one embodiment, obtaining a first classification score based on the feature map, the candidate box, and the initial classification result, through the DAEFLM module and the first detection head, includes:

[0027] Inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing;

[0028] A first classification score is obtained by weightedly summing the initial classification result output by the first detection head and the weighted classification score map.

[0029] In a second aspect, the present application also provides a small sample detection device for fetal anatomical structure based on ultrasound images. The device comprises:

[0030] A data acquisition module is used to obtain an initial deep learning model and a standard ultrasound image of a fetus, and perform feature extraction on the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, to which the TKRM module and the DAEFLM module are added;

[0031] A first processing module is configured to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and obtain a fused feature map containing multi-topology information based on the feature map and through the TKRM module;

[0032] A second processing module is configured to obtain a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; and perform a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0033] A loss function construction module, configured to construct a standard Faster-RCNN loss function based on the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0034] A model training module is used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0035] A detection module is used to perform small sample detection of fetal anatomical structure based on the target deep learning model.

[0036] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0037] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features are extracted from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0038] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0039] Obtaining a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0040] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0041] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and using the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0042] A small sample of fetal anatomical structures is detected based on the target deep learning model.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0044] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features are extracted from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0045] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0046] Obtaining a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0047] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0048] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and using the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0049] A small sample of fetal anatomical structures is detected based on the target deep learning model.

[0050] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0051] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features are extracted from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0052] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0053] Obtaining a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0054] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0055] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and using the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0056] A small sample of fetal anatomical structures is detected based on the target deep learning model.

[0057] The above-mentioned method, device, computer equipment, storage medium and computer program product for detecting small samples of fetal anatomical structures based on ultrasound images obtain an initial deep learning model and a standard ultrasound image of the fetus, and extract features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and the TKRM module and the DAEFLM module are added to the basic framework; based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topological information is obtained through the TKRM module; based on the fused feature map, a second classification score and a detection frame are obtained through the second detection head in the initial deep learning model; a weighted sum of the first classification score and the second classification score is performed to obtain a target classification score; based on the target classification score and the classification result corresponding to the standard ultrasound image, and the detection frame, a standard Faster-RCNN loss function is constructed; based on the Faster-RCNN loss function, the model parameters of the initial deep learning model are adjusted, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model; and small sample detection of fetal anatomical structures is performed according to the target deep learning model. Throughout the entire process, a TKRM module is added to the initial deep learning model to effectively learn the topological map knowledge in ultrasound images, helping the model to classify and locate new category objects based on basic category objects, thereby improving the accuracy and robustness of detection. In addition, a DAEFLM module with enhanced discriminant ability is added to extract rich discriminant features, highlight the differences between different categories, and avoid the model from confusing highly similar basic categories with new categories, thereby improving the accuracy and robustness of classification. The basic category training stage and fine-tuning stage are subdivided into two sub-stages, allowing the model to focus on TKRM training, which can achieve accurate small-sample detection of fetal anatomical structures based on ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 FIG2 is a diagram showing an application environment of a small sample detection method for fetal anatomical structure based on ultrasound images in one embodiment;

[0059] Figure 2 1 is a flow chart of a method for detecting a small sample of fetal anatomical structure based on ultrasound images in one embodiment;

[0060] Figure 3 is a flowchart of a method for detecting a small sample of fetal anatomical structure based on ultrasound images in another embodiment;

[0061] Figure 4 The figure is a flowchart of a small sample detection method of fetal anatomical structure based on ultrasound images in an application example;

[0062] Figure 5 is a structural block diagram of a small sample detection device for fetal anatomical structure based on ultrasound images in one embodiment;

[0063] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0065] The method for detecting small samples of fetal anatomical structures based on ultrasound images provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104 or placed on the cloud or other network servers. Terminal 102 sends a detection request to server 104. Server 104 responds to the request, obtains an initial deep learning model and a standard ultrasound image of the fetus, and extracts features from the standard ultrasound image to obtain a feature map. The basic framework of the initial deep learning model is DeFRCN, and the TKRM module and the DAEFLM module are added to the basic framework. A first classification score is obtained based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model. A fused feature map containing multi-topology information is obtained based on the feature map and through the TKRM module. A second classification score and a detection frame are obtained based on the fused feature map and through the second detection head in the initial deep learning model. A weighted sum is performed on the first classification score and the second classification score to obtain a target classification score. A standard Faster-RCNN loss function is constructed based on the classification result corresponding to the target classification score and the standard ultrasound image, as well as the detection frame. The model parameters of the initial deep learning model are adjusted based on the Faster-RCNN loss function, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model. A small sample detection of fetal anatomical structure is performed based on the target deep learning model. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0066] In one embodiment, Figure 2 As shown in the figure, a small sample detection method of fetal anatomical structure based on ultrasound image is provided. Figure 1Taking the server 104 in the example as an example, the following steps are included:

[0067] S100: Obtain an initial deep learning model and a standard ultrasound image of the fetus, and perform feature extraction on the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and the TKRM module and DAEFLM module are added to the basic framework.

[0068] The TKRM module is designed to capture and model temporal or sequential relationships in the input data (here, fetal ultrasound images). Although ultrasound images themselves are not time series data, the TKRM module can be useful if multiple frames are acquired at consecutive time points or if the images contain dynamic information that changes over time (such as the fetal heartbeat or movement). The TKRM module can be designed as an independent network layer or component that accepts a series of image frames or feature maps as input and outputs a feature representation that incorporates temporal information. This can be achieved by using structures such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and transformers.

[0069] The DAEFLM module is a data-adaptive enhanced feature learning module that aims to improve the model's ability to learn features from input data. This module can automatically adjust its internal parameters or structure based on the characteristics of the input data to more effectively extract and represent key information in the data.

[0070] The DAEFLM module can be designed as a network component consisting of multiple submodules, each responsible for extracting one or more types of features from the input data. These submodules can include convolutional layers, pooling layers, attention mechanisms, residual connections, and more, and these submodules can be automatically selected and combined in a data-driven manner. Furthermore, the DAEFLM module can also incorporate self-supervised learning or meta-learning mechanisms to enable it to adaptively learn and optimize its internal parameters.

[0071] Integrating the TKRM and DAEFLM modules into the basic DeFRCN framework can be achieved through the following steps: 1. Determine the integration location: First, it is necessary to determine in which stage or component of DeFRCN the TKRM and DAEFLM modules will be integrated. This can be determined based on the specific task requirements and model structure. 2. Modify the network structure: Insert the TKRM and DAEFLM modules at the selected locations and adjust the network structure accordingly to ensure smooth connection and data flow between modules. 3. Initialize module parameters: Initialize the parameters for the TKRM and DAEFLM modules. These parameters can be set through random initialization, pre-training, or other methods. 4. Train the model: Use a labeled fetal ultrasound image dataset to train the entire model, including the basic DeFRCN framework and the newly added TKRM and DAEFLM modules. During training, appropriate loss functions and optimization algorithms can be used to optimize the model parameters.

[0072] S200: Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module.

[0073] First, a feature map is extracted from the input fetal ultrasound image. This feature map may come from an intermediate layer in DeFRCN, such as the output of the backbone network. Next, the feature map is passed to the DAEFLM module. The role of the DAEFLM module is to adaptively enhance feature learning based on the characteristics of the input data. It may include various mechanisms, such as attention mechanisms, dynamic convolution kernels, and deformable convolutions, to achieve effective processing of feature maps. The output of the DAEFLM module is an enhanced feature map that contains more discriminative information, which helps improve the performance of subsequent classification and detection tasks.

[0074] The enhanced feature map output by the DAEFLM module is passed to the first detection head in the initial deep learning model. The detection head is generally responsible for converting the feature map into outputs such as classification scores and bounding box coordinates. In the first detection head, the feature map passes through a series of convolutional layers, fully connected layers, and other network layers, ultimately outputting classification scores for each candidate region (such as the candidate bounding box generated by the RPN). These classification scores represent the probability that the candidate region belongs to different categories.

[0075] At the same time, the original feature map (or the enhanced feature map output by the DAEFLM module) is passed to the TKRM module. The TKRM module is designed to capture and model the temporal or sequential relationships in the input data, as well as multi-topology information. In the TKRM module, the feature map may undergo a series of operations, such as temporal convolution, graph convolution, recurrent neural networks, etc., to extract and fuse features at different time steps or different topological structures. These operations can help the model understand the dynamic changes and complex structures in the data. Ultimately, the TKRM module outputs a fused feature map, which contains the information in the original feature map and the multi-topology information captured by the TKRM module. This fused feature map can be used for subsequent tasks, such as further target detection, segmentation, or recognition.

[0076] S300: Obtain a second classification score and a detection frame based on the fused feature map and the second detection head in the initial deep learning model; perform a weighted summation of the first classification score and the second classification score to obtain a target classification score.

[0077] The TKRM module generates a fused feature map containing multiple topological information. Next, this fused feature map is passed to the second detection head in the initial deep learning model. The structure and function of the second detection head are similar to those of the first detection head, but the parameters and weights of the second detection head may differ due to the difference in the input feature map (here, the fused feature map processed by the TKRM module). In the second detection head, the fused feature map passes through a series of network layers (such as convolutional layers, pooling layers, fully connected layers, etc.) and is ultimately converted into a second classification score for each candidate region. These scores also represent the probability that the candidate region belongs to different categories. The weights of the weighted summation can be set according to specific needs. These weights can be fixed or learned through training. After the weighted summation of the first and second classification scores, the target classification score is obtained.

[0078] S400: Construct a standard Faster-RCNN loss function based on the target classification score, the classification result corresponding to the standard ultrasound image, and the detection box.

[0079] When constructing the standard Faster R-CNN loss function, we need to consider two main parts: classification loss and regression loss. Here, we will construct this loss function based on the given target classification score, the classification result corresponding to the standard ultrasound image, and the detection box (Bounding Boxes). The goal of the classification loss is to ensure that the model can accurately classify the detected target into the correct category. In Faster R-CNN, this is usually achieved through cross-entropy loss. The goal of the regression loss is to ensure that the model can accurately predict the location of the target in the image (i.e., the coordinates of the detection box). For each ROI (Region of Interest), the model predicts a detection box (including the coordinates of the center point and the width and height). This predicted detection box needs to be compared with the true detection box (usually given in the annotated data).

[0080] S500: Adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model.

[0081] When training the Faster R-CNN model, we usually use backpropagation and gradient descent or its variants (such as Adam, RMSProp, etc.) to adjust the model parameters according to the Faster R-CNN loss function.

[0082] The specific training and tuning process is as follows: 1. Initialize model parameters: Before training, you need an initial deep learning model in which all parameters (such as convolutional layer weights and biases) are randomly initialized, or pre-trained weights can be used for initialization. 2. Forward propagation: Feed a batch of input images into the Faster R-CNN model. Forward propagation is performed through the convolutional layers, RPN, RoI Pooling layers, and classification and regression layers to obtain classification scores and bounding box predictions for each RoI. 3. Calculate loss: Calculate the Faster R-CNN loss function based on the model's predicted classification scores, bounding box predictions, and the true labels (category and bounding box coordinates). This includes the classification and regression losses for the RPN and the classification and regression losses for the RoI. 4. Backpropagation: Using the gradient information calculated from the loss function, the backpropagation algorithm is used to calculate the gradients of the model parameters with respect to the loss function. 5. Parameter update: Use an optimization algorithm (such as gradient descent or Adam) to update the model parameters based on the calculated gradients. This step gradually reduces the model loss. 6. Iterative training: Repeat steps 2 to 5, using multiple batches of training data to iteratively train the model. In each iteration, the model updates its parameters based on the gradient of the loss function.

[0083] S600: Perform small sample detection of fetal anatomical structure based on the target deep learning model.

[0084] Based on the target deep learning model obtained by S500 and the fetal ultrasound image to be tested, a small sample test of the fetal anatomical structure is performed to obtain accurate test results.

[0085] The above-mentioned small sample detection method of fetal anatomical structure based on ultrasound image obtains an initial deep learning model and a standard ultrasound image of the fetus, and extracts features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and the TKRM module and the DAEFLM module are added to the basic framework; based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topological information is obtained through the TKRM module; according to the fused feature map, a second classification score and a detection frame are obtained through the second detection head in the initial deep learning model; a weighted sum of the first classification score and the second classification score is performed to obtain a target classification score; according to the classification result corresponding to the target classification score and the standard ultrasound image, and the detection frame, a standard Faster-RCNN loss function is constructed; based on the Faster-RCNN loss function, the model parameters of the initial deep learning model are adjusted, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model; and small sample detection of fetal anatomical structure is performed according to the target deep learning model. Throughout the entire process, a TKRM module is added to the initial deep learning model to effectively learn the topological map knowledge in ultrasound images, helping the model to classify and locate new category objects based on basic category objects, thereby improving the accuracy and robustness of detection. In addition, a DAEFLM module with enhanced discriminant ability is added to extract rich discriminant features, highlight the differences between different categories, and avoid the model from confusing highly similar basic categories with new categories, thereby improving the accuracy and robustness of classification. The basic category training stage and fine-tuning stage are subdivided into two sub-stages, allowing the model to focus on TKRM training, which can achieve accurate small-sample detection of fetal anatomical structures based on ultrasound images.

[0086] To illustrate in detail the technical solution of the method for detecting small samples of fetal anatomical structures based on ultrasound images in this application, the entire deep learning model and the modules it contains will be introduced in detail below.

[0087] In the backbone module of the deep learning model, general features suitable for downstream tasks are extracted. The backbone neural network architecture is based on ResNet101. Specifically, the first layer is the input layer, which takes a 224*224*3 pixel matrix as input. The second layer is the first convolutional layer, which receives the 224*224*3 pixel matrix from the input layer and uses 64 7*7 convolution kernels with a stride of 2. This layer then undergoes a 3*3 max pooling with a stride of 2, resulting in a 56*56*64 output matrix. The third layer is the second convolutional layer, which contains three residual blocks using three layers of convolution. The first convolution layer of the residual block uses 64 1*1 convolution kernels. A dashed structure is also added to the first layer to adjust the number of channels. The dashed structure uses 256 1*1 convolution kernels with a stride of 1, enabling the addition of the main branch and the right branch of the residual block. The second layer uses 64 3*3 convolution kernels, and the third layer uses 256 1*1 convolution kernels. After three such residual blocks, the output matrix is ​​56*56*256. The fourth layer is the third convolution layer, which contains four residual blocks using three layers of convolution. The first convolution layer of the residual block uses 128 1*1 convolution kernels. A dotted structure is added to the first layer to adjust the number of channels and image size. The dotted structure uses 512 1*1 convolution kernels with a stride of 2, allowing the main branch and right branch of the residual block to be added together. The second layer uses 128 3*3 convolution kernels, and the third layer uses 512 1*1 convolution kernels. After four such residual blocks, the output matrix is ​​28*28*512. The fifth layer is the fourth convolutional layer, which contains twenty-three residual blocks using three layers of convolution. The first convolution layer of the residual block uses 256 1x1 kernels. A dashed structure is added to the first layer to adjust the number of channels and image size. The dashed structure uses 1024 1x1 kernels with a stride of 2, allowing the main branch and right branch of the residual block to be summed. The second layer uses 256 3x3 kernels, and the third layer uses 1024 1x1 kernels. After four such residual blocks, the output matrix is ​​14x14x1024. The sixth layer is the fifth convolutional layer, which contains three residual blocks using three layers of convolution. The first convolution layer of the residual block uses 512 1x1 kernels. A dashed structure is added to the first layer to adjust the number of channels and image size. The dashed structure uses 2048 1x1 kernels with a stride of 2, allowing the main branch and right branch of the residual block to be summed. The second layer uses 512 3x3 kernels, and the third layer uses 2048 1x1 kernels. After four such residual blocks, the output matrix is ​​7x7x2048. The seventh layer is an average pooling downsampling layer, a fully connected layer, and a softmax output.

[0088] In the RPN (Region Proposal Network) module, candidate boxes are extracted from the feature map. The RPN's specific network structure is as follows: the first layer is the input layer, which receives the feature map from the backbone network as input. The second layer is the Anchor Generation layer. At each location in the feature map, a series of anchors are generated based on a preset scale and aspect ratio. These anchors serve as initial bounding box hypotheses for subsequent classification and regression. The third layer is the RPN convolutional layer. The RPN is a lightweight convolutional neural network that processes the feature map from the backbone network. A sliding window operation is performed through a 3x3 convolutional layer to extract features at each anchor location. The fourth layer is the classification and regression branch: The RPN has two output branches: a classification layer that predicts whether the anchor point is foreground or background, and a bounding box regression layer that adjusts the anchor point position to more closely surround the object. The classification layer is typically implemented using a 1x1 convolution and outputs the probability that each anchor point is foreground. The regression layer, also implemented using a 1x1 convolution, outputs the bounding box adjustment parameters for each anchor point. The fifth layer is candidate region screening and NMS (non-maximum suppression): candidate region proposals are generated based on the classification scores and the anchor points adjusted by regression. The NMS algorithm is applied to reduce overlapping proposals, retaining the proposals with the highest scores and less overlap as the final candidate regions. The sixth layer is the output layer, which outputs the screened and adjusted candidate region proposals. These proposals will be used in subsequent target detection stages, such as ROI Pooling or ROI Align, as well as final classification and refined bounding box regression. In the ROI Align module, the feature map of the candidate region of any size is converted into a small feature map of a fixed size.

[0089] In TKRM, we employ a graph neural network (GNN) that can efficiently and adaptively learn spatial relationships between anatomical structures. We use candidate regions generated by a region proposal network (RPN) as nodes and map the high-level semantic visual embeddings of the candidate regions to graph node features. We construct an adjacency matrix for the graph nodes based on their similarity. We define an operation that leverages topological structure information to generate association weights using a Gaussian kernel and use it for information diffusion and aggregation in the GNN.

[0090] In the DAEFLM module, deformable pooling is used to sample features from different subregions, extending beyond the candidate region. We employ weighted pooling to dynamically assign higher weights to distinct sampling points to extract distinctive features. The detection head module is responsible for classification and regression, predicting the object's category and location.

[0091] like Figure 3 As shown, in one embodiment, S200 includes:

[0092] S220: Extract candidate boxes from the feature map.

[0093] A deep learning model (such as a convolutional neural network (CNN)) first processes the input image, ultimately generating one or more feature maps through a series of operations such as convolution and pooling. These feature maps contain a high-level representation of the input image. From these feature maps, candidate regions of interest (ROIs), or anchors, are extracted. These candidate boxes are regions that may contain fetal anatomical structures. Specifically, a RPN network is used to extract candidate boxes from the resulting feature maps.

[0094] S240: Based on the feature map, the candidate box, and the first detection head in the initial deep learning model, an initial classification result of the fetal anatomical structure is obtained.

[0095] These candidate boxes are classified using the first detection head in the initial deep learning model. This detection head usually contains some convolutional layers and fully connected layers to output the probability that each candidate box is a fetal anatomical structure (such as head, limbs, etc.).

[0096] S260: Obtain a first classification score based on the feature map, the candidate box, and the initial classification result, through the DAEFLM module and the first detection head.

[0097] Based on the feature map, candidate bounding boxes, and initial classification results, the DAEFLM module further processes this information. DAEFLM may be an attention mechanism module that assigns higher weights to key regions in the feature map, helping the model more accurately identify fetal anatomical structures. After processing by the DAEFLM module, the first detection head returns a more precise classification score, the first classification score. These scores indicate the confidence that each candidate bounding box represents a specific fetal anatomical structure.

[0098] S280: Obtain a fused feature map containing multi-topology information based on the feature map and the initial classification result through the TKRM module.

[0099] The TKRM module is responsible for combining the feature map with the initial classification results and incorporating multi-topological information. Multi-topological information may refer to representations of fetal anatomy at different scales, orientations, or morphologies. The TKRM module may use some form of graph convolutional network (GCN) or other topological learning techniques to capture this multi-topological information. After processing by the TKRM module, a new fused feature map is obtained. This feature map not only contains the information of the original feature map, but also integrates the multi-topological information and the initial classification results.

[0100] In one embodiment, based on the feature map, the candidate box and the first detection head in the initial deep learning model, the initial classification result of the fetal anatomical structure is obtained, including: using the ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate area corresponding to the candidate box to the feature map to pool the feature map into a regional feature map of uniform size; sending the regional feature map to the fully connected layer to obtain a feature map of fixed size; inputting the fixed-size feature map into the first detection head in the initial deep learning model to obtain the initial classification result of the fetal anatomical structure and the coordinates of the key structure positions.

[0101] In the initial deep learning model, a region proposal network (RPN) is typically used to generate candidate boxes (also called regions of interest (ROIs) or anchors). These candidate boxes are potential regions that may contain fetal anatomical structures. The ROI pooling layer maps the candidate regions corresponding to candidate boxes of different sizes from the feature map and pools them into a uniformly sized regional feature map. This is because the fully connected layer (or subsequent convolutional layer) requires a fixed-size input. Specifically, for each candidate box, the ROI pooling layer maps it to the feature map and divides the region into a fixed number of grids (e.g., 7x7). Max pooling is then performed within each grid to produce a fixed-size (e.g., 7x7) regional feature map. The regional feature map obtained through ROI pooling is fed into a fully connected layer (or a series of convolutional layers). These layers further extract and integrate features to produce a fixed-size feature vector. This fixed-size feature vector is then input to the first detection head in the initial deep learning model. This detection head typically contains several fully connected layers (or convolutional layers) for classification and bounding box regression. For classification tasks, the detection head outputs the probability of each candidate box corresponding to different fetal anatomical structure categories (i.e., the initial classification result). For bounding box regression tasks, the detection head outputs the coordinate offset of each candidate box, which is used to adjust the position and size of the candidate box to more accurately locate the key structures of the fetal anatomy. The initial classification result of the fetal anatomy corresponding to each candidate box and the coordinates of the key structure locations are obtained.

[0102] In one embodiment, obtaining a fused feature map containing multiple topological information based on the feature map and the initial classification results through the TKRM module includes: inputting the fixed-size feature map, the initial classification results, and the key structure position coordinates into the TKRM module to obtain a fused feature map containing multiple topological information.

[0103] The fixed-size feature map, initial classification results, and key structure location coordinates are fed into the TKRM module as input. The TKRM module may contain multiple submodules or processing steps to extract and integrate multi-topological information. Multi-topological information may refer to the representation of fetal anatomical structures at different scales, orientations, or morphologies. Within the TKRM module, graph convolutional networks (GCNs), attention mechanisms, or other topological learning techniques may be used to process the input information. For example, GCNs can be used to model the spatial relationship between anatomical structures, or attention mechanisms can be used to focus on important feature areas. The TKRM module may also combine the initial classification results and key structure location coordinates and integrate this information into the feature map in some way to generate a richer fused feature map.

[0104] In one embodiment, obtaining a first classification score based on the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head includes: inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing; and obtaining the first classification score by weighted summation of the initial classification result output by the first detection head and the weighted classification score map.

[0105] First, there's the region of interest feature map (ROI feature map) generated through ROI pooling. This feature map corresponds to the region where the candidate bounding box is mapped on the feature map and is pooled to a uniform size. Next, there's a fixed-size feature map, which is the result of further processing the region feature map through fully connected or convolutional layers. Finally, there's the initial classification result output by the first detection head, which represents the probability that each candidate bounding box belongs to a different class. The region feature map and the fixed-size feature map are fed into the DAEFLM module as input. The DAEFLM module dynamically adjusts the weights of different regions based on the input feature map, generating a weighted classification score map. This score map may highlight key regions in the feature map related to fetal anatomy and downweight irrelevant regions. Within the DAEFLM module, attention mechanisms or other techniques may be used to dynamically generate weights, which are applied to the input feature map to generate the weighted classification score map. The initial classification result output by the first detection head and the weighted classification score map generated by the DAEFLM module are weighted summed. This step combines the classification information from the detection head with the weighted feature information from the DAEFLM module to produce a more accurate and rich first-class score. The specific method of weighted summation may include element-wise multiplication of the initial classification result and the weighted classification score map, followed by summing or averaging, depending on the model design and requirements. After weighted summation, the first-class score is obtained. This score reflects the likelihood that the candidate box contains a specific fetal anatomical structure and integrates information from multiple modules.

[0106] Overall, if Figure 4As shown, the model training in the small sample detection method of fetal anatomical structure based on ultrasound image of the present application includes two stages: in the first stage, a1) obtains a standard ultrasound image of the fetus; a2) uses the backbone to extract features from the dataset obtained in step (a1) to obtain a feature map, uses the RPN network to extract candidate boxes on the obtained feature map, uses the ROI pooling layer to obtain the feature map and the extracted candidate boxes, maps the candidate regions corresponding to the candidate boxes to the feature map, and pools the feature map into a region feature map of uniform size. The region feature map is sent to the fully connected layer to obtain a fixed-size feature map, which is input to the detection head. After passing through the two branches of the fully connected layer in the detection head, the classification of the fetal anatomical structure and the precise location coordinates of the key structures are obtained; at the same time, the feature map with the candidate region and the pooled feature map are input to the DAEFLM module, and then after passing through a classification head, a weighted classification score map is obtained. The output of the detection head is weightedly added to the weighted classification score map to finally obtain the category score. In the second stage, the TKRM module and a new detection head are added, and then the other components are frozen for training. Based on the first stage, the pooled feature map and the output of the detection head in the first stage are input into TKRM to obtain a fused feature map containing more topological information. This map is then sent to the new detection head to obtain new classification scores and region boundaries. Finally, the final category score is obtained by weighted addition with the category score of stage 1.

[0107] In order to further illustrate the model / module-related architecture and specific functional implementation details involved in the small sample detection method of fetal anatomical structure based on ultrasound images in this application, some related content will be introduced again below.

[0108] 1. Topological knowledge reasoning

[0109] We use the candidate regions generated by the RPN as nodes and select nodes with high feature similarity as adjacent nodes to create an undirected graph. We use the distances and angles between candidate centers to establish topological information. This topological information is then passed to a Gaussian kernel to generate relationship weights between adjacent nodes. This information is then diffused and aggregated within the GNN, enabling it to learn the topological knowledge in the image and generate features that incorporate this knowledge.

[0110] 1) Initialization of graph node representation

[0111] The weight of the classifier in the detection head is recorded as W∈R (C+1)×D , the classification score of the candidate area output by the classifier and softmax function is recorded as S ori∈R N×(C+1) Where N represents the number of candidate regions, C represents the number of classes, and D represents the feature dimension of the classifier input. Since W contains high-level semantic information, it is used as the visual embedding representation of the candidate region. In order to map the visual embedding into the graph node representation of the candidate region, the classification score S can be ori Multiplying with the classifier weight W yields the initial graph node X = S ori W, X∈R N×D .

[0112] 2) Graph node adjacency construction

[0113] The features obtained from RoI Align After passing through two fully connected layers and one ReLU activation layer, feature Z is obtained. T Multiply to get the feature similarity matrix M=ZZ between the candidate region nodes T The feature similarity between any two candidate regions, node i and node j, is given by M ij Given. When the number of candidate regions is large, if all nodes are fully connected, the amount of calculation will be large, which is not conducive to model training. Therefore, for each node, only the Kth node with the highest feature similarity is selected as the adjacent node:

[0114] Neighbor(i)=TOP k (M i1 , M i2 ,…,M iN ).

[0115] 3) Topology information construction

[0116] First, we need to construct topological structure information so that the GNN can learn the topological knowledge in the ultrasound image. This topology is constructed based on the spatial relative positions of the target objects in the image, and can measure distances and angles. The distances and angles between candidate regions are calculated based on the center points of the rectangular bounding boxes generated by the detection head:

[0117]

[0118] Where (x i ,y i )) and (x j ,y j ) represent the centers of the two candidate region bounding boxes respectively. The calculation function of the relative position relationship between the two proposal region boxes is denoted as R(i, j) = (d, θ).

[0119] 4) Topological knowledge reasoning

[0120] After constructing the network topology, the spatial relationships between adjacent nodes are used to help the GNN learn the topological relationships in ultrasound images. We treat the relative position distribution between different regions as a Gaussian distribution and use a Gaussian kernel to convert the relative positions into scalars. These scalars serve as relationship weights during the GNN's information propagation and aggregation process. Therefore, the model can learn topological knowledge in ultrasound images by learning these distributions. Given a graph node i with adjacent nodes, the information propagation and aggregation operations for this node can be expressed as:

[0121] x′ im =∑ j∈Neighbour(i) ω m (R(i,j))x j ,

[0122]

[0123] μ′ m =R(i, j)-μ m ,

[0124] In the above formula, x j is the feature of the adjacent node j, ω m is the mth Gaussian kernel, μ m is a learnable 2×1 mean vector, Σ m is a learnable 2×2 covariance matrix. For m new features, these features are concatenated and input into a linear transformation layer to obtain the new feature x′ of node i. i .

[0125] After information propagation and aggregation through this operation, the GNN learns the topological knowledge of the ultrasound image and outputs node representations that carry this topological knowledge. These representations combine local and global information, significantly enhancing the topological structure. This topological structure information helps the model perform structured reasoning and prediction, imposing constraints for classifying and localizing objects of new categories. This information helps mitigate the impact of low-quality candidate regions on new categories, improving the prediction accuracy of candidate region classification and localization. These candidate region node representations are concatenated with the original features f through a linear transformation layer and a ReLU layer. The resulting fused features are input into a new detection head to produce the final classification score and region boundaries.

[0126] 2. Feature learning to enhance discriminative ability

[0127] The feature learning process of Faster-RCNN consists of multiple stages, including feature learning of the backbone network, RoIPooling, etc. Our main focus is on learning discriminative features in the RoI Align and RoI Pooling stages. Although traditional RoI Align and RoI Pooling methods can meet most object detection scenarios, their sampling strategies may be more suitable for extracting discriminative features. Therefore, in the RoI Align and RoI Pooling processes, we apply deformable pooling to resample features from different regions, rather than just being limited to candidate regions. We use weighted pooling to adaptively assign higher weights to discriminative sampling points to extract discriminative features at a fine-grained scale.

[0128] 1) Sub-region resampling

[0129] Input a feature map X and a candidate region P, the candidate region is divided into k×k sub-regions, and then these sub-regions are average pooled to obtain the feature map X1∈R C×W×H , where C represents the number of channels, W represents the width of the image, and H represents the height of the image. Then, X1 is alternately passed through three fully connected layers and two ReLU layers to obtain the sub-region offset predictions. The prediction results are then processed using bilinear interpolation. Position-sensitive RoI Pooling is used to combine the feature map X and the candidate region P with these offset predictions to obtain the new feature map X new ∈R C×4W×4H This process helps to incorporate rich local context information into the new feature map X new In

[15] , the impact of low-quality proposals generated for new categories is reduced because the locations of the resampled subregions are not constrained but are related to those of the original proposals.

[0130] 2) Adaptive weighted pooling

[0131] In order to obtain the pooling result, we transform the feature map X new The network is divided into 2k×2k subregions, each containing 4 points. We assign different weights to each point in the subregion. First, each subregion passes through a convolutional layer, a normalization layer, and an activation layer to generate a matrix containing the corresponding weights for each point in the subregion.

[0132] Then, the weight matrix corresponding to each sub-region is used as the convolution kernel. After performing the convolution operation on the sub-region, a new feature map X2∈R is obtained. C×2W×2H Finally, we perform average pooling on X2 to obtain the feature map X dis ∈R C×W×H .

[0133] The X of all candidate regionsdis It is concatenated with the original pooled feature f and fed into a fully connected layer and a separate classification head to generate a new classification score map S dis , and compare it with the original classification score map S ori After weighted addition, the final classification score graph S is obtained. final :

[0134] S final =(1-γ)S ori +γS dis ,

[0135] In this formula, γ represents the classification score map S dis The weight of .

[0136] 3. Training strategy

[0137] The basic framework used in this application is DeFRCN, a transfer learning-based approach with two training phases: training on the base class and fine-tuning on all classes. To enable the GNN in the TKRM to more effectively learn the relative positional relationships between anatomical structures, we aim to train this module after obtaining high-quality high-level features, classification scores, and rectangular box boundaries. This allows the model to focus on training the TKRM and accelerate the convergence of TKRM parameters. Therefore, each of the two original training phases is subdivided into two sub-phases. The first sub-phase excludes the TKRM, while the second sub-phase includes it and freezes the training of other components.

[0138] 4. Loss Function

[0139] We use the standard Faster-RCNN loss function, which consists of a classification loss L rpn_cls and rectangular box regression loss L rpn_loc The RPN loss is composed of the classification loss L cls and rectangular box regression loss L box_reg The detector loss composed of is the sum of RPN loss and detector loss:

[0140] L total =L rpn_cls +L rpn_loc +L cls +L box_reg .

[0141] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0142] Based on the same inventive concept, embodiments of the present application also provide an ultrasound-based small sample fetal anatomical structure detection device for implementing the ultrasound-based small sample fetal anatomical structure detection method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more of the following embodiments of the ultrasound-based small sample fetal anatomical structure detection device can be found in the limitations of the ultrasound-based small sample fetal anatomical structure detection method described above and will not be repeated here.

[0143] In one embodiment, Figure 5 As shown, a small sample detection device for fetal anatomical structure based on ultrasound images is provided.

[0144] The data acquisition module 100 is used to obtain an initial deep learning model and a standard ultrasound image of a fetus, and perform feature extraction on the standard ultrasound image to obtain a feature map. The basic framework of the initial deep learning model is DeFRCN, to which the TKRM module and DAEFLM module are added.

[0145] A first processing module 200 is configured to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and obtain a fused feature map containing multiple topological information based on the feature map and through the TKRM module;

[0146] The second processing module 300 is configured to obtain a second classification score and a detection box based on the fused feature map and the second detection head in the initial deep learning model; and perform a weighted summation of the first classification score and the second classification score to obtain a target classification score.

[0147] A loss function construction module 400 is used to construct a standard Faster-RCNN loss function based on the target classification score, the classification result corresponding to the standard ultrasound image, and the detection box;

[0148] A model training module 500 is used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0149] The detection module 600 is used to perform small sample detection of fetal anatomical structure based on the target deep learning model.

[0150] In one embodiment, the first processing module 200 is also used to extract candidate boxes of the feature map; based on the feature map, the candidate boxes and the first detection head in the initial deep learning model, an initial classification result of the fetal anatomical structure is obtained; according to the feature map, the candidate boxes, and the initial classification result, and through the DAEFLM module and the first detection head, a first classification score is obtained; according to the feature map and the initial classification result, and through the TKRM module, a fused feature map containing multiple topological information is obtained.

[0151] In one embodiment, the first processing module 200 is further configured to extract a candidate box from the obtained feature map using an RPN network.

[0152] In one embodiment, the first processing module 200 is also used to use the ROI pooling layer to obtain the feature map and the extracted candidate box, map the candidate area corresponding to the candidate box into the feature map, and pool the feature map into a regional feature map of uniform size; send the regional feature map into the fully connected layer to obtain a feature map of fixed size; input the fixed-size feature map into the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the coordinates of the key structure positions.

[0153] In one embodiment, the first processing module 200 is further configured to input the fixed-size feature map, the initial classification result, and the key structure position coordinates into the TKRM module to obtain a fused feature map containing multiple topological information.

[0154] In one embodiment, the first processing module 200 is also used to input the regional feature map and the fixed-size feature map into the DAEFLM module, and after classification processing, obtain a weighted classification score map; the initial classification result output by the first detectionhead and the weighted classification score map are weightedly summed to obtain the first classification score.

[0155] Each module in the aforementioned ultrasound-based fetal anatomical structure small sample detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0156] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting a small sample of fetal anatomical structure based on ultrasound images is implemented.

[0157] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0158] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method for detecting a small sample of fetal anatomical structure based on ultrasound images when executing the computer program.

[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting a small sample of fetal anatomical structure based on ultrasound images is implemented.

[0160] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the above-mentioned method for detecting a small sample of fetal anatomical structure based on ultrasound images.

[0161] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0162] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting small samples of fetal anatomical structures based on ultrasound images, characterized in that: The method comprises: An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features are extracted from the standard ultrasound image to obtain a feature map. The basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework. The TKRM module is used to combine the feature map with the initial classification results and introduce multi-topological information, which is a representation of the fetal anatomical structure at different scales, directions, or morphologies. The DAEFLM module is used to dynamically adjust the weights of different regions based on the input feature map to generate a weighted classification score map. The score map is used to highlight key regions related to the fetal anatomical structure in the feature map and reduce the weights of irrelevant regions. Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module; Obtaining a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score; Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame; Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and using the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model; Performing fetal anatomical structure small sample testing based on the target deep learning model; The method of obtaining a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; obtaining a fused feature map containing multiple topological information based on the feature map and through the TKRM module includes: extracting a candidate box from the feature map; obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model; obtaining a first classification score based on the feature map, the candidate box, and the initial classification result and through the DAEFLM module and the first detection head; obtaining a fused feature map containing multiple topological information based on the feature map and the initial classification result and through the TKRM module; Obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model includes: using an ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate region corresponding to the candidate box to the feature map, so as to pool the feature map into a regional feature map of uniform size; sending the regional feature map to a fully connected layer to obtain a feature map of fixed size; inputting the fixed-size feature map to the first detection head in the initial deep learning model to obtain an initial classification result of the fetal anatomical structure and the coordinates of the key structure positions; Obtaining a first classification score based on the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head, includes: inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing; and obtaining a first classification score by weighted summing the initial classification result output by the first detection head and the weighted classification score map; The fused feature map containing multiple topological information is obtained based on the feature map and the initial classification result and through the TKRM module, including: inputting the fixed-size feature map, the initial classification result, and the key structure position coordinates into the TKRM module to obtain a fused feature map containing multiple topological information.

2. The method according to claim 1, characterized in that The extracting of the candidate frame of the feature map includes: The RPN network is used to extract candidate boxes on the obtained feature map.

3. A small sample detection device for fetal anatomical structure based on ultrasound images, characterized in that: The device comprises: A data acquisition module is configured to acquire an initial deep learning model and a standard ultrasound image of a fetus, and perform feature extraction on the standard ultrasound image to obtain a feature map. The basic framework of the initial deep learning model is DeFRCN, to which a TKRM module and a DAEFLM module are added. The TKRM module is configured to combine the feature map with the initial classification results and introduce multi-topological information, which is a representation of the fetal anatomical structure at different scales, directions, or morphologies. The DAEFLM module is configured to dynamically adjust the weights of different regions based on the input feature map to generate a weighted classification score map. The score map is configured to highlight key regions in the feature map related to the fetal anatomical structure and reduce the weights of irrelevant regions. A first processing module is configured to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and obtain a fused feature map containing multi-topology information based on the feature map and through the TKRM module; A second processing module is configured to obtain a second classification score and a detection frame based on the fused feature map and through the second detection head in the initial deep learning model; and perform a weighted summation of the first classification score and the second classification score to obtain a target classification score; A loss function construction module, configured to construct a standard Faster-RCNN loss function based on the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame; A model training module is used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model; a detection module, configured to perform small sample detection of fetal anatomical structures based on the target deep learning model; The first processing module is also used to extract candidate boxes from the feature map; use the ROI pooling layer to obtain the feature map and the extracted candidate boxes, map the candidate areas corresponding to the candidate boxes to the feature map, and pool the feature map into a regional feature map of uniform size; send the regional feature map to the fully connected layer to obtain a feature map of fixed size; input the fixed-size feature map to the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the key structure position coordinates; input the regional feature map and the fixed-size feature map into the DAEFLM module, and after classification processing, obtain a weighted classification score map; weighted sum the initial classification result output by the first detection head and the weighted classification score map to obtain a first classification score; input the fixed-size feature map, the initial classification result, and the key structure position coordinates into the TKRM module to obtain a fused feature map containing multiple topological information.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images