Fetal anatomical structure small sample detection method and device based on ultrasonic image

Abstract

The invention relates to a fetus anatomical structure small sample detection method and device based on an ultrasonic image, computer equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining an initial deep learning model and a standard ultrasonic image of a fetus; performing feature extraction to obtain a feature map; based on the feature map, a first classification score is obtained through a DAEFLM module and a first portion head in an initial deep learning model; based on the feature map, a fusion feature map containing multi-topology information is obtained through a TKRM module; according to the fused feature map, obtaining a second classification score and a detection frame through a second detection head in the initial deep learning model; performing weighted summation on the first classification score and the second classification score; constructing a standard Faster-RCNN loss function according to the target classification score, a classification result corresponding to the standard ultrasonic image and the detection frame; adjusting model parameters of the initial deep learning model; and performing fetal anatomical structure small sample detection according to the target deep learning model.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for detecting a small sample of fetal anatomical structure based on ultrasound images. Background Art

[0002] Ultrasound is a widely used imaging modality for diagnosis, screening, and treatment of many diseases due to its portability, low cost, and non-invasiveness. Over the years, ultrasound has been proven to be the screening method of choice during pregnancy and is often used to assess fetal growth and development, as well as to monitor pregnancy and evaluate clinical suspicion. Analysis of ultrasound anatomical structures is one of the most important parts of fetal screening because it provides direct evidence of possible fetal malformations, impaired placental positioning, and risk of preterm birth. Fetal multi-anatomical structure detection is a common task in ultrasound anatomical structure analysis. By detecting multiple anatomical structures at the same time, a clearer understanding of the associations and effects between different anatomical structures can be obtained, providing more comprehensive information to help doctors diagnose more accurately. In addition, multi-anatomical structure detection can also assist downstream tasks such as standard position detection.

[0003] In recent years, deep learning methods have performed well in fetal ultrasound multi-anatomical structure tasks. They significantly outperform traditional methods in detection speed and accuracy, and have been widely used in the analysis of anatomical structures such as the heart and brain. However, deep learning-based object detection methods still have limitations. The accuracy and precision of these models rely heavily on a large amount of training data, so the scarcity of labeled data is a problem faced by these methods. In some cases, due to ethical and privacy regulations, a large amount of data is inaccessible, and it is difficult to obtain available samples for rare cases such as rare diseases. In this case, deep learning models cannot provide satisfactory detection results. Few-shot learning aims to improve the accuracy and generalization ability of the model to recognize new categories by training on rich basic category data and less new category data, providing a solution to the data scarcity problem of deep learning models. Few-shot learning is widely used in the research of medical image analysis, such as tumor segmentation, disease classification, image registration, etc. A large number of studies have shown that in most cases, few-shot learning can solve the problem of limited training data for medical images. Although there have been many studies on few-shot learning for medical images, the research on few-shot object detection (FSOD) has yet to be explored. Generally speaking, ultrasound images have low contrast and high noise, and it is difficult to achieve high detection performance by identifying and locating anatomical structures based solely on visual features.

[0004] It can be seen that the traditional small sample detection of fetal anatomical structure based on ultrasound images has the defect of poor accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate method, device, computer equipment, computer-readable storage medium and computer program product for detecting small samples of fetal anatomical structures based on ultrasound images in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for detecting a small sample of fetal anatomical structure based on ultrasound images. The method comprises:

[0007] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features of the standard ultrasound image are extracted to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module (topological knowledge reasoning module) and a DAEFLM module (discrimination ability enhanced feature learning module) are added to the basic framework;

[0008] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0009] Obtaining a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0010] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0011] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and taking the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0012] A small sample detection of fetal anatomical structure is performed based on the target deep learning model.

[0013] In one embodiment, the obtaining of a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and the obtaining of a fused feature map containing multi-topology information based on the feature map and through the TKRM module include:

[0014] Extracting a candidate box of the feature map;

[0015] Based on the feature map, the candidate box, and the first detectionhead in the initial deep learning model, obtaining an initial classification result of the fetal anatomical structure;

[0016] Obtaining a first classification score according to the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head;

[0017] According to the feature map and the initial classification result, a fused feature map containing multi-topology information is obtained through the TKRM module.

[0018] In one embodiment, extracting a candidate frame of the feature map includes:

[0019] The RPN network is used to extract candidate boxes on the obtained feature map.

[0020] In one embodiment, obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model includes:

[0021] Using the ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate area corresponding to the candidate box to the feature map, so as to pool the feature map into a regional feature map of uniform size;

[0022] Sending the regional feature map to a fully connected layer to obtain a feature map of a fixed size;

[0023] The fixed-size feature map is input into the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the coordinates of the key structure positions.

[0024] In one embodiment, obtaining a fused feature map containing multi-topology information according to the feature map and the initial classification result through the TKRM module includes:

[0025] The fixed-size feature map, the initial classification result, and the key structure position coordinates are input into the TKRM module to obtain a fused feature map containing multi-topology information.

[0026] In one embodiment, obtaining a first classification score according to the feature map, the candidate box, and the initial classification result through the DAEFLM module and the first detection head includes:

[0027] Inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing;

[0028] A first classification score is obtained by weightedly summing the initial classification result output by the first detection head and the weighted classification score map.

[0029] In a second aspect, the present application also provides a small sample detection device for fetal anatomical structure based on ultrasound images. The device comprises:

[0030] A data acquisition module is used to acquire an initial deep learning model and a standard ultrasound image of a fetus, and extract features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0031] A first processing module, configured to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and obtain a fused feature map containing multi-topology information based on the feature map and through the TKRM module;

[0032] A second processing module is used to obtain a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; and perform weighted summation on the first classification score and the second classification score to obtain a target classification score;

[0033] A loss function construction module, used to construct a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0034] A model training module, used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0035] A detection module is used to perform small sample detection of fetal anatomical structure according to the target deep learning model.

[0036] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0037] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features of the standard ultrasound image are extracted to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0038] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0039] Obtaining a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0040] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0041] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and taking the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0042] A small sample detection of fetal anatomical structure is performed based on the target deep learning model.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features of the standard ultrasound image are extracted to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0045] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0046] Obtaining a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0047] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0048] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and taking the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0049] A small sample detection of fetal anatomical structure is performed based on the target deep learning model.

[0050] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0051] An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features of the standard ultrasound image are extracted to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework;

[0052] Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module;

[0053] Obtaining a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0054] Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame;

[0055] Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and taking the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0056] A small sample detection of fetal anatomical structure is performed based on the target deep learning model.

[0057] The above-mentioned method, device, computer equipment, storage medium and computer program product for detecting small samples of fetal anatomical structures based on ultrasound images obtain an initial deep learning model and a standard ultrasound image of the fetus, and extract features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework; a first classification score is obtained based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; a fused feature map containing multi-topological information is obtained based on the feature map and through the TKRM module; a second classification score and a detection frame are obtained based on the fused feature map and through the second detection head in the initial deep learning model; a weighted sum of the first classification score and the second classification score is performed to obtain a target classification score; a standard Faster-RCNN loss function is constructed based on the classification result corresponding to the target classification score and the standard ultrasound image, and the detection frame; the model parameters of the initial deep learning model are adjusted based on the Faster-RCNN loss function, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model; and a small sample of fetal anatomical structure detection is performed according to the target deep learning model. During the whole process, a TKRM module is added to the initial deep learning model to effectively learn the topological map knowledge in ultrasound images, help the model to classify and locate new category objects based on basic category objects, and improve the accuracy and robustness of detection. In addition, a DAEFLM module with enhanced discriminative ability is added to extract rich discriminative features, highlight the differences between different categories, and avoid the model confusing highly similar basic categories and new categories, thereby improving the accuracy and robustness of classification. The basic category training stage and the fine-tuning stage are subdivided into two sub-stages, so that the model can focus on the training of TKRM, which can achieve accurate small sample detection of fetal anatomical structures based on ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A diagram of an application environment of a small sample detection method of fetal anatomical structure based on ultrasound images in one embodiment;

[0059] Figure 2 is a schematic flow chart of a method for detecting a small sample of fetal anatomical structure based on ultrasound images in one embodiment;

[0060] Figure 3 It is a flowchart of a method for detecting a small sample of fetal anatomical structure based on ultrasound images in another embodiment;

[0061] Figure 4 A flowchart of a small sample detection method of fetal anatomical structure based on ultrasound images in an application example;

[0062] Figure 5 is a structural block diagram of a small sample detection device for fetal anatomical structure based on ultrasound images in one embodiment;

[0063] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] The small sample detection method of fetal anatomical structure based on ultrasound image provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 sends a detection request to the server 104, and the server 104 responds to the request, obtains an initial deep learning model and a standard ultrasound image of the fetus, and extracts features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework; based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module; according to the fused feature map, a second classification score and a detection frame are obtained through the second detection head in the initial deep learning model; the first classification score and the second classification score are weightedly summed to obtain a target classification score; according to the classification result corresponding to the target classification score and the standard ultrasound image, and the detection frame, a standard Faster-RCNN loss function is constructed; based on the Faster-RCNN loss function, the model parameters of the initial deep learning model are adjusted, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model; a small sample of fetal anatomical structure is detected according to the target deep learning model. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0066] In one embodiment, Figure 2 As shown, a method for detecting small samples of fetal anatomical structures based on ultrasound images is provided. Figure 1Taking the server 104 in the example as an example, the following steps are included:

[0067] S100: Obtain an initial deep learning model and a standard ultrasound image of a fetus, and extract features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework.

[0068] The TKRM module is designed to capture and model temporal or sequential relationships in the input data (here, fetal ultrasound images). Although ultrasound images themselves are not time series data, the TKRM module can be useful if multiple frames are acquired at consecutive time points or if the images contain dynamic information that changes over time (such as the heartbeat or movement of the fetus). The TKRM module can be designed as an independent network layer or component that accepts a series of image frames or feature maps as input and outputs a feature representation that incorporates temporal information. This can be achieved by using structures such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), transformers, etc.

[0069] The DAEFLM module is a data-adaptive enhanced feature learning module that aims to improve the model's ability to learn features from input data. This module can automatically adjust its internal parameters or structure according to the characteristics of the input data to more effectively extract and represent key information in the data.

[0070] The DAEFLM module can be designed as a network component containing multiple submodules, each of which is responsible for extracting one or more types of features from the input data. These submodules can include convolutional layers, pooling layers, attention mechanisms, residual connections, etc., and these submodules can be automatically selected and combined in a data-driven manner. In addition, the DAEFLM module can also include some self-supervised learning or meta-learning mechanisms to enable it to adaptively learn and optimize its internal parameters.

[0071] The integration of TKRM and DAEFLM modules in the basic framework of DeFRCN can be achieved through the following steps: 1. Determine the integration location: First, it is necessary to determine in which stage or component of DeFRCN to integrate the TKRM and DAEFLM modules. This can be determined based on the specific task requirements and model structure. 2. Modify the network structure: Insert the TKRM and DAEFLM modules at the selected location, and adjust the network structure accordingly to ensure smooth connection and data flow between modules. 3. Initialize module parameters: Initialize parameters for the TKRM and DAEFLM modules. These parameters can be set by random initialization, pre-training or other methods. 4. Train the model: Use the labeled fetal ultrasound image dataset to train the entire model, including the basic framework of DeFRCN and the newly added TKRM and DAEFLM modules. During the training process, the parameters of the model can be optimized using appropriate loss functions and optimization algorithms.

[0072] S200: Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module.

[0073] First, a feature map is extracted from the input fetal ultrasound image. This feature map may come from an intermediate layer in DeFRCN, such as the output of the backbone network. Next, the feature map is passed to the DAEFLM module. The role of the DAEFLM module is to adaptively enhance feature learning based on the characteristics of the input data. It may contain multiple mechanisms, such as attention mechanism, dynamic convolution kernel, deformable convolution, etc., to achieve effective processing of feature maps. The output of the DAEFLM module is an enhanced feature map, which contains more discriminative information and helps to improve the performance of subsequent classification and detection tasks.

[0074] The enhanced feature map output by the DAEFLM module is passed to the first detection head in the initial deep learning model. The detection head is usually responsible for converting the feature map into outputs such as classification scores and bounding box coordinates. In the first detection head, the feature map will pass through a series of network layers such as convolutional layers and fully connected layers, and finally output the classification score of each candidate region (such as the candidate box generated by RPN). These classification scores represent the probability that the candidate region belongs to different categories.

[0075] At the same time, the original feature map (or the enhanced feature map output by the DAEFLM module) is passed to the TKRM module. The TKRM module is designed to capture and model temporal or sequential relationships in the input data, as well as multi-topological information. In the TKRM module, the feature map may undergo a series of operations, such as temporal convolution, graph convolution, recurrent neural network, etc., to extract and fuse features at different time steps or different topological structures. These operations can help the model understand the dynamic changes and complex structures in the data. Finally, the TKRM module outputs a fused feature map, which contains the information in the original feature map and the multi-topological information captured by the TKRM module. This fused feature map can be used for subsequent tasks, such as further target detection, segmentation or recognition.

[0076] S300: Obtain a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; perform weighted summation on the first classification score and the second classification score to obtain a target classification score.

[0077] The fused feature map containing multi-topological information is obtained through the TKRM module. Next, this fused feature map is passed to the second detection head in the initial deep learning model. The structure and function of the second detection head are similar to those of the first detection head, but the parameters and weights of the second detection head may be different due to the difference in the input feature map (here is the fused feature map processed by the TKRM module). In the second detection head, the fused feature map will pass through a series of network layers (such as convolutional layers, pooling layers, fully connected layers, etc.), and will eventually be converted into the second classification score of each candidate region. These scores also represent the probability that the candidate region belongs to different categories. The weights of the weighted summation can be set according to specific needs. These weights can be fixed or obtained through training and learning. After weighted summation of the first classification score and the second classification score, the target classification score is obtained.

[0078] S400: Construct a standard Faster-RCNN loss function based on the target classification score, the classification result corresponding to the standard ultrasound image, and the detection box.

[0079] When constructing the standard Faster R-CNN loss function, we need to consider two main parts: Classification Loss and Regression Loss. Here, we will construct this loss function based on the given target classification score, the classification result corresponding to the standard ultrasound image, and the detection box. The goal of the classification loss is to ensure that the model can accurately classify the detected target into the correct category. In Faster R-CNN, this is usually achieved through cross-entropy loss. The goal of regression loss is to ensure that the model can accurately predict the location of the target in the image (that is, the coordinates of the detection box). For each ROI (Region of Interest), the model predicts a detection box (including the coordinates of the center point and the width and height). This predicted detection box needs to be compared with the true detection box (usually given in the labeled data).

[0080] S500: Adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model.

[0081] When training the Faster R-CNN model, we usually use backpropagation and gradient descent or its variants (such as Adam, RMSProp, etc.) to adjust the parameters of the model according to the Faster R-CNN loss function.

[0082] The specific training and adjustment process is as follows: 1. Initialize model parameters: Before starting training, you need an initial deep learning model in which all parameters (such as weights of convolutional layers, bias terms, etc.) are randomly initialized, or can be initialized using pre-trained weights. 2. Forward propagation: Feed a batch of input images into the Faster R-CNN model. Forward propagation is performed through the convolutional layer, RPN, RoI Pooling layer, and classification and regression layers to obtain the classification score and bounding box prediction for each ROI. 3. Calculate loss: Calculate the Faster R-CNN loss function based on the classification score predicted by the model, bounding box prediction, and true label (category and bounding box coordinates). This includes the classification and regression loss of RPN, as well as the classification and regression loss of RoI. 4. Backward propagation: Using the gradient information calculated by the loss function, the back propagation algorithm is used to calculate the gradient of the model parameters relative to the loss function. 5. Parameter update: Use an optimization algorithm (such as gradient descent, Adam, etc.) to update the model parameters based on the calculated gradients. This step will gradually reduce the loss of the model. 6. Iterative training: Repeat steps 2 to 5 and iteratively train the model using multiple batches of training data. In each iteration, the model updates its parameters based on the gradient of the loss function.

[0083] S600: Perform small sample detection of fetal anatomical structure based on the target deep learning model.

[0084] Based on the target deep learning model obtained by S500 and the fetal ultrasound image to be tested, a small sample test of the fetal anatomical structure is performed to obtain accurate test results.

[0085] The above-mentioned small sample detection method of fetal anatomical structure based on ultrasound image obtains an initial deep learning model and a standard ultrasound image of the fetus, and extracts features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and TKRM module and DAEFLM module are added to the basic framework; based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topological information is obtained through the TKRM module; according to the fused feature map, a second classification score and a detection frame are obtained through the second detection head in the initial deep learning model; the first classification score and the second classification score are weightedly summed to obtain a target classification score; according to the classification result corresponding to the target classification score and the standard ultrasound image, and the detection frame, a standard Faster-RCNN loss function is constructed; based on the Faster-RCNN loss function, the model parameters of the initial deep learning model are adjusted, and the model corresponding to the minimum Faster-RCNN loss function is used as the target deep learning model; and a small sample detection of fetal anatomical structure is performed according to the target deep learning model. During the whole process, a TKRM module is added to the initial deep learning model to effectively learn the topological map knowledge in ultrasound images, help the model to classify and locate new category objects based on basic category objects, and improve the accuracy and robustness of detection. In addition, a DAEFLM module with enhanced discriminative ability is added to extract rich discriminative features, highlight the differences between different categories, and avoid the model confusing highly similar basic categories and new categories, thereby improving the accuracy and robustness of classification. The basic category training stage and the fine-tuning stage are subdivided into two sub-stages, so that the model can focus on the training of TKRM, which can achieve accurate small sample detection of fetal anatomical structures based on ultrasound images.

[0086] In order to explain in detail the technical solution of the small sample detection method of fetal anatomical structure based on ultrasound images in this application, the entire deep learning model and the modules it contains will be introduced below.

[0087] In the backbone module of the deep learning model, general features suitable for downstream tasks are extracted. The neural network structure of the backbone is based on resnet101. Specifically, the first layer is the input layer, which inputs a 224*224*3 pixel matrix. The second layer is the first convolution layer, which receives a 224*224*3 pixel matrix from the input layer. This layer uses 64 convolution kernels of size 7*7, and the step size of this layer is 2. Then, after a 3*3 maximum pooling sampling, the step size is 2, and finally a 56*56*64 output matrix is ​​obtained; the third layer is the second convolution layer, which contains three residual blocks using three layers of convolution; the first layer of convolution of the residual block uses 64 convolution kernels of size 1*1, and a dotted line structure is added to the first layer to adjust the number of channels. The dotted line structure uses 256 1*1 convolution kernels with a step size of 1, so that the main branch and the right branch of the residual block can be added. The second layer uses 64 3*3 convolution kernels, and the third layer uses 256 1*1 convolution kernels. After three such residual blocks, an output matrix of 56*56*256 is obtained. The fourth layer is the third convolution layer, which contains four residual blocks using three layers of convolution; the first layer of convolution of the residual block uses 128 convolution kernels of size 1*1, and a dotted line structure is added to the first layer to adjust the number of channels and the size of the image. The dotted line structure uses 512 1*1 convolution kernels with a step size of 2, so that the main branch and the right branch of the residual block can be added. The second layer uses 128 3*3 convolution kernels, and the third layer uses 512 1*1 convolution kernels. After four such residual blocks, an output matrix of 28*28*512 is obtained. The fifth layer is the fourth convolutional layer, which contains twenty-three residual blocks using three layers of convolution. The first layer of convolution of the residual block uses 256 convolution kernels of size 1*1, and the first layer also adds a dotted line structure to adjust the number of channels and the size of the image. The dotted line structure uses 1024 1*1 convolution kernels with a step size of 2, so that the main branch and the right branch of the residual block can be added. The second layer uses 256 3*3 convolution kernels, and the third layer uses 1024 1*1 convolution kernels. After four such residual blocks, an output matrix of 14*14*1024 is obtained. The sixth layer is the fifth convolutional layer, which contains three residual blocks using three layers of convolution; the first layer of convolution of the residual block uses 512 convolution kernels of size 1*1, and the first layer also adds a dotted structure to adjust the number of channels and the size of the image. The dotted structure uses 2048 1*1 convolution kernels with a step size of 2, so that the main branch and the right branch of the residual block can be added. The second layer uses 512 3*3 convolution kernels, and the third layer uses 2048 1*1 convolution kernels. After four such residual blocks, an output matrix of 7*7*2048 is obtained. The seventh layer is an average pooling downsampling layer, a fully connected layer, and a softmax output.

[0088] In the RPN (Region Proposal Network) module, candidate boxes are extracted from the feature map. The specific network structure of RPN is as follows: the first layer is the input layer, which receives the feature map from the backbone as input. The second layer is Anchor Generation. At each position in the feature map, a series of anchors are generated according to the preset scale and aspect ratio. These anchors are the initial bounding box hypotheses for subsequent classification and regression. The third layer is the RPN convolution layer. RPN is a lightweight convolutional neural network that processes the feature map from the backbone network. A sliding window operation is performed through a 3x3 convolution layer to extract the features of each anchor position. The fourth layer is the classification and regression branch: RPN has two output branches: a classification layer for predicting whether the anchor is foreground or background, and a bounding box regression layer for adjusting the anchor position to more closely surround the target. The classification layer is usually implemented by a 1x1 convolution, outputting the probability that each anchor is the foreground. The regression layer is also implemented by a 1x1 convolution, outputting the bounding box adjustment parameters of each anchor. The fifth layer is candidate region screening and NMS (non-maximum suppression): candidate region proposals are generated based on the classification scores and regression-adjusted anchors. The NMS algorithm is applied to reduce overlapping proposals, and the proposals with the highest scores and less overlap are retained as the final candidate regions. The sixth layer is the output layer, which outputs the screened and adjusted candidate region proposals, which will be used in subsequent target detection stages such as ROI Pooling or ROI Align, as well as final classification and refined bounding box regression. In the ROI Align module, the feature map of the candidate region of arbitrary size is converted into a small feature map with a fixed size.

[0089] In TKRM, we adopt a graph neural network (GNN) that can effectively and adaptively learn the spatial relationships between anatomical structures. We use the candidate regions generated by the region proposal network (RPN) as nodes, and map the high-level semantic visual embeddings of the candidate regions to the graph node features. We construct the adjacency matrix of the graph nodes based on their similarities. We define an operation that uses topological structure information to generate association weights through a Gaussian kernel and use it for information diffusion and aggregation in GNN.

[0090] In the DAEFLM module, features are sampled from different sub-regions through deformable pooling and extended beyond the candidate region. We use weighted pooling to dynamically assign higher weights to distinguished sampling points to extract distinguished features. In the detection head module, it is responsible for classification and regression to predict the category and location of the target.

[0091] like Figure 3 As shown, in one embodiment, S200 includes:

[0092] S220: Extract candidate boxes of feature maps.

[0093] A deep learning model (such as a convolutional neural network (CNN)) first processes the input image, and finally obtains one or more feature maps through a series of convolution, pooling and other operations. These feature maps contain a high-level representation of the input image. Candidate boxes (Region of Interest, ROI) or anchors are extracted from the feature maps. These candidate boxes are regions that may contain fetal anatomical structures. Specifically, the RPN network can be used to extract candidate boxes on the obtained feature maps.

[0094] S240: Based on the feature map, the candidate box, and the first detection head in the initial deep learning model, an initial classification result of the fetal anatomical structure is obtained.

[0095] These candidate boxes are classified using the first detection head in the initial deep learning model. This detection head usually contains some convolutional layers and fully connected layers to output the probability that each candidate box is a fetal anatomical structure (such as head, limbs, etc.).

[0096] S260: Obtain a first classification score according to the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head.

[0097] Based on the feature map, candidate boxes, and initial classification results, the DAEFLM module further processes this information. DAEFLM may be an attention mechanism module that gives higher weights to key areas in the feature map, thereby helping the model to more accurately identify fetal anatomical structures. After being processed by the DAEFLM module, a more accurate classification score, namely the first classification score, is obtained again through the first detectionhead. These scores represent the confidence that each candidate box is a specific fetal anatomical structure.

[0098] S280: Obtain a fused feature map containing multi-topology information based on the feature map and the initial classification result through the TKRM module.

[0099] The TKRM module is responsible for combining the feature map with the initial classification results and introducing multi-topological information. Multi-topological information may refer to the representation of fetal anatomical structures at different scales, directions, or morphologies. The TKRM module may use some form of graph convolutional network (GCN) or other topological learning techniques to capture this multi-topological information. After processing by the TKRM module, a new fused feature map is obtained, which not only contains the information of the original feature map, but also integrates multi-topological information and the initial classification results.

[0100] In one of the embodiments, based on the feature map, the candidate box and the first detection head in the initial deep learning model, obtaining the initial classification result of the fetal anatomical structure includes: using the ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate area corresponding to the candidate box to the feature map, so as to pool the feature map into a regional feature map of uniform size; sending the regional feature map to the fully connected layer to obtain a feature map of fixed size; inputting the fixed-size feature map into the first detection head in the initial deep learning model to obtain the initial classification result of the fetal anatomical structure and the coordinates of the key structure positions.

[0101] In the initial deep learning model, there is usually a region proposal network (RPN) to generate candidate boxes (also called regions of interest ROI or anchors). These candidate boxes are potential areas that may contain fetal anatomical structures. The role of the ROI pooling layer is to map the candidate regions corresponding to the candidate boxes of different sizes from the feature map and pool them into a regional feature map of uniform size. This is because the fully connected layer (or subsequent convolutional layer) requires a fixed-size input. Specifically, for each candidate box, the ROI pooling layer maps it to the feature map and divides the area into a fixed number of grids (such as 7x7), and then performs a maximum pooling operation within each grid to obtain a regional feature map of a fixed size (such as 7x7). The regional feature map obtained by ROI pooling is fed into the fully connected layer (or a series of convolutional layers). These layers further extract and integrate features to obtain a fixed-size feature vector. The fixed-size feature vector obtained above is input to the first detection head in the initial deep learning model. This detection head usually contains several fully connected layers (or convolutional layers) for classification and bounding box regression. For classification tasks, the detection head outputs the probability of different fetal anatomical structure categories corresponding to each candidate box (i.e., the initial classification result). For bounding box regression tasks, the detection head outputs the coordinate offset of each candidate box, which is used to adjust the position and size of the candidate box to more accurately locate the key structural positions of the fetal anatomical structure. The initial classification results of the fetal anatomical structure corresponding to each candidate box and the coordinates of the key structural positions are obtained.

[0102] In one embodiment, obtaining a fused feature map containing multiple topological information based on a feature map and an initial classification result through a TKRM module includes: inputting a fixed-size feature map, an initial classification result, and key structure position coordinates into the TKRM module to obtain a fused feature map containing multiple topological information.

[0103] The fixed-size feature map, initial classification results, and key structure location coordinates are fed as input to the TKRM module. The TKRM module may contain multiple submodules or processing steps to extract and integrate multi-topological information. Multi-topological information may refer to the representation of fetal anatomical structures at different scales, orientations, or morphologies. Within the TKRM module, graph convolutional networks (GCNs), attention mechanisms, or other topological learning techniques may be used to process the input information. For example, GCNs can be used to model spatial relationships between anatomical structures, or attention mechanisms can be used to focus on important feature areas. The TKRM module may also combine the initial classification results and key structure location coordinates to integrate this information into the feature map in some way to generate a richer fused feature map.

[0104] In one embodiment, according to the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head, obtaining the first classification score includes: inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing; and weightedly summing the initial classification result output by the first detection head and the weighted classification score map to obtain the first classification score.

[0105] First, there is a region feature map obtained by ROI pooling, which corresponds to the mapping area of ​​the candidate box on the feature map and is pooled to a uniform size. Next, there is a fixed-size feature map, which is the result of further processing of the region feature map by a fully connected layer or a convolutional layer. Finally, there is the initial classification result output by the first detection head, which represents the probability that each candidate box belongs to a different category. The region feature map and the fixed-size feature map are taken as input and fed into the DAEFLM module. The role of the DAEFLM module is to dynamically adjust the weights of different regions according to the input feature map to generate a weighted classification score map. This score map may highlight the key areas in the feature map related to the fetal anatomy and reduce the weights of irrelevant areas. Inside the DAEFLM module, attention mechanisms or other techniques may be used to dynamically generate weights and apply them to the input feature map to obtain a weighted classification score map. The initial classification result output by the first detection head is weighted summed with the weighted classification score map generated by the DAEFLM module. This step combines the classification information from the detection head and the weighted feature information from the DAEFLM module to obtain a more accurate and rich first classification score. The specific method of weighted summation may include element-by-element multiplication of the initial classification result and the weighted classification score map, and then summing or averaging, depending on the design and needs of the model. After weighted summation, the first classification score is obtained. This score reflects the possibility that the candidate box contains a specific fetal anatomical structure and integrates information from multiple modules.

[0106] Overall, if Figure 4As shown, the model training in the small sample detection method of fetal anatomical structure based on ultrasound image of the present application includes two stages: in the first stage, a1) obtains the standard ultrasound image of the fetus; a2) uses backbone to extract features of the data set obtained in step (a1) to obtain a feature map, uses RPN network to extract candidate boxes on the obtained feature map, uses ROI pooling layer to obtain feature map and extracted candidate boxes, maps the candidate area corresponding to the candidate box to the feature map, and pools the feature map into a regional feature map of uniform size, sends the regional feature map to the fully connected layer, thereby obtaining a fixed size feature map, inputs the feature map into the detection head, and after passing through the two branches of the fully connected layer in the detection head, the classification of the fetal anatomical structure and the precise position coordinates of the key structure are obtained; at the same time, the feature map with the candidate area and the pooled feature map are input into the DAEFLM module, and then a weighted classification score map is obtained after passing through a classification head, and the output of the detection head is weightedly added with the weighted classification score map to finally obtain the category score. In the second stage, the TKRM module and a new detection head are added, and then the other components are frozen for training. Based on the first stage, the pooled feature map and the output of the detection head in the first stage are input into TKRM to obtain a fused feature map containing more topological information, which is then sent to the new detection head to obtain new classification scores and region boundaries. Finally, the weighted addition is performed with the category scores of stage one to obtain the final category score.

[0107] In order to further illustrate the model / module-related architecture and specific functional implementation details involved in the small sample detection method of fetal anatomical structure based on ultrasound images in this application, some related contents will be introduced again below.

[0108] 1. Topological knowledge reasoning

[0109] The candidate regions generated by RPN are used as nodes, and nodes with high feature similarity are selected as adjacent nodes to create an undirected graph. We use the distance and angle between candidate centers to establish topological information. This topological information is then passed to the Gaussian kernel to generate relationship weights between adjacent nodes. Subsequently, we diffuse and aggregate the information in the GNN, enabling the GNN to learn topological knowledge in the image and generate features that contain topological knowledge.

[0110] 1) Initialization of graph node representation

[0111] The weight of the classifier in the detection head is recorded as W∈R (C+1)×D , the classification score of the candidate region output by the classifier and the softmax function is recorded as S ori∈R N×(C+1) Where N is the number of candidate regions, C is the number of classes, and D is the feature dimension of the classifier input. Since W contains high-level semantic information, it is used as the visual embedding representation of the candidate region. In order to map the visual embedding into the graph node representation of the candidate region, the classification score S ori Multiply it with the classifier weight W to get the initial graph node X = S ori W,X∈R N×D .

[0112] 2) Graph node adjacency construction

[0113] The features obtained from RoI Align After passing through two fully connected layers and one ReLU activation layer, feature Z is obtained. T Multiply to get the feature similarity matrix M=ZZ between the candidate region nodes T The feature similarity between any two candidate regions, node i and node j, is given by M ij Given. When the number of candidate regions is large, if all nodes are fully connected, the amount of calculation will be large, which is not conducive to model training. Therefore, for each node, only the Kth node with the highest feature similarity is selected as the adjacent node:

[0114] Neighbour(i)=TOP k (M i1 , M i2 , …, M iN ).

[0115] 3) Topological structure information construction

[0116] First, we need to construct topological information so that GNN can learn the topological knowledge in ultrasound images. The topology is built based on the spatial relative positions between target objects in the image, and can measure distances and angles. The distances and angles between candidate regions are calculated based on the center points of the rectangular box boundaries generated by the detection head:

[0117]

[0118] Where (x i ,y i )) and (x j ,y j ) represent the centers of the two candidate region bounding boxes respectively. The calculation function of the relative position relationship between the two proposed region boxes is recorded as R(i, j) = (d, θ).

[0119] 4) Topological knowledge reasoning

[0120] After constructing the network topology, the spatial relationship between adjacent nodes is used to help GNN learn the topological relationship in ultrasound images. We regard the relative position distribution between different regions as a Gaussian distribution and use a Gaussian kernel to convert the relative position into a scalar. These scalars are used as relationship weights in the information propagation and aggregation process of GNN. Therefore, the model can learn the topological knowledge in ultrasound images by learning these distributions. Given a graph node i with adjacent nodes, the information propagation and aggregation operations of the node can be expressed as:

[0121] x′ im =∑ j∈Neighbour(i) ω m (R(i,j))x j ,

[0122]

[0123] μ′ m =R(i, j)-μ m ,

[0124] In the above formula, x j is the feature of the adjacent node j, ω m is the mth Gaussian kernel, μ m is a learnable 2×1 mean vector, Σ m is a learnable 2×2 covariance matrix. For m new features, these features are concatenated and input into a linear transformation layer to obtain the new feature x′ of node i. i .

[0125] After information propagation and aggregation through this operation, GNN learns the topological knowledge in the ultrasound image and outputs node representations that carry this topological knowledge. These representations combine local and global information and significantly enhance the topological structure information. Topological structure information can help the model perform structured reasoning and prediction, and impose constraints on the classification and positioning of new category objects. This information helps to mitigate the impact of low-quality candidate regions on new categories and improve the prediction accuracy of candidate region classification and positioning. These candidate region node representations are connected to the original feature f through a linear transformation layer and a ReLU layer. The resulting fused features are input into a new detection head to produce the final classification score and region boundaries.

[0126] 2. Feature learning to enhance discriminative ability

[0127] The feature learning process of Faster-RCNN includes multiple stages, including feature learning of the backbone network, RoIPooling, etc. Our main focus is on learning discriminative features in the RoI Align and RoI Pooling stages. Although traditional RoI Align and RoI Pooling methods can meet most object detection scenarios, their sampling strategies may be more suitable for extracting discriminative features. Therefore, in the RoI Align and RoI Pooling processes, we apply deformable pooling to resample features in different regions, rather than just limited to candidate regions. We use weighted pooling to adaptively assign higher weights to discriminative sampling points to extract discriminative features at a fine-grained scale.

[0128] 1) Sub-region resampling

[0129] Input a feature map X and a candidate region P, the candidate region is divided into k×k sub-regions, and then these sub-regions are average pooled to obtain the feature map X1∈R C×W×H , where C represents the number of channels, W represents the width of the image, and H represents the height of the image. Then, X1 passes through three fully connected layers and two ReLU layers alternately to obtain the sub-region offset predictions. The prediction results are then processed using bilinear interpolation. Position-sensitive RoI Pooling is used to combine the feature map X and the candidate region P with these offset predictions to obtain the new feature map X new ∈R C×4W×4H This process helps to incorporate rich local context information into the new feature map X new In this approach, the impact of low-quality proposals generated for new categories is reduced because the locations of the resampled subregions are not constrained but are related to those of the original proposals.

[0130] 2) Adaptive weighted pooling

[0131] In order to obtain the pooling result, we transform the feature map X new It is divided into 2k×2k sub-regions, each of which contains 4 points. We assign different weights to each point in the sub-region. First, each sub-region passes through a convolutional layer, a normalization layer, and an activation layer to generate a matrix containing the corresponding weights of each point in the sub-region.

[0132] Then, the weight matrix corresponding to each sub-region is used as the convolution kernel. After performing the convolution operation on the sub-region, a new feature map X2∈R is obtained. C×2W×2H Finally, we perform average pooling on X2 to obtain the feature map X dis ∈R C×W×H .

[0133] The X of all candidate regionsdis It is concatenated with the original pooled feature f and input into a fully connected layer and a separate classification head to generate a new classification score map S dis , and compare it with the original classification score map S ori After weighted addition, the final classification score graph S is obtained. final :

[0134] S final =(1-γ)S ori +γS dis ,

[0135] In this formula, γ represents the classification score map S dis The weight of .

[0136] 3. Training strategy

[0137] The basic framework used in this application is DeFRCN, which is a transfer learning-based method with two training stages: training for basic classes and fine-tuning for all classes. In order to make the GNN in TKRM more effective in learning the relative position relationship between anatomical structures, we hope to train this module after obtaining high-quality high-level features, classification scores, and the boundaries of rectangular boxes, so that the model can focus on the training of TKRM and accelerate the convergence of TKRM parameters. Therefore, each of the original two training stages is subdivided into two sub-stages. The first sub-stage does not include TKRM, while the second sub-stage includes this module and freezes other components for training.

[0138] 4. Loss Function

[0139] We use the standard Faster-RCNN loss function, which consists of a classification loss L rpn_cls and the rectangular box regression loss L rpn_loc The RPN loss is composed of the classification loss L cls and the rectangular box regression loss L box_reg The detector loss composed of RPN loss and detector loss is:

[0140] L total =L rpn_cls +L rpn_loc +L cls +L box_reg .

[0141] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a fetal anatomical structure small sample detection device based on ultrasound images for implementing the fetal anatomical structure small sample detection method based on ultrasound images involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more fetal anatomical structure small sample detection device embodiments based on ultrasound images provided below can refer to the limitations of the fetal anatomical structure small sample detection method based on ultrasound images above, and will not be repeated here.

[0143] In one embodiment, Figure 5 As shown, a small sample detection device for fetal anatomical structure based on ultrasound images is provided.

[0144] The data acquisition module 100 is used to acquire the initial deep learning model and the standard ultrasound image of the fetus, and extract the features of the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and the TKRM module and the DAEFLM module are added to the basic framework;

[0145] The first processing module 200 is used to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map and through the TKRM module, obtain a fused feature map containing multi-topology information;

[0146] The second processing module 300 is used to obtain a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; perform weighted summation of the first classification score and the second classification score to obtain a target classification score;

[0147] A loss function construction module 400 is used to construct a standard Faster-RCNN loss function according to the target classification score and the classification result corresponding to the standard ultrasound image, and the detection frame;

[0148] A model training module 500 is used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model;

[0149] The detection module 600 is used to perform a small sample detection of the fetal anatomical structure according to the target deep learning model.

[0150] In one embodiment, the first processing module 200 is also used to extract candidate boxes of the feature map; based on the feature map, the candidate boxes and the first detection head in the initial deep learning model, an initial classification result of the fetal anatomical structure is obtained; according to the feature map, the candidate boxes, and the initial classification result, and through the DAEFLM module and the first detection head, a first classification score is obtained; according to the feature map and the initial classification result, and through the TKRM module, a fused feature map containing multi-topological information is obtained.

[0151] In one embodiment, the first processing module 200 is further configured to extract a candidate box on the obtained feature map using an RPN network.

[0152] In one of the embodiments, the first processing module 200 is also used to use the ROI pooling layer to obtain the feature map and the extracted candidate box, map the candidate area corresponding to the candidate box into the feature map, and pool the feature map into a regional feature map of uniform size; send the regional feature map to the fully connected layer to obtain a feature map of a fixed size; input the fixed-size feature map into the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the coordinates of the key structure positions.

[0153] In one embodiment, the first processing module 200 is further used to input the fixed-size feature map, the initial classification result, and the key structure position coordinates into the TKRM module to obtain a fused feature map containing multi-topology information.

[0154] In one embodiment, the first processing module 200 is also used to input the regional feature map and the fixed-size feature map into the DAEFLM module, and after classification processing, a weighted classification score map is obtained; the initial classification result output by the first detectionhead and the weighted classification score map are weightedly summed to obtain the first classification score.

[0155] Each module in the above-mentioned fetal anatomical structure small sample detection device based on ultrasound image can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0156] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a small sample detection method of fetal anatomical structure based on ultrasound images is implemented.

[0157] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned method for detecting a small sample of fetal anatomical structure based on ultrasound images is implemented.

[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting a small sample of fetal anatomical structure based on ultrasound images is implemented.

[0160] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the above-mentioned method for detecting a small sample of fetal anatomical structure based on ultrasound images.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting a small sample of fetal anatomical structure based on ultrasound images, characterized in that: The method comprises: An initial deep learning model and a standard ultrasound image of a fetus are obtained, and features of the standard ultrasound image are extracted to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework; Based on the feature map, a first classification score is obtained through the DAEFLM module and the first detection head in the initial deep learning model; based on the feature map, a fused feature map containing multi-topology information is obtained through the TKRM module; Obtaining a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; performing a weighted summation of the first classification score and the second classification score to obtain a target classification score; Constructing a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame; Adjusting the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and taking the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model; A small sample detection of fetal anatomical structure is performed based on the target deep learning model.

2. The method according to claim 1, characterized in that The first classification score is obtained based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; The fused feature map containing multi-topology information obtained based on the feature map and through the TKRM module includes: Extracting a candidate box of the feature map; Obtaining an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model; Obtaining a first classification score according to the feature map, the candidate box, and the initial classification result, and through the DAEFLM module and the first detection head; According to the feature map and the initial classification result, a fused feature map containing multi-topology information is obtained through the TKRM module.

3. The method according to claim 2, characterized in that The extracting of the candidate frame of the feature map comprises: The RPN network is used to extract candidate boxes on the obtained feature map.

4. The method according to claim 2, characterized in that: The obtaining of an initial classification result of the fetal anatomical structure based on the feature map, the candidate box, and the first detection head in the initial deep learning model includes: Using the ROI pooling layer to obtain the feature map and the extracted candidate box, mapping the candidate area corresponding to the candidate box to the feature map, so as to pool the feature map into a regional feature map of uniform size; Sending the regional feature map to a fully connected layer to obtain a feature map of a fixed size; The fixed-size feature map is input into the first detection head in the initial deep learning model to obtain the initial classification results of the fetal anatomical structure and the coordinates of the key structure positions.

5. The method according to claim 4, characterized in that The step of obtaining a fused feature map containing multi-topology information according to the feature map and the initial classification result and through the TKRM module includes: The fixed-size feature map, the initial classification result, and the key structure position coordinates are input into the TKRM module to obtain a fused feature map containing multi-topology information.

6. The method according to claim 4, characterized in that The obtaining of a first classification score according to the feature map, the candidate box, and the initial classification result through the DAEFLM module and the first detection head includes: Inputting the regional feature map and the fixed-size feature map into the DAEFLM module, and obtaining a weighted classification score map after classification processing; A first classification score is obtained by weightedly summing the initial classification result output by the first detection head and the weighted classification score map.

7. A small sample detection device for fetal anatomical structure based on ultrasound images, characterized in that: The device comprises: A data acquisition module is used to acquire an initial deep learning model and a standard ultrasound image of a fetus, and extract features from the standard ultrasound image to obtain a feature map; the basic framework of the initial deep learning model is DeFRCN, and a TKRM module and a DAEFLM module are added to the basic framework; A first processing module, configured to obtain a first classification score based on the feature map and through the DAEFLM module and the first detection head in the initial deep learning model; and obtain a fused feature map containing multi-topology information based on the feature map and through the TKRM module; A second processing module is used to obtain a second classification score and a detection frame according to the fused feature map and through the second detection head in the initial deep learning model; and perform weighted summation on the first classification score and the second classification score to obtain a target classification score; A loss function construction module, used to construct a standard Faster-RCNN loss function according to the target classification score, the classification result corresponding to the standard ultrasound image, and the detection frame; A model training module, used to adjust the model parameters of the initial deep learning model based on the Faster-RCNN loss function, and use the model corresponding to the minimum Faster-RCNN loss function as the target deep learning model; A detection module is used to perform small sample detection of fetal anatomical structure according to the target deep learning model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images