A standard section detection system and method for early pregnancy fetal congenital heart disease

Abstract

The present application proposes a system and method for detecting standard sections of fetal congenital heart disease in early pregnancy, which relates to the field of medical image processing technology. The system includes a semantic segmentation model, an ultrasound plane classification module, a blood flow classification module, a co-occurrence knowledge graph module, an expert knowledge scoring module and a standard section classification module. Among them, the semantic segmentation model locates key anatomical structures; the ultrasound plane classification module and the blood flow classification module classify sections and blood flows respectively; the co-occurrence knowledge graph module determines the symbiotic relationship of anatomical structures; the expert knowledge scoring module outputs scores in combination with preset rules; the standard section classification module integrates the results of each module to obtain the first classification result of the standard section, and obtains the second classification result in combination with the expert score. This scheme can realize automatic and accurate detection of standard sections of fetal congenital heart disease in early pregnancy by combining technologies such as semantic segmentation, ultrasound plane classification, blood flow classification, co-occurrence knowledge graph and expert knowledge scoring.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a system and method for detecting standard sections of fetal congenital heart disease in early pregnancy. Background Art

[0002] In early pregnancy fetal ultrasound examination, accurate identification of standard sections of congenital heart disease is crucial for early diagnosis and treatment of the disease. However, due to the complexity of fetal anatomical structure and the uncertainty of ultrasound examination, accurate detection of standard sections has always been a challenge in clinical practice. Therefore, it is of great clinical value to develop a system that can automatically and accurately detect standard sections of congenital heart disease in early pregnancy. Summary of the invention

[0003] The purpose of this application is to provide a system and method for detecting standard cross-sections of fetal congenital heart disease in early pregnancy, which realizes automatic and accurate detection of standard cross-sections of fetal congenital heart disease in early pregnancy by combining technologies such as semantic segmentation, ultrasound plane classification, blood flow classification, co-occurrence knowledge graph and expert knowledge scoring.

[0004] This application is implemented as follows:

[0005] In a first aspect, the present application provides a standard section detection system for early pregnancy fetal congenital heart disease, comprising:

[0006] A semantic segmentation model is used to segment and locate the key anatomical structures required for standard section detection of early pregnancy fetal congenital heart disease from an input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section. An ultrasound plane classification module is used to perform section classification processing based on the key anatomical structures segmented and located to obtain section classification results. A blood flow classification module is used to perform blood flow classification processing based on the key anatomical structures segmented and located to obtain blood flow classification results. A co-occurrence knowledge graph module is used to determine the co-occurrence matrix diagram of the key anatomical structures based on the key anatomical structures segmented and located; and to determine the blood flow-key anatomical structure co-occurrence matrix based on the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results. An expert knowledge scoring module is used to integrate the key anatomical structures, section classification results and blood flow classification results according to preset expert knowledge rules to output corresponding expert scoring results. The standard section classification module is used to obtain the first classification result of the standard section of early pregnancy fetal congenital heart disease based on the key anatomical structures determined by segmentation, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, and to obtain the second classification result of the standard section of early pregnancy fetal congenital heart disease based on the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease.

[0007] Furthermore, based on the aforementioned scheme, the semantic segmentation model is further used to fuse the input two-dimensional grayscale image and color Doppler image of the same section, and segment the key anatomical structures required for standard section detection of fetal congenital heart disease in early pregnancy based on the image obtained by the fusion process.

[0008] Further, based on the above scheme, the semantic segmentation model includes an encoder network unit and a key anatomical structure segmentation network unit; the ultrasound plane classification module includes a first fully connected layer, a second fully connected layer, a third fully connected layer and a fourth fully connected layer; the number of channels of the fourth fully connected layer is consistent with the number of types of section classification. The first fully connected layer is calculated by the last encoder module of the encoder network unit, and the second fully connected layer is calculated by the last decoder module of the key anatomical structure segmentation network; the third fully connected layer is calculated by cascading the first fully connected layer and the second fully connected layer, and the fourth fully connected layer is calculated by the third fully connected layer.

[0009] Furthermore, based on the aforementioned scheme, the blood flow classification module includes a fifth fully connected layer, a sixth fully connected layer and a seventh fully connected layer; the fifth fully connected layer is calculated by the last encoder module of the encoder network unit, the sixth fully connected layer is calculated by the fifth fully connected layer, and the sixth fully connected layer is calculated to obtain three fully connected layers respectively, and these three fully connected layers are cascaded to obtain the seventh fully connected layer.

[0010] Further, based on the above scheme, the determining of the co-occurrence matrix diagram of the key anatomical structure according to the key anatomical structure located by segmentation includes: performing multiplication and addition operations on any two values ​​on the predicted image carrying the key anatomical structure located by segmentation, so as to determine the category interaction diagram according to the calculated interaction value. The co-occurrence matrix of the key anatomical structure is determined according to the category interaction diagram and the dictionary matrix; the value of the dictionary matrix is ​​obtained by the interaction of the category values ​​of the key anatomical structure.

[0011] Further, based on the aforementioned scheme, the second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained according to the expert scoring result and the first classification result of the standard section of early pregnancy fetal congenital heart disease, including: when the expert scoring result and the first classification result of the standard section of early pregnancy fetal congenital heart disease both indicate that the section corresponding to the input image is a standard section, the second classification result is determined to be that the section is a standard section. When the first classification result of the standard section of early pregnancy fetal congenital heart disease indicates that the section corresponding to the input image is a standard section, and the expert scoring result cannot confirm that the section is a standard section, the second classification result is that congenital heart disease is considered to exist.

[0012] Furthermore, based on the aforementioned solution, the system adopts a focal loss function when calculating the loss to balance the data between the standard slice and the non-standard slice.

[0013] In a second aspect, the present application provides a method for detecting standard sections of congenital heart disease in early pregnancy fetuses, which comprises the following steps:

[0014] The key anatomical structures required for the detection of standard sections of early pregnancy fetal congenital heart disease are segmented and located from the input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section. Section classification processing is performed according to the segmented and located key anatomical structures to obtain section classification results. Blood flow classification processing is performed according to the segmented and located key anatomical structures to obtain blood flow classification results. According to the segmented and located key anatomical structures, a co-occurrence matrix diagram of the key anatomical structures is determined; and according to the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results, a blood flow-key anatomical structure co-occurrence matrix is ​​determined. According to the preset expert knowledge rules, the key anatomical structures, the section classification results and the blood flow classification results are integrated to output the corresponding expert scoring results. According to the segmented key anatomical structures, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, a first classification result of the standard section of early pregnancy fetal congenital heart disease is obtained, and according to the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease, a second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained.

[0015] In a third aspect, the present application provides an electronic device, comprising a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, a method as described in any one of the second aspects is implemented.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method as described in any one of the above-mentioned second aspects.

[0017] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0018] This application proposes a standard section detection system for early pregnancy fetal congenital heart disease. Through data interaction between multiple modules, the system can comprehensively utilize various information in the image, such as anatomical structure, blood flow status, etc., so as to more accurately perform section classification and abnormality detection. The interaction between different modules forms a complete processing flow. Each module can verify and supplement the output of other modules, thereby enhancing the robustness of the system and enabling it to better cope with various complex ultrasound images. Among them, the co-occurrence knowledge graph module can construct a symbiotic relationship between key anatomical structures and blood flow. This knowledge can be shared by other modules within the system, and can also be used for subsequent medical research to promote the accumulation and inheritance of knowledge. The output of the expert knowledge scoring module can provide doctors with additional decision support information, helping doctors to have a more comprehensive understanding of the fetal heart condition, so as to make a more accurate diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is a schematic diagram of the framework of a standard section detection system for early pregnancy fetal congenital heart disease in one embodiment of the present application;

[0021] Figure 2 The grayscale section of the four-chamber heart, the blood flow section of the four-chamber heart and their annotated images in one embodiment of the present application;

[0022] Figure 3 It is a dictionary diagram in one embodiment of the present application;

[0023] Figure 4 A co-occurrence matrix diagram of key anatomical structures in an embodiment of the present application;

[0024] Figure 5 The co-occurrence relationship of key anatomical structures in one embodiment of the present application;

[0025] Figure 6 This is a flow chart of an embodiment of a method for detecting congenital heart disease in a fetus during early pregnancy according to the present application;

[0026] Figure 7 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0027] Icon: 201, processor; 202, memory; 203, communication interface. DETAILED DESCRIPTION

[0028] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all the embodiments. The components of the embodiment of the present application described and shown in the drawings here can be arranged and designed in various different configurations. In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. At the same time, the naming or numbering of the steps in the present application does not mean that the steps in the method flow must be executed in the time / logic sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0029] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0030] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] Example 1

[0032] An embodiment of the present application provides a system for detecting standard sections of fetal congenital heart disease in early pregnancy, wherein the detection of standard sections of fetal congenital heart disease in early pregnancy is implemented in two stages: the first stage adopts a multi-task learning strategy to synchronously locate the key anatomical structures required for the detection of standard sections of fetal congenital heart disease in early pregnancy, and preliminarily detects the type of the current section and whether the section is standard; the second stage uses the key anatomical structure segmentation results of the first stage as input, constructs a symbiotic relationship diagram, and introduces expert knowledge to automatically identify standard sections of congenital heart disease in early pregnancy with high interpretability.

[0033] See also Figure 1 The standard section detection system for early pregnancy fetal congenital heart disease includes a semantic segmentation model, an ultrasound plane classification module, a blood flow classification module, a co-occurrence knowledge graph module, an expert knowledge scoring module and a standard section classification module.

[0034] Among them, the semantic segmentation model is used to segment and locate the key anatomical structures required for the standard section detection of early pregnancy fetal congenital heart disease from the input image; the key anatomical structures include the four-chamber heart section and the three-vessel tracheal section. The ultrasound plane classification module is used to perform section classification processing according to the key anatomical structures segmented and located to obtain section classification results. The blood flow classification module is used to perform blood flow classification processing according to the key anatomical structures segmented and located to obtain blood flow classification results. The co-occurrence knowledge graph module is used to determine the co-occurrence matrix diagram of the key anatomical structures based on the key anatomical structures segmented and located; and determine the blood flow-key anatomical structure co-occurrence matrix based on the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results. The expert knowledge scoring module is used to integrate the key anatomical structures, section classification results and blood flow classification results according to preset expert knowledge rules to output the corresponding expert scoring results. The standard section classification module is used to obtain the first classification result of the standard section of early pregnancy fetal congenital heart disease based on the key anatomical structures determined by segmentation, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, and to obtain the second classification result of the standard section of early pregnancy fetal congenital heart disease based on the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease.

[0035] During the working process of the early pregnancy fetal congenital heart disease standard section detection system, the input ultrasound image data will be received first, and these data will serve as the basis for all subsequent module processing. That is, the semantic segmentation model receives ultrasound image data (grayscale image and Doppler image of the same section), performs semantic segmentation on the image, identifies and extracts key anatomical structures, such as the heart and blood vessels, and outputs the segmentation results to other modules. Among them, the key anatomical structures mainly include the left atrium, right atrium, left ventricle, right ventricle, mitral valve blood flow, tricuspid valve blood flow, aortic arch and ductus arteriosus in the four-chamber heart section and the three-vessel tracheal section.

[0036] Next, the ultrasound plane classification module and the blood flow classification module respectively receive the key anatomical structure information output by the semantic segmentation model. The ultrasound plane classification module performs plane classification on the ultrasound image according to the key anatomical structure information, while the blood flow classification module realizes the key anatomical structure segmentation of the four-chamber heart view and the three-vessel tracheal view through the segmentation network according to the key anatomical structure information.

[0037] For example, Deeplab v3 can be used as a segmentation network to achieve the segmentation of key anatomical structures in the four-chamber heart view and the three-vessel tracheal view. In the encoder network part, ResNet50 is used as the backbone network of the encoder network part. Its output is the encoder feature, the decoder feature and the segmentation prediction layer feature of the key anatomical structure, and the segmentation prediction layer feature can output the segmentation result of the key anatomical structure.

[0038] Next, the co-occurrence knowledge graph module receives the output results of the ultrasound plane classification module and the blood flow classification module, and based on this information, constructs a co-occurrence knowledge graph between key anatomical structures and blood flows, and outputs the graph information to the expert knowledge scoring module and the standard section classification module.

[0039] Finally, the expert knowledge scoring module receives the image information output by the co-occurrence knowledge graph module, and scores the key anatomical structures and blood flow status in the image based on the preset expert knowledge rules, and outputs the scoring results to the standard section classification module. The standard section classification module receives the output results of the ultrasound plane classification module, the co-occurrence knowledge graph module, and the expert knowledge scoring module, and integrates this information to perform the final standard section classification on the input ultrasound image.

[0040] In summary, the standard section detection system for early pregnancy fetal congenital heart disease can make comprehensive use of various information in the image, such as anatomical structure, blood flow status, etc., through data interaction between multiple modules, so as to more accurately perform section classification and abnormality detection. The interaction between different modules forms a complete processing flow, and each module can verify and supplement the output of other modules, thereby enhancing the robustness of the system and enabling it to better cope with various complex ultrasound images. Among them, the co-occurrence knowledge graph module can construct a symbiotic relationship between key anatomical structures and blood flow. This knowledge can be shared by other modules within the system, and can also be used for subsequent medical research to promote the accumulation and inheritance of knowledge. The output of the expert knowledge scoring module can provide doctors with additional decision support information, helping doctors to have a more comprehensive understanding of the fetal heart condition and make more accurate diagnoses.

[0041] In one implementation of the present application, the semantic segmentation model is further used to fuse the input two-dimensional grayscale image and color Doppler image of the same section, and segment the image obtained by the fusion process to locate the key anatomical structure required for the standard section detection of early pregnancy fetal congenital heart disease, thereby more accurately locating the key anatomical structure required for the standard section detection of early pregnancy fetal congenital heart disease. The introduction of this method significantly improves the detection accuracy and reliability of the system.

[0042] Among them, the two-dimensional grayscale image shows the basic structure and morphology of the fetal heart, while the color Doppler image provides blood flow information, which is helpful for analyzing heart function and abnormal conditions. The semantic segmentation model first preprocesses the input two-dimensional grayscale image and color Doppler image, such as denoising and enhancement, to improve image quality. Then, the model uses appropriate image fusion techniques, such as pixel-level fusion or feature-level fusion, to fuse the information of the two images together. The fused image contains both morphological structure information and blood flow information, providing a richer data basis for subsequent segmentation. Finally, based on the fused image, the semantic segmentation model uses a deep learning algorithm to segment key anatomical structures. By learning a large amount of labeled training data, the model can identify and extract key anatomical structures related to the standard section detection of fetal congenital heart disease in early pregnancy, such as the four chambers of the heart and blood vessels.

[0043] By fusing the information of two-dimensional grayscale images and color Doppler images, the system can have a more comprehensive understanding of the structure and function of the fetal heart. This helps to locate key anatomical structures more accurately and improve the accuracy and reliability of detection. In addition, the fusion of multiple image information enables the system to better cope with image quality differences and noise interference in different situations. Even in some cases of poor image quality, the system can improve the accuracy and stability of segmentation by fusing multiple information. For example, Figure 2 As shown, in the actual processing process, the two-dimensional grayscale image and color Doppler image of the same section can be spliced ​​left to right first, then right to left, and finally the two spliced ​​images are stacked as the input of the neural network, so that the convolutional neural network can automatically learn the anatomical organ information of the two-dimensional grayscale image and the functional blood flow information of the color Doppler image, thereby outputting a fused image.

[0044] In one implementation of the present application, the semantic segmentation model includes an encoder network unit and a key anatomical structure segmentation network unit; the ultrasound plane classification module includes a first fully connected layer, a second fully connected layer, a third fully connected layer and a fourth fully connected layer; the number of channels of the fourth fully connected layer is consistent with the number of types of section classification. The first fully connected layer is calculated by the last encoder module of the encoder network unit, and the second fully connected layer is calculated by the last decoder module of the key anatomical structure segmentation network; the third fully connected layer is calculated by cascading the first fully connected layer and the second fully connected layer, and the fourth fully connected layer is calculated by the third fully connected layer.

[0045] In the above implementation, by considering the global features extracted from the entire image and the local features for the key anatomical structures at the same time, the ultrasound plane classification module can more accurately determine the plane type of the ultrasound image. Due to the combination of the precise anatomical structure information provided by the semantic segmentation model, the ultrasound plane classification module can more accurately identify the standard section, providing strong support for subsequent disease detection. Specifically, the encoder network unit is responsible for extracting multi-scale feature information from the input image. These features may include shape, texture, edge, etc., which are crucial for subsequent image segmentation. The encoder network unit usually contains multiple convolutional layers, pooling layers, and possible normalization layers and activation functions to gradually reduce the spatial dimension and increase the number of feature channels. The key anatomical structure segmentation network unit receives the feature map output by the encoder network unit and generates a pixel-level segmentation result. In actual application, it includes an upsampling layer (such as deconvolution or transposed convolution), a jump connection (for combining low-level and high-level feature information) and a final convolutional layer to generate a segmentation map. The output segmentation map distinguishes key anatomical structures (such as heart, blood vessels, etc.) from the background, providing a basis for subsequent ultrasound plane classification and congenital heart disease detection.

[0046] For the ultrasound plane classification module, the first fully connected layer is connected to the output of the encoder network unit, which means that it receives global features extracted from the entire input image. The second fully connected layer is connected to the output of the key anatomical structure segmentation network unit, and it receives segmentation features for specific anatomical structures. The feature vectors output by the first and second fully connected layers are concatenated and then passed to the third fully connected layer. This step realizes the fusion of global and local features, which helps to more accurately determine the plane type of the ultrasound image. The output of the third fully connected layer is further passed to the fourth fully connected layer, whose number of channels is consistent with the number of types of section classification. This means that each output channel of the fourth fully connected layer corresponds to a specific ultrasound plane category, and the probability distribution of each category can be obtained through the softmax activation function.

[0047] Furthermore, in one implementation of the present application, the blood flow classification module includes a fifth fully connected layer, a sixth fully connected layer and a seventh fully connected layer; the fifth fully connected layer is calculated by the last encoder module of the encoder network unit, the sixth fully connected layer is calculated by the fifth fully connected layer, and the sixth fully connected layer is calculated to obtain three fully connected layers respectively, and these three fully connected layers are cascaded to obtain the seventh fully connected layer.

[0048] In the above implementation, the fifth fully connected layer receives the output from the encoder network unit as input, and further processes and integrates these features to generate a feature vector suitable for blood flow classification. Then, the output of the fifth fully connected layer is connected in parallel to multiple sixth fully connected layers. This parallel structure allows the model to learn multiple different feature representations at the same time. Each sixth fully connected layer may focus on different blood flow features or patterns, and can also enhance the diversity and flexibility of the model, so that it can better adapt to complex blood flow classification tasks. The output of each sixth fully connected layer is connected to a seventh fully connected layer. The function of the seventh fully connected layer is to further map the specific blood flow features extracted by the sixth fully connected layer to the final classification result. Exemplarily, the output of the seventh fully connected layer may be a probability distribution processed by a softmax function, indicating the probability that the input image belongs to different blood flow categories.

[0049] In the above implementation, through multiple parallel sixth fully connected layers, the model can learn more diverse blood flow features, thereby enhancing the ability to recognize complex blood flow patterns. In addition, since each sixth fully connected layer focuses on different blood flow features, combining their results for the final classification decision can improve the accuracy and robustness of the classification. The model's capabilities can also be further expanded to adapt to more diverse blood flow classification tasks by adding more parallel sixth fully connected layers or adjusting network parameters.

[0050] In an implementation of the present application, determining a co-occurrence matrix of the key anatomical structure based on the segmented and located key anatomical structure includes:

[0051] Multiplication and addition operations are performed on any two values ​​on the predicted image carrying the key anatomical structure located by segmentation to determine a category interaction graph based on the calculated interaction values; a co-occurrence matrix of the key anatomical structure is determined based on the category interaction graph and a dictionary matrix; the values ​​of the dictionary matrix are obtained by interacting with the category values ​​of the key anatomical structure.

[0052] In the process of determining the co-occurrence matrix of the key anatomical structures based on the key anatomical structures located by segmentation, the input ultrasound image is first processed by the semantic segmentation model to accurately segment and locate the key anatomical structures. Then, the segmented key anatomical structures are mapped to a new predicted image. Each pixel value on this predicted image represents a specific anatomical structure category. On the predicted image, any two pixel values ​​are multiplied. This multiplication operation can be understood as judging whether two pixels (i.e., two anatomical structures) appear simultaneously in the same area. If the product of the category values ​​of the two pixels is not zero (i.e., they both belong to a certain anatomical structure category), it is considered that the two anatomical structures have a co-occurrence relationship at this position. Then, the results of the multiplication operation are added to count the total number of co-occurrence relationships at each position. In this way, a category interaction map is obtained, which reflects the co-occurrence of different anatomical structures at various positions in the image.

[0053] The dictionary matrix is ​​a predefined matrix whose values ​​are obtained by interacting the category values ​​of key anatomical structures. Specifically, each anatomical structure can be regarded as a node, and then a connection weight between nodes is defined, which can be set according to the actual symbiotic relationship between the anatomical structures. Each element in the dictionary matrix corresponds to the connection weight between a pair of anatomical structure nodes.

[0054] Finally, the co-occurrence matrix of key anatomical structures is determined based on the category interaction graph and the dictionary matrix. The co-occurrence matrix is ​​a matrix used to represent the symbiotic relationship between different anatomical structures. Its construction method is to multiply the number of symbiosis at each position in the category interaction graph by the corresponding connection weight in the dictionary matrix, and then add the results at all positions to obtain the final co-occurrence matrix. Each element in the co-occurrence matrix represents the degree of symbiosis of a pair of anatomical structures in the entire image. By analyzing the co-occurrence matrix, we can understand the symbiotic relationship between different anatomical structures, and then provide valuable reference information for the diagnosis of fetal congenital heart disease.

[0055] In one implementation of the present application, obtaining a second classification result of a standard section of early pregnancy fetal congenital heart disease based on the expert scoring result and the first classification result of a standard section of early pregnancy fetal congenital heart disease includes:

[0056] When the expert scoring result and the first classification result of the standard section of early pregnancy fetal congenital heart disease both indicate that the section corresponding to the input image is a standard section, the second classification result is determined to be that the section is a standard section; when the first classification result of the standard section of early pregnancy fetal congenital heart disease indicates that the section corresponding to the input image is a standard section, and the expert scoring result cannot confirm that the section is a standard section, the second classification result is that congenital heart disease is considered to be present.

[0057] On the one hand, the above implementation combines the advantages of experts and models, that is, the professional knowledge and experience of experts can make up for the shortcomings of models in some cases, while the automation and efficiency of models can reduce the workload of experts; on the other hand, it also improves the accuracy and reliability of classification: by comprehensively considering the judgment results of experts and models, the possibility of misjudgment and missed judgment can be reduced, and the accuracy and reliability of classification can be improved. Among them, when the expert scoring result and the automatic classification result (i.e., the first classification result) both indicate that the section corresponding to the input image is a standard section, the system will directly determine the second classification result as the section being a standard section. This means that the expert and the model have reached a consensus on this point and believe that the section meets the requirements of the standard section. When the automatic classification result (first classification result) indicates that the section corresponding to the input image is a standard section, but the expert scoring result cannot confirm that the section is a standard section, the system sets the second classification result to "consider the presence of congenital heart disease". This processing method takes into account the uncertainty of experts and assumes that when the expert cannot clearly determine that it is a standard section, there may be other abnormalities related to congenital heart disease.

[0058] In one implementation of the present application, the system uses a focal loss loss function when calculating the loss to balance the data between the standard section and the non-standard section. In the process of detecting standard sections of fetal congenital heart disease in early pregnancy, positive samples (such as standard sections) are often far less than negative samples (such as non-standard sections), which may cause the model to pay too much attention to negative samples during training, thereby affecting the ability to recognize positive samples. By introducing the focal loss loss function and adding a modulation factor to the standard cross entropy loss function, the model can pay more attention to difficult-to-classify samples during training, while reducing attention to easy-to-classify samples. In this way, even if the number of negative samples is far greater than the positive samples, the model can effectively learn the characteristics of the positive samples and improve the accuracy of classification.

[0059] In order to enable those skilled in the art to understand the present application more intuitively, a specific example will be used here to illustrate.

[0060] The present application first uses a semantic segmentation model to segment and locate the key anatomical structures required for standard section detection of early pregnancy fetal congenital heart disease from the input image (grayscale image and Doppler image of the same section), including the left atrium, right atrium, left ventricle, right ventricle, mitral valve blood flow, tricuspid valve blood flow, aortic arch and ductus arteriosus of the four-chamber heart section and the three-vessel tracheal section. Exemplarily, the data can be preprocessed, and the grayscale image of the four-chamber heart section or the three-vessel tracheal section and the Doppler image are fused as the input of the network. Figure 2As shown, the two-dimensional grayscale image and color Doppler image of the same section can be spliced ​​left to right, then right to left, and finally the two spliced ​​images are stacked as the input of the neural network, allowing the convolutional neural network to automatically learn the anatomical organ information of the two-dimensional grayscale image and the functional blood flow information of the color Doppler image. Then, Deeplab v3 is used as the segmentation network to realize the segmentation of the key anatomical structures of the four-chamber heart section and the three-vessel tracheal section. Among them, in the encoder network part, ResNet50 is used as the backbone network of the encoder network part. The output of this part is the encoder features, decoder features and segmentation prediction layer features of the key anatomical structures. The segmentation prediction layer features can output segmentation results of key anatomical structures.

[0061] Considering that doctors will scan other sections besides the four-chamber heart section and the three-vessel tracheal section when the model is used clinically, the ultrasound plane classification module can be set to output 3D features to determine whether it belongs to the four-chamber heart section, the three-vessel tracheal section, or other sections. Category 0 represents other sections, category 1 represents the four-chamber heart section, and category 2 represents the three-vessel tracheal section.

[0062] In addition to using the high-level semantic features of the encoder network, the prediction layer features of the CHD key anatomical structure segmentation network can also be used. As input to optimize the classification of standard sections of congenital heart disease. That is, first, a 2048-dimensional fully connected layer is connected to the feature layer of the encoder network and the predicted features of the key anatomical structure segmentation network respectively. These two fully connected layers are used to reduce the feature dimension and enhance the semantic information of the model. The features of these two fully connected layers are then cascaded to output a 4096-dimensional feature, and then the nonlinear expression ability of the network is increased through the ReLU activation function. Next, a fully connected layer with an output channel of 512 dimensions is used after the 4096-dimensional feature to further reduce the dimension and optimize the feature. Similarly, the ReLU activation function is also used after this fully connected layer to enhance the nonlinear information of the model. Finally, a fully connected layer with a channel number of 3 is used after the 512-dimensional feature to output a 3-dimensional ultrasound plane classification feature F pc And the feature and the corresponding label information are calculated through the softmax loss function to get L sc .

[0063] In the previous stage, the key anatomical structure segmentation of early pregnancy fetal congenital heart disease was synchronously achieved through convolutional neural network. On this basis, the key anatomical structure segmentation results can also be used as input to first construct the co-occurrence matrix between the background, left atrium, right atrium, left ventricle, right ventricle, mitral valve blood flow, tricuspid valve blood flow, aortic arch and arterial duct.

[0064] In constructing the co-occurrence matrix of key anatomical structures:

[0065] Assume that the key anatomical structure is segmented and predicted image P s The size is m×m, for P s Any two points x on i and x j (0<i≤m×m, 0<j≤m×m), first calculate its category interaction graph M in Let e ij ∈M in , that is, e ij It belongs to the interaction diagram M in A point on, then e ij Can be defined as: ij =10×x i × j +(x i +x j ), that is, by multiplying and adding any two values ​​on the predicted image to obtain the corresponding interaction value, this method can more accurately map the interaction between different category values ​​compared to only multiplication and addition operations. s By prediction layer features Calculated via the argmax operation.

[0066] Then construct an 8×8 dictionary matrix D, such as Figure 3 As shown in the figure, the category values ​​corresponding to the background, left atrium, right atrium, left ventricle, right ventricle, mitral valve blood flow, tricuspid valve blood flow, aortic arch and ductus arteriosus in the key anatomical structure segmentation prediction image are 0, 1, 2, 3, 4, 5, 6, 7, and 8 respectively. Let d kl ∈D(0<k≤8,0<l≤8), then the value d in the dictionary matrix kl , which can be defined as: d kl =10×k×l+k+l, that is, the value of the dictionary matrix D is obtained by interactively obtaining the category values ​​of the key anatomical structures of congenital heart disease, see Figure 3 Next, for different d in the dictionary matrix D kl Value, traversal search category interaction graph M in , we can get each d kl The number of times the value appears c kl , that is, the number of co-occurrences between the target with category value (k-1) and the target with category value (l-1). kl Replace the value with d kl The corresponding position in the anatomical structure can be obtained by the co-occurrence matrix diagram C1, see Figure 4 .

[0067] After obtaining the co-occurrence matrix diagram between different category values, blood flow information is introduced to comprehensively consider the blood flow quantity, blood flow shape and blood flow color. Two fully connected layers are connected in series after the encoder features behind ResNet50, and the number of output channels is 2048 and 512 respectively, which is used to reduce the dimension of the features. At the same time, a ReLU activation function is added after these two fully connected layers to enhance the nonlinear ability of the network. Then, three more fully connected layers are connected in parallel, and the output channels are 3D, 3D and 2D respectively, which are used to predict the blood flow quantity, blood flow shape and blood flow color respectively.

[0068] The number of blood flows includes one blood flow, two blood flows, and other blood flows. The number of other blood flows as the background category corresponds to label 0. One blood flow and two blood flows correspond to label 1 and label 2 respectively. The blood flow shape includes parallel blood flow, V-shaped blood flow, and other blood flow conditions. Other blood flow conditions correspond to label 0, parallel blood flow corresponds to label 1, and V-shaped blood flow corresponds to label 2. The blood flow color includes the same blood flow color and other blood flow color conditions. Other blood flow color conditions correspond to label 0, and the same blood flow color corresponds to label 1.

[0069] During training, the three fully connected layers used to predict blood flow quantity, blood flow shape, and blood flow color are calculated through the Softmax cross entropy loss function and the corresponding annotation information. The loss functions are denoted as L bn , L bs and L bc The total loss of blood flow classification can be defined as: L b =L bn +L bs +L bc Then, the three fully connected layers are subjected to argmax operation to obtain the three predicted values ​​p of blood flow quantity, blood flow shape and blood flow color. bn 、p bs and p bc Finally, the three predicted values ​​of the blood flow classification network are combined into a 1×3 blood flow prediction matrix M b .

[0070] After obtaining the blood flow prediction matrix M b Then, the key anatomical structure co-occurrence matrix C1 is processed. C1 is a symmetric matrix. In order to extract the co-occurrence information of the anatomical structure more efficiently, we take the 36 elements in the upper right corner of C1 to form a 36×1 matrix C2, that is, the first row of the matrix takes 9 numbers starting from the first value, the second row of the matrix takes 8 numbers starting from the second value, and so on. The ninth row of the matrix takes 1 number starting from the ninth number, a total of 36 numbers, see Figure 5Then, let the values ​​greater than 0 in the C2 matrix be equal to 1, and we can get a 36×1 0,1 matrix C3. 1 means that the two categories of organs co-occur, and 0 means that the two categories of organs do not co-occur. Finally, the 36×1 matrix C3 is combined with the 1×3 matrix M b By performing matrix multiplication operations, a blood flow-key anatomical structure co-occurrence matrix C4 with a dimension of 36×1 can be obtained.

[0071] When obstetricians and gynecologists mark the standard sections of early pregnancy fetal congenital heart disease, they use multiple indicators of expert consensus to determine whether the current section belongs to the standard section of the four-chamber heart section or the three-vessel tracheal section. It depends on the professional knowledge of obstetricians and gynecologists. That is, when different doctors mark the standard sections of the four-chamber heart section or the three-vessel tracheal section of the same person, there is a great possibility of differences. Even when the same standard section is marked multiple times for the same person, there may be differences. Therefore, the judgment of the standard section of early pregnancy fetal congenital heart disease is full of subjectivity and the label information is weak. That is, many standard sections may be detected in the video sequence of the same patient.

[0072] In order to address the above problems and achieve the need for high-precision interpretable standard sections of early pregnancy fetal congenital heart disease, this application realizes the detection of standard sections of early pregnancy fetal congenital heart disease from two dimensions. The first dimension is to integrate the anatomical structure segmentation results, section classification results and blood flow classification results according to the rules of expert consensus, and automatically generate scores for multiple indicators. Then set a threshold and evaluate the total score. The second dimension is to integrate the results of anatomical organ segmentation, section classification results, key anatomical structure co-occurrence matrix diagram and blood flow-key anatomical structure co-occurrence matrix diagram, and use a convolutional neural network to learn the classification results of standard sections of early pregnancy fetuses.

[0073] When using expert knowledge for scoring, the learned results are used, and the integrated early pregnancy fetal scoring indicators can be as follows: For the four-chamber heart section: 1) detect the left atrium and right atrium; 2) detect the left ventricle and right ventricle; 3) detect two blood flows; 4) the two blood flows flow into the left and right ventricles respectively; 5) the left and right ventricular blood flows are the same color; 6) the left and right blood flows are parallel. For the three-vessel tracheal section: 1) detect the aorta and pulmonary artery; 2) detect two blood flows; 3) the two blood flows merge at the aortic arch arterial ductus, presenting a V-shaped structure.

[0074] For the four-chamber heart view, each index is marked with 1 point, for a total of 6 points. When the total score is not less than 4 points, it is considered that the model has a high probability of learning the standard section of the four-chamber heart view. For the three-vessel tracheal view, the first and second indexes are marked with 1 point, and the third index is marked with 2 points, for a total of 4 points. When the total score is not less than 3 points, it is considered that the model has a high probability of learning the standard section of the four-chamber heart view. The specific operation process can be as follows:

[0075] 1) First, the ultrasound section classification task determines whether the current section belongs to the four-chamber heart section or the three-vessel tracheal section.

[0076] 2) If it is a four-chamber heart section, the detection results of the left atrium, right atrium, left ventricle and right ventricle can be given by the key anatomical structure segmentation results, and the blood flow quantity, color and shape can be given by the blood flow classification results. To determine whether the blood flow flows into the left and right ventricles, the Doppler color ultrasound image and the key anatomical structure segmentation results can be interactively obtained, that is, by setting a threshold to extract the blood flow area, and then finding the intersection of the blood flow area with the left and right ventricles, it can be determined whether the blood flow flows into the left and right ventricles.

[0077] 3) If it is a three-vessel tracheal section, the detection of the aorta and pulmonary artery can be given by the key anatomical structure segmentation results, and the blood flow quantity and blood flow shape can be given by the blood flow classification results.

[0078] When considering from the second dimension, in order to better learn the interpretable standard sections required for early pregnancy fetal congenital heart disease detection, this application pre-implements the key anatomical structure segmentation, blood flow segmentation and section plane segmentation required for early pregnancy congenital heart disease detection, and on this basis constructs the key anatomical structure co-occurrence matrix diagram and the blood flow-anatomical organ co-occurrence matrix diagram. The specific implementation process can be as follows:

[0079] 1) Prediction layer features at the output of the key anatomical structure network Followed by a fully connected layer with a channel number of 512 dimensions, a 512-dimensional feature vector F is obtained sc1 . In the key anatomical structure prediction image P s To enhance the interpretability of the model, the fully connected layer aims to reduce the dimension of features.

[0080] 2) The 8×8 anatomical structure co-occurrence matrix C1 and the 36×3 blood flow-anatomical structure co-occurrence matrix C4 are converted into 64- and 108-dimensional feature vectors respectively and concatenated to obtain a 172-dimensional feature vector F sc2 .

[0081] 3) The 512-dimensional feature vector F sc1 , 172-dimensional feature vector F sc2 and the 3D plane classification feature vector F pc Cascaded and followed by a ReLu activation function, outputting a 687-dimensional feature vector F sc3 .

[0082] 4) In the 687-dimensional feature vector F sc3A fully connected layer with an output channel of 1 is then connected to obtain a 1D feature. Then, a softmax operation is performed on this 1D feature to normalize its value to the interval [0,1] to obtain the standard section prediction value p t , used to determine whether the current section is a standard section.

[0083] 5) Next, the standard section prediction feature p t The loss is calculated with the corresponding annotation information. Considering the large difference in the number of standard and non-standard sections of fetal congenital heart disease in early pregnancy, the use of L1 and L2 losses will lead to overfitting of the model. This application uses Focal loss to solve the data imbalance problem between standard sections and non-standard sections. Focal loss is defined as:

[0084] focalloss(p t )=-α t (1-p t ) γ log(p t ), where α t It can suppress the imbalance of the number of positive and negative samples, and γ can control the imbalance of the number of simple / difficult to distinguish samples. In this application, α is set t =1,γ=2.

[0085] 6) The standard section detection tasks include key anatomical structure segmentation, ultrasound plane classification, blood flow classification and congenital heart disease classification. The total loss is defined as: L = focalloss(p t )+L S +L sc +L b .

[0086] Then, in the test phase, two maximum and minimum thresholds are set to predict whether the current section belongs to the standard section. The maximum threshold is set to 0.7 and the minimum threshold is set to 0.3. When the model's prediction value of whether the current section is a standard section is greater than 0.7, it is considered that the current section has a high probability of belonging to the standard section. If the prediction value is less than 0.3, it is considered that the current section has a high probability of not belonging to the standard section. If the threshold is between [0.3, 0.7], the model cannot determine whether the section belongs to the standard section.

[0087] Finally, in clinical applications, it is considered that extremely high accuracy is required for diagnoses involving people. Therefore, under normal circumstances, the section is considered to be a standard section only when and only when both the expert knowledge score and the standard section classification model detect that the tested section is a standard section. If the standard section classification model can determine that the tested section is a standard section, but the expert knowledge score is still unable to assess that the tested section is a standard section, it is considered that the patient may have other symptoms such as congenital heart disease. If the system screens out multiple standard sections of four-chamber heart sections and three-vessel tracheal sections, the section with the highest score of the standard section classification model is selected as the standard section.

[0088] Example 2

[0089] See also Figure 6 The present application embodiment provides a method for detecting early pregnancy fetal congenital heart disease in a standard section, which comprises the following steps:

[0090] The key anatomical structures required for the detection of standard sections of early pregnancy fetal congenital heart disease are segmented and located from the input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section. Section classification processing is performed according to the segmented and located key anatomical structures to obtain section classification results. Blood flow classification processing is performed according to the segmented and located key anatomical structures to obtain blood flow classification results. According to the segmented and located key anatomical structures, a co-occurrence matrix diagram of the key anatomical structures is determined; and according to the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results, a blood flow-key anatomical structure co-occurrence matrix is ​​determined. According to the preset expert knowledge rules, the key anatomical structures, the section classification results and the blood flow classification results are integrated to output the corresponding expert scoring results. According to the segmented key anatomical structures, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, a first classification result of the standard section of early pregnancy fetal congenital heart disease is obtained, and according to the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease, a second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained.

[0091] For the specific implementation process of the above method, please refer to the standard section detection system for early pregnancy fetal congenital heart disease provided in Example 1, which will not be described in detail here.

[0092] Example 3

[0093] See also Figure 7, the embodiment of the present application provides an electronic device, which includes at least one processor 201 and at least one memory 202; wherein the processor 201 and the memory 202 are directly connected to each other, or communicate with each other through a communication interface 203, or are electrically connected through one or more communication buses or signal lines to achieve data transmission or interaction; the memory 202 stores program instructions that can be executed by the processor 201, and the processor 201 calls the program instructions to execute a standard section detection method for early pregnancy fetal congenital heart disease. For example, the following is realized:

[0094] The key anatomical structures required for the detection of standard sections of early pregnancy fetal congenital heart disease are segmented and located from the input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section. Section classification processing is performed according to the segmented and located key anatomical structures to obtain section classification results. Blood flow classification processing is performed according to the segmented and located key anatomical structures to obtain blood flow classification results. According to the segmented and located key anatomical structures, a co-occurrence matrix diagram of the key anatomical structures is determined; and according to the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results, a blood flow-key anatomical structure co-occurrence matrix is ​​determined. According to the preset expert knowledge rules, the key anatomical structures, the section classification results and the blood flow classification results are integrated to output the corresponding expert scoring results. According to the segmented key anatomical structures, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, a first classification result of the standard section of early pregnancy fetal congenital heart disease is obtained, and according to the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease, a second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained.

[0095] Among them, the memory 202 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0096] The processor 201 may be an integrated circuit chip with signal processing capability. The processor 201 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0097] Understandably, Figure 7 The structure shown is for illustration only. The electronic device may also include Figure 7 More or fewer components as shown, or with Figure 7 Different configurations shown. Figure 7 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0098] Example 4

[0099] The present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by the processor 201, implements a method for detecting congenital heart disease in a fetus in early pregnancy with a standard section. For example, the following is implemented:

[0100] The key anatomical structures required for the detection of standard sections of early pregnancy fetal congenital heart disease are segmented and located from the input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section. Section classification processing is performed according to the segmented and located key anatomical structures to obtain section classification results. Blood flow classification processing is performed according to the segmented and located key anatomical structures to obtain blood flow classification results. According to the segmented and located key anatomical structures, a co-occurrence matrix diagram of the key anatomical structures is determined; and according to the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results, a blood flow-key anatomical structure co-occurrence matrix is ​​determined. According to the preset expert knowledge rules, the key anatomical structures, the section classification results and the blood flow classification results are integrated to output the corresponding expert scoring results. According to the segmented key anatomical structures, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, a first classification result of the standard section of early pregnancy fetal congenital heart disease is obtained, and according to the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease, a second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained.

[0101] If the above functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0102] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A standard section detection system for early pregnancy fetal congenital heart disease, characterized in that: include: A semantic segmentation model for segmenting and locating key anatomical structures required for standard section detection of early pregnancy fetal congenital heart disease from an input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section; An ultrasound plane classification module is used to perform section classification processing according to the key anatomical structures located by segmentation to obtain section classification results; A blood flow classification module is used to perform blood flow classification processing according to the key anatomical structures located by segmentation to obtain a blood flow classification result; A co-occurrence knowledge graph module is used to determine a co-occurrence matrix diagram of key anatomical structures according to the key anatomical structures located by segmentation, specifically including: performing multiplication and addition operations on any two values ​​on the predicted image carrying the key anatomical structures located by segmentation, so as to determine a category interaction diagram according to the calculated interaction value; determining a co-occurrence matrix of key anatomical structures according to the category interaction diagram and a dictionary matrix; the values ​​of the dictionary matrix are obtained by the interaction of the category values ​​of the key anatomical structures; and determining a blood flow-key anatomical structure co-occurrence matrix according to the co-occurrence matrix diagram of the key anatomical structures and the blood flow classification results, specifically including: performing a matrix multiplication operation on the co-occurrence matrix diagram of the key anatomical structures and the blood flow prediction matrix corresponding to the blood flow classification results, so as to obtain a blood flow-key anatomical structure co-occurrence matrix; An expert knowledge scoring module is used to integrate key anatomical structures, section classification results and blood flow classification results according to preset expert knowledge rules to output corresponding expert scoring results; The standard section classification module is used to obtain the first classification result of the standard section of early pregnancy fetal congenital heart disease based on the key anatomical structures determined by segmentation, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, and to obtain the second classification result of the standard section of early pregnancy fetal congenital heart disease based on the expert scoring results and the first classification result of the standard section of early pregnancy fetal congenital heart disease.

2. The standard section detection system for early pregnancy fetal congenital heart disease as claimed in claim 1, characterized in that: The semantic segmentation model is further used to fuse the input two-dimensional grayscale image and color Doppler image of the same section, and segment the key anatomical structures required for standard section detection of fetal congenital heart disease in early pregnancy based on the image obtained by the fusion process.

3. The standard section detection system for early pregnancy fetal congenital heart disease as claimed in claim 1, characterized in that: The semantic segmentation model includes an encoder network unit and a key anatomical structure segmentation network unit; the ultrasound plane classification module includes a first fully connected layer, a second fully connected layer, a third fully connected layer and a fourth fully connected layer; the number of channels of the fourth fully connected layer is consistent with the number of types of section classification; The first fully connected layer is calculated by the last encoder module of the encoder network unit, and the second fully connected layer is calculated by the last decoder module of the key anatomical structure segmentation network; The third fully connected layer is calculated by cascading the first fully connected layer and the second fully connected layer, and the fourth fully connected layer is calculated by the third fully connected layer.

4. A standard section detection system for early pregnancy fetal congenital heart disease as claimed in claim 3, characterized in that: The blood flow classification module includes a fifth fully connected layer, a sixth fully connected layer and a seventh fully connected layer; The fifth fully connected layer is calculated by the last encoder module of the encoder network unit, the sixth fully connected layer is calculated by the fifth fully connected layer, the sixth fully connected layer is respectively calculated to obtain three fully connected layers, and the three fully connected layers are cascaded to obtain the seventh fully connected layer.

5. The standard section detection system for early pregnancy fetal congenital heart disease as claimed in claim 1, characterized in that: The second classification result of the standard section of early pregnancy fetal congenital heart disease is obtained according to the expert scoring result and the first classification result of the standard section of early pregnancy fetal congenital heart disease, including: When the expert scoring result and the first classification result of the standard section of early pregnancy fetal congenital heart disease both indicate that the section corresponding to the input image is a standard section, determining the second classification result that the section is a standard section; When the first classification result of the standard section of early pregnancy fetal congenital heart disease indicates that the section corresponding to the input image is a standard section, and the expert scoring result cannot confirm that the section is a standard section, the second classification result is that congenital heart disease is considered to exist.

6. The standard section detection system for early pregnancy fetal congenital heart disease according to claim 1, characterized in that: The system uses a focal loss function when calculating the loss to balance the data between the standard slices and the non-standard slices.

7. A method for detecting congenital heart disease in early pregnancy fetuses with standard sections, characterized in that: The following steps are involved: Segmenting and locating key anatomical structures required for standard section detection of early pregnancy fetal congenital heart disease from an input image; the key anatomical structures include a four-chamber heart section and a three-vessel tracheal section; Performing section classification processing according to the key anatomical structures located by segmentation to obtain section classification results; Perform blood flow classification processing according to the key anatomical structures located by segmentation to obtain blood flow classification results; Determine a co-occurrence matrix diagram of the key anatomical structure according to the key anatomical structure located by segmentation, specifically including: multiply and add any two values ​​on the predicted image carrying the key anatomical structure located by segmentation to determine a category interaction diagram according to the calculated interaction value; determine a co-occurrence matrix of the key anatomical structure according to the category interaction diagram and a dictionary matrix; the value of the dictionary matrix is ​​obtained by interacting with the category values ​​of the key anatomical structure; and determine a blood flow-key anatomical structure co-occurrence matrix according to the co-occurrence matrix diagram of the key anatomical structure and a blood flow classification result, specifically including: perform a matrix multiplication operation on the co-occurrence matrix diagram of the key anatomical structure and a blood flow prediction matrix corresponding to the blood flow classification result to obtain a blood flow-key anatomical structure co-occurrence matrix; According to the preset expert knowledge rules, key anatomical structures, section classification results and blood flow classification results are integrated to output the corresponding expert scoring results; According to the key anatomical structures determined by segmentation, the section classification results, the co-occurrence matrix diagram of the key anatomical structures and the blood flow-key anatomical structure co-occurrence matrix, the first classification results of the standard sections for early pregnancy fetal congenital heart disease are obtained. And according to the expert scoring results and the first classification results of the standard sections for early pregnancy fetal congenital heart disease, the second classification results of the standard sections for early pregnancy fetal congenital heart disease are obtained.

8. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to claim 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claim 7 is implemented.

Images