Second pregnancy prenatal II-level ultrasonic standard fetal plane recognition method

Abstract

The invention relates to a second pregnancy prenatal II-level ultrasonic standard fetal plane recognition method, and belongs to the technical field of fetal ultrasonic image processing. The method comprises the following steps: pre-training a residual neural network model based on a second pregnancy prenatal II-level ultrasound standard fetal plane image set, reserving a convolutional layer of the pre-trained residual neural network model, and generating a feature map in the shape of [n, FZ, FZ] based on the convolutional layer of the residual neural network model; and combining the singular vector and the singular value after the feature map decomposition to construct a matrix gene vector MGV, decoding the matrix gene vector MGV through large-step cross-kernel convolution, or decoding the matrix gene vector MGV through a feature fusion method, and outputting a classification result of the prenatal II-level ultrasound standard fetus plane image in the second pregnancy period. According to the method, classification of the fetal II-level ultrasonic images can be realized, and distribution information and significant information on the feature images can be better reserved.

Description

Technical Field

[0001] The invention belongs to the technical field of fetal ultrasound image processing and relates to a method for identifying a standard fetal plane for prenatal grade II ultrasound in the second trimester. Background Art

[0002] Ultrasound technology is low-cost, radiation-free, and widely used for prenatal observation, measurement, and diagnosis of fetal diseases, playing a key role in fetal health care. Obtaining standardized ultrasound views of the fetus is the basic prerequisite for various examinations in the field of fetal ultrasound diagnosis. In general, obtaining standardized ultrasound views of the fetus usually requires skilled medical professionals and is affected by factors such as equipment quality. Given the scarcity of ultrasound physicians, it becomes necessary to integrate standard plane recognition technology to reduce the risk of misdiagnosis and improve efficiency. However, current work is mainly limited to the recognition of a few areas or slices, and most studies focus on the standard ultrasound (level I ultrasound) classification of low-risk early pregnancy. In fact, mid-pregnancy is the key stage for screening abnormalities, in which accurate identification of standard fetal views plays a key role in detecting fetal malformations and congenital diseases. However, in clinical practice, mid-pregnancy prenatal examinations require secondary ultrasound evaluation by medical professionals, and about 10 fetal ultrasound standard slices need to be considered, rather than a few standard slices. Therefore, there is an urgent need to explore classification algorithms that can effectively deal with 10 or more slices.

[0003] Some fetal ultrasound images are usually located in the same area (such as Figure 1 Middle (1) Cross section of lateral ventricle, Figure 1 (2) Transverse section of thalamus and Figure 1 (3) The cross-section of the cerebellum is located in the same area and has great similarity. Therefore, in order to effectively distinguish these areas, the model is required to have a higher detail extraction capability.

[0004] Residual neural networks have been widely used in the field of medical imaging by combining residual structures. However, the inventors of the present invention have noticed that the convolution part of the residual neural network uses batch normalization and rectified linear units to effectively extract important information and suppress unimportant information, resulting in the output feature map being often a sparse matrix, such as Figure 2As shown in (1). Its characteristic is that the values ​​in the matrix are either 0 or very large. The larger values ​​in the matrix are called significant information, and the distribution positions of those larger values ​​are called distribution information. It is easy to know that using the global average pooling layer to calculate the average value of the feature map will lose distribution information and significance information. And because there are many 0 values ​​in the matrix, taking the average will weaken the significant information. In order to improve the model's ability to extract image details, the model is required to retain this information as much as possible. Because different sections of the same area are similar, the average values ​​of their feature maps are also close. At this time, it is necessary to further use distribution information and significant information to distinguish them. The inventors of the present invention have found that spectral graph techniques such as truncated singular value decomposition, which are often used in sparse matrix processing, can be used as an important means to extract sparse matrix information. From Figure 2 It can be clearly seen from (2) that the sparse matrix reconstructed by three sets of singular values ​​and singular vectors shows the ability to capture distribution information, that is, it can capture the location information of larger values. However, its ability to capture significant information is still limited because it is different from Figure 2 Some of the reconstructed values ​​are significantly smaller than some of the larger values ​​of the matrix in (1). Figure 2 The matrix reconstructed using five sets of singular vectors and singular values ​​in (3) shows excellent performance in capturing distribution information and salient information, which is very similar to the original sparse matrix graph. It is worth noting that Figure 2 There is no significant difference between the image reconstructed using 7 sets and the image reconstructed using 5 sets in (4). This shows that for each sparse feature map extracted by the residual neural network, only a small number of singular values ​​and singular vectors are sufficient to reflect most of the distribution information and important information of the sparse matrix.

[0005] There are a few studies that involve designing pooling layers in combination with singular value decomposition. However, they only combine the largest singular value of the singular value decomposition, and the inventors of the present invention believe that this is not enough to accurately extract detailed information. This is because the singular vectors and singular values ​​of the singular value decomposition are both necessary parts for reconstructing the original matrix. Summary of the invention

[0006] The purpose of the present invention is to solve the problems existing in the background technology, provide a method for identifying the standard fetal plane for second-trimester prenatal grade II ultrasound, and propose a singular pooling layer based on singular value decomposition to replace the global average pooling layer and the fully connected layer at the back end of the traditional residual neural network model to improve the model's ability to capture image detail information.

[0007] To achieve the above object, the technical solution of the present invention is: a method for identifying the standard fetal plane of level II ultrasound for the second trimester of pregnancy, comprising: Pre-training a residual neural network model based on a second trimester prenatal grade II ultrasound standard fetal planar image set, retaining the convolutional layer of the pre-trained residual neural network model (i.e., freezing the parameters of the convolutional layer in the pre-trained residual neural network model, deleting the global average pooling layer and the fully connected layer, and retaining only the convolutional layer), and generating a feature map of shape [n, FZ, FZ] based on the convolutional layer of the residual neural network model, where n represents the number of feature maps generated, and the two FZs represent the width and height of the feature map, respectively; The singular vectors and singular values ​​after feature map decomposition are combined to construct a matrix gene vector MGV, and the matrix gene vector MGV is decoded by a two-layer one-dimensional convolutional neural network with different strided kernel convolution parameters, or by a feature fusion method, i.e., fusing the convolution kernel length and step length of the second one-dimensional convolutional neural network with two parameter settings to decode the matrix gene vector MGV, and the classification results of the second trimester prenatal grade II ultrasound standard fetal plane images are output.

[0008] In one embodiment of the present invention, the residual neural network model can adopt an open source pre-training model, that is, the pre-training process is cancelled.

[0009] In one embodiment of the present invention, before pre-training the residual neural network model, the second-trimester prenatal level II ultrasound standard fetal plane image set needs to be adjusted to a uniform size using bilinear interpolation.

[0010] In one embodiment of the present invention, the singular vectors and singular values ​​after decomposing the feature graph are combined to construct a matrix gene vector MGV, which is specifically implemented as follows: Singular value decomposition is used to decompose the feature map into a left singular vector matrix, a right singular vector matrix and a singular value matrix, that is, A=USV T ; where A represents the feature map; U represents the left singular vector matrix, and ,u i is a left singular vector, 1≤i≤FZ; V represents a right singular vector matrix, and , v i are right singular vectors; S represents the singular value matrix, and , diag represents a diagonal matrix, is a singular value, and the singular values ​​are sorted from large to small; Select the top k largest singular values ​​of each feature map The corresponding left singular vector Multiply the corresponding After completing the product of the first k singular values ​​of all feature maps and the corresponding left singular vectors, they are concatenated into a large vector, namely the matrix gene vector (abbreviated as MGV), as follows Figure 3 shown.

[0011] Since the larger the singular value, the more information the corresponding singular vector contains, the more important it is. Therefore, the first k left singular vectors can reflect the information of the matrix to a large extent, while reducing the amount of calculation. In addition, the singular value is the embodiment of the importance of the singular vector, and the operation of "singular value multiplied by the corresponding left singular vector" is equivalent to the process of assigning the importance weight to the corresponding left singular vector, that is, weighted combination.

[0012] Here, the right singular vector can also be processed in a similar way to the left singular vector to construct the matrix gene vector MGV.

[0013] In one embodiment of the present invention, a large kernel strided convolution is proposed to decode the matrix gene vector MGV, and the large kernel strided convolution is composed of two layers of one-dimensional convolutional neural networks with different parameter settings.

[0014] For two pictures of the same category, if the categories of the two are more similar, the feature maps obtained by the residual neural network model will be more similar, and it is reasonable to infer that the singular vectors and singular values ​​of the feature map matrices of the two are more similar. Therefore, an effective algorithm that can reflect the similarity is needed, that is, a one-dimensional convolutional neural network. Its essence is to do a dot product between the convolution kernel and a small sequence of the matrix gene vector MGV. The larger the dot product value, the higher the similarity.

[0015] The one-dimensional convolutional neural network with a kernel length of 1 and a stride of 1 as the first layer is equivalent to performing a dimensionality increase operation on the matrix gene vector MGV, which will not destroy the data structure of the matrix gene vector MGV. Therefore, the kernel length and stride of the first layer of the one-dimensional convolutional neural network are both set to 1.

[0016] The traditional one-dimensional convolutional neural network with a kernel length of 3 and a stride of 1 is mainly designed for pixel and text data. Although the traditional setting has a certain effect on local information extraction, it has significant deficiencies in the matrix gene vector MGV, resulting in excessive computation and excessive output data dimensions. Figure 4As shown in (1), when the convolution kernel spans two left singular vectors, the convolution operation is meaningless. Since the convolution kernel has the characteristic of parameter sharing, this operation may affect the parameter update of the convolution kernel, and thus require a longer training time to effectively filter out these invalid information. Therefore, the present invention proposes two methods for setting the parameters of the second-layer one-dimensional convolutional neural network for the decoding matrix gene vector MGV: the first method is to set the kernel length and step size of the second-layer one-dimensional convolutional neural network to FZ. The core concept of this method is to sequentially calculate the dot product similarity between the convolution kernel and each left singular vector and singular value combination. This hyperparameter setting method not only reduces the computational complexity to a certain extent, but also effectively avoids Figure 4 The situation shown in (1) occurs. Its visualization is as follows Figure 4 As shown in (2). This hyperparameter setting method is based on the length of a single singular vector, which makes it difficult to distinguish the information between feature maps, but the extraction of local information is relatively complete; the second method is to set the kernel length and step size of the second-layer one-dimensional convolutional neural network to k×FZ. This method performs convolution operations on the basis of the sum of the lengths of all the first k singular vectors of a feature map, that is, it calculates the similarity between the convolution kernel and the concatenation of the combination of the k left singular vectors and singular values ​​of the feature map. Compared with the first hyperparameter setting method, this can more effectively distinguish the information between feature maps, so the information obtained is more global. Its visualization is as follows Figure 4 As shown in (3).

[0017] In one embodiment of the present invention, a feature fusion method is proposed, which combines the advantages of the two parameter settings of the convolution kernel length and the step size of the second layer of the one-dimensional convolutional neural network, and decodes the matrix gene vector MGV, which is specifically implemented as follows: Construct two branches with the following structure: Figure 7 As shown. The advantage of using a one-dimensional convolutional neural network with a kernel length and a step size of FZ is that it can extract information from the perspective of the singular vectors of a feature map, because its span is the length of a singular vector, but it is difficult to distinguish the information between different feature maps, that is, the extracted information is local information. The advantage of using a one-dimensional convolutional neural network with a kernel length and a step size of K×FZ is that its span is the sum of the lengths of k singular vectors of a feature map. Therefore, the information between different feature maps can be distinguished, but the extraction of detailed information is also sacrificed to a certain extent, that is, the extracted information is global information. The feature fusion method proposed in the present invention retains the advantages of both methods.

[0018] For branch 1, it mainly extracts local information. The matrix gene vector enters the one-dimensional convolutional neural network with a kernel length and step size of 1, the Mish activation function, the one-dimensional convolutional neural network with a kernel length and step size of FZ, and the Mish activation function in turn, and then its output enters the one-dimensional maximum pooling layer with a window size and step size of k, and the one-dimensional average pooling layer with a window size and step size of k, and then the outputs of the one-dimensional maximum pooling layer and the one-dimensional average pooling layer are added to obtain the output of branch 1; There are two reasons why the parameters of the one-dimensional average pooling layer and the one-dimensional maximum pooling layer are set to have a window size and a step size of k. First, the one-dimensional maximum pooling layer can extract the most significant one of all left singular vectors and singular value combinations of each feature map. The one-dimensional average pooling layer extracts the average features of all left singular vectors and singular value combinations of each feature map. By fusing the two, more representative local features can be obtained. Second, this can make the data size of branch 1 match the data size of branch 2.

[0019] For branch 2, the matrix gene vector sequentially enters the attention mechanism, the one-dimensional convolutional neural network with a kernel length and a step size of 1, the Mish activation function, the one-dimensional convolutional neural network with a kernel length and a step size of k×FZ, and the Mish activation function to obtain the output of branch 2; For the attention mechanism of branch 2, its implementation is as follows Figure 8 As shown in Figure 1. The sum of the first k singular values ​​of each feature map is used as its attention score. Then the attention scores of all feature maps are input into the SoftMax layer to obtain the corresponding attention weights. Finally, these attention weights are multiplied by the concatenation of the first k left singular vectors and singular values ​​of the corresponding feature map (matrix gene vector MGV), that is, the weights are assigned.

[0020] The inventor has multiple motivations for proposing this attention. First, the traditional idea of ​​using average values ​​to determine attention weights (such as spatial attention) is not applicable to the matrix gene vector MGV data structure. Because for the concatenation of the first k left singular vectors and singular values ​​of a feature graph, its average value is meaningless. The left singular vector essentially represents a direction, and its modulus (length) is indefinite and can be arbitrarily long or short. Therefore, the result obtained by averaging it is uncertain.

[0021] The present invention innovatively proposes that the reasons for using the first k singular values ​​of the feature map as attention scores include: 1. The singular value of each feature map is the energy score of the matrix in the corresponding direction, which is a direct reflection of the importance. 2. The largest singular value accounts for the largest proportion in the sum of the first k singular values. The largest singular value is also the spectral norm of the corresponding feature map. In matrix theory, the norm is a measure of the size and influence of the matrix space, which reflects the importance of the feature map to a certain extent. 3. The set of singular values ​​of the matrix is ​​fixed and does not change with the modulus of the singular vector, but is determined by the structure and properties of the matrix itself. 4. The singular values ​​in the structure of the present invention have already been obtained, so the introduction of this attention will not add too much extra computational burden.

[0022] Finally, the outputs of branch 1 and branch 2 are added to obtain the result of matrix gene vector MGV decoding.

[0023] In one embodiment of the present invention, the result after decoding the matrix gene vector is sequentially input into the flattening layer and the fully connected layer, and then outputs the 10 classification results of the second trimester prenatal grade II ultrasound standard fetal plane image.

[0024] In one embodiment of the present invention, the matrix gene vector part, the large kernel strided convolution, or the new structure that integrates the advantages of the two large strided kernel convolution parameter settings, and the output classification result part are collectively referred to as the singular pooling layer, and the back propagation algorithm is used to update the parameters of the singular pooling layer. Since only the parameters of the singular pooling layer are updated, all pictures can be converted into matrix gene vector MGV data and stored in advance before training the singular pooling layer. In this way, the matrix gene vector MGV data can be directly input into the model, and the conversion operation of the picture to the matrix gene vector MGV data structure is avoided in each round of training.

[0025] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention mainly focuses on the in-depth research and optimization of a high-precision 10-classification algorithm for the second trimester prenatal grade II ultrasound standard fetal plane, which can realize the classification of fetal grade II ultrasound images.

[0026] (2) The present invention proposes a matrix gene vector (MGV) data structure, which converts two-dimensional data into one-dimensional form without parameters, while retaining a large amount of important information of the original matrix. It also proposes to regard singular values ​​as a measure of the importance of left singular vectors, and to achieve the fusion of the left singular vectors and the corresponding singular values ​​by multiplying them by a weighted operation.

[0027] (3) Considering the characteristics of MGV, the present invention proposes a large-core strided convolution module to decode MGV, which reduces the amount of calculation and significantly improves the performance. Specifically, it is proposed to first use a one-dimensional convolutional neural network with a kernel length and stride of 1. This convolution is used to increase the dimension of MGV without destroying the data structure of MGV. Then, a one-dimensional convolutional neural network with a kernel length and stride of FZ or k×FZ is entered. The first type focuses on detail information, and the second type focuses on global information.

[0028] (4) The present invention also proposes a feature fusion method for decoding the MGV data structure. By integrating the advantages of two large-stride convolution parameter settings, it is not only possible to distinguish the global information between different feature maps, but also to extract the local detail information within the feature map. The present invention also introduces a special attention mechanism for the MGV data structure. This attention mechanism uses the sum of the first k singular values ​​of the feature map as the attention score. The singular values ​​are already obtained, so no additional trainable parameters are required, and the amount of calculation is small. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 These are some fetal ultrasound images; (1) is the cross section of the lateral ventricle, (2) is the cross section of the thalamus, and (3) is the cross section of the cerebellum.

[0030] Figure 2 Feature maps extracted from the convolution part of the residual neural network and matrices reconstructed using different singular values ​​and singular vectors; among them, (1) is a visualization of a sparse matrix with 10 rows and 10 columns, where the two horizontal axes are the rows and columns of the matrix and the vertical axis is the value of the matrix at that position, (2) is a matrix reconstructed using three sets of singular values ​​and singular vectors, (3) is a matrix reconstructed using five sets of singular values ​​and singular vectors, and (4) is a matrix reconstructed using seven sets of singular values ​​and singular vectors.

[0031] Figure 3 It is the overall framework of the model of the present invention; represents the jth singular value of the i-th feature map, Represents the jth left singular vector of the i-th feature map.

[0032] Figure 4 is the hyperparameter setting of the one-dimensional convolutional neural network; where (1) is a one-dimensional convolutional neural network with a kernel length of 3 and a step size of 1, (2) is a one-dimensional convolutional neural network with a kernel length and a step size of FZ, and (3) is a one-dimensional convolutional neural network with a kernel length and a step size of k×FZ; where represents the jth singular value of the i-th feature map, Represents the jth left singular vector of the i-th feature map.

[0033] Figure 5: This figure is a comparison of the best confusion matrices obtained by the traditional residual neural network model and the large-stride kernel convolution of the second layer one-dimensional convolutional neural network in the model of the method of the present invention, where the kernel length and step size are both K×FZ; wherein (a) is the decoding result of the traditional residual neural network model, and (b) is the decoding result of the large-stride kernel convolution in the model of the method of the present invention.

[0034] Figure 6 This is a graph showing the relationship between the number of training times and the accuracy of the model of the present invention under different parameter setting methods of the second layer one-dimensional convolutional neural network with large kernel strided convolution.

[0035] Figure 7 This is the feature fusion method proposed in the present invention.

[0036] Figure 8 A special attention mechanism proposed for the present invention; wherein Represents the j-th singular value of the i-th feature map. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0038] The present invention provides a method for identifying a second trimester prenatal grade II ultrasound standard fetal plane, comprising: Pre-train the residual neural network model based on the second trimester prenatal grade II ultrasound standard fetal planar image set, retain the convolutional layer of the pre-trained residual neural network model (i.e., freeze the parameters of the convolutional layer in the pre-trained residual neural network model, delete the global average pooling layer and the fully connected layer, and only retain the convolutional layer), and generate a feature map [n, FZ, FZ] based on the convolutional layer of the residual neural network model, where the first and second FZs represent the width and height of the feature map, respectively, and n represents the number of feature maps generated; The singular vectors and singular values ​​after feature graph decomposition are combined to construct the matrix gene vector MGV, and a two-layer one-dimensional convolutional neural network ( Figure 3 The two-layer one-dimensional convolutional neural network is simplified and only one one-dimensional convolutional neural network is used to represent it) to decode the matrix gene vector MGV, or to decode the matrix gene vector MGV through the feature fusion method, that is, fusing the two parameter setting methods of the convolution kernel length and the step length of the second layer of the one-dimensional convolutional neural network with the strided kernel convolution, and output the classification result of the second trimester prenatal grade II ultrasound standard fetal plane image.

[0039] The overall framework of the model constructed by the method of the present invention is as follows Figure 3 As shown, is the jth singular value of the i-th feature map; is the jth left singular vector of the i-th feature map; [K×FZ,n], [n,FZ,FZ], etc. represent the tensor shape of the corresponding data, and T represents transpose.

[0040] The following is the specific implementation process of the present invention.

[0041] Step 1 (data collection): GE E-8 advanced color Doppler ultrasound machine and Samsung WS80 advanced color Doppler ultrasound machine were used for data collection. Ten images were collected, including the cross section of the lateral ventricle, the cross section of both eyes, the cross section of the kidney, the sagittal plane of the kidney, the cross section of the thalamus, the cross section of the cerebellum, the coronal plane of the nasolabial lip, the sagittal plane of the spine, the cross section of the abdomen, and the mid-sagittal plane of the face, and were labeled from 0 to 9. The shape of the image was [3,768,576]. Then bilinear interpolation was used to uniformly adjust it to [3,200,200].

[0042] Step 2 (pre-training step): The present invention uses a common basic residual block as a basic unit of the residual neural network. The model structure of the residual neural network of the present invention is shown in Table 1: Table 1 As can be seen from Table 1, the shape of the input data of the present invention is [64, 3, 200, 200], where 64 is the batch size, 3 is the number of channels of the picture, and 200 is the width and height of the picture. Then the traditional model is trained on the 10-class dataset of the standard section of the grade II fetus of the present invention. After the model achieves a better effect, the present invention removes the 5th layer of global average pooling layer and the 6th layer of fully connected layer. At this time, the model now acts as a feature extractor to extract image features and convert them into feature maps. Its output is [64, 64, 10, 10], the first 64 is the batch size, the second 64 is the number of feature maps for each picture, and the following two 10s represent the width and height of the output feature map.

[0043] Step 3 (singular decomposition step): The present invention uses singular value decomposition technology to perform singular value decomposition on the extracted feature graph. Decompose into 10 left singular vectors, 10 right singular vectors and 10 singular values. Then sort the singular values ​​in order from large to small. The length of each singular vector is 10.

[0044] Step 4 (weighted combination step): Then take the first 5 largest singular values ​​and the corresponding left singular vectors for each feature map. Multiply these singular values ​​by the corresponding left singular vectors. Then concatenate them into a large vector called the matrix gene vector MGV. The shape of the resulting data tensor is [64, 1, 3200], where 64 is the batch size and 3200 is the length of the matrix gene vector.

[0045] The calculation process for the length of the matrix gene vector is as follows: each image is converted into 64 feature maps with 10 rows and 10 columns, and the first 5 singular values ​​of each feature map are multiplied by the corresponding left singular vector. Then they are all concatenated into a large vector, so it is 64*5*10=3200.

[0046] Step 5 (Decoding step): The steps of using stride-cross-core convolution decoding are as follows: The matrix gene vector MGV is input into the "large kernel strided convolution" and the fully connected layer, and its structure is shown in Table 2: Table 2 There are two parameter setting methods for the second layer of one-dimensional convolutional neural network: a=b=10 and a=b=50. The output of the fully connected layer finally has an output shape of [64, 10], where 64 is the batch size and 10 is the number of task classifications of the present invention.

[0047] The decoding steps of the new structure that combines two stride large kernel convolutions are as follows: This example shows the data flow and the data shape of the intermediate process. For branch 1, the input channel of the first convolution is 1 and the output channel is 18, and the kernel length and stride are both 1. For the second convolution, the input channel is 18 and the output channel is 36, and the kernel length and stride are both 10. For the one-dimensional maximum pooling layer and the one-dimensional average pooling layer, the window size and stride are both 5.

[0048] The shape of the data in the intermediate process: the output shape of the first convolution of MGV input is [64,18, 3200], then enters the Mish activation function, and then enters the second convolution to obtain a shape of [64,36,320], and then enters the Mish activation function. Then this data enters the one-dimensional maximum pooling layer and the one-dimensional average pooling layer respectively. The output data of the two layers are both in the shape of [64,36,64]. Finally, the output data of the two layers are added together to obtain data of the shape of [64,36,64] as the output of branch 1.

[0049] For branch 2, the input channel of the first convolution is 1 and the output channel is 18, and the kernel length and stride are both 1. For the second convolution, the input channel is 18 and the output channel is 36, and the kernel length and stride are both 50.

[0050] The shape of the data in the intermediate process: The shape of the data obtained by the MGV data input attention mechanism is [64, 1, 3200]. The shape is not changed but the weight is assigned. Then the data is input into the first one-dimensional convolutional neural network and the shape is [64, 18, 3200]. Then the Mish activation function is input, and then the output shape of the second one-dimensional convolutional neural network is [64, 36, 64]. Then the Mish activation function is input to get the output of branch 2.

[0051] Finally, the outputs of branch 1 and branch 2 are added together, and the resulting data shape is [64, 36, 64]. Then, it enters the flattening layer and the fully connected layer. The resulting data shape is [64, 10], which is the classification result of 64 fetal cross-section images.

[0052] Step 6 (model parameter update): The present invention adopts the back propagation algorithm to update the parameters of the singular pooling layer, the learning rate is 0.001, the optimizer is Adam, the loss function is MSELoss, and training is performed 30 times.

[0053] experiment: First, the data set collected includes 10 standard sections of level II ultrasound examinations, among which the data collected in X year Y month constitutes the training set, with a total of 1289 images, and the data collected in X year Z month is used as the test set, with a total of 580 images.

[0054] The experimental results of the present invention include: 1. Performance comparison of the model of the present invention and other 6 models. 2. Ablation experiment of singular pooling layer. 3. Ablation experiment of two parameter setting methods of one-dimensional convolution of strided kernel convolution. 4. Performance of the new structure that combines two hyperparameter setting methods of strided kernel convolution.

[0055] (1) Performance comparison between the model of the present invention and the other six models The model of the present invention is compared with two Transformer-based models and four convolutional neural network-based models. The performance comparison is shown in Table 3: Table 3 The present invention observed that the Transformer-based model performed poorly. This is mainly due to the relatively small number of pictures in the dataset of the present invention. At the same time, the parameters of the two Transformer-based models are too large, which inevitably leads to overfitting of the model during the training process. In contrast, the convolutional neural network-based model with fewer parameters showed better performance, and the model of the present invention achieved the best performance in multiple key indicators, proving its advantages in dealing with such problems. In addition, the accuracy of the model of the present invention is significantly better than other models.

[0056] (2) Ablation experiment of singular pooling layer (ablation experiment of two types of large-stride convolution) The core goal of this experiment is to verify the effectiveness of the singular pooling layer in the deep learning model. The experiment first trained the traditional residual neural network and recorded the performance index values ​​obtained on the test set as a benchmark comparison. Subsequently, in order to introduce the singular pooling layer, the present invention froze the parameters of the convolutional neural network part in the residual neural network architecture, and replaced the original global average pooling layer and fully connected layer with the singular pooling layer. The key to the experiment is to compare the performance differences between the original residual neural network (using global average pooling and fully connected layers) and the model replaced with the singular pooling layer. The comparison results are shown in Table 4: Table 4 Parameter setting 3 in Table 4 is a one-dimensional convolutional neural network parameter setting using a convolution kernel size and a step size of 50, that is, the sum of the lengths of all left singular vectors of a feature map is used as the standard. Parameter setting 2 is a convolution kernel size and a step size of 10, that is, the length of a left singular vector is used as the standard. Parameter setting 1 is a one-dimensional convolutional neural network parameter setting method using a traditional convolution kernel size of 3 and a step size of 1. It is easy to find that the singular pooling layer using the two one-dimensional convolutional neural network parameter setting methods proposed in the present invention can achieve an effect that is significantly higher than that of a traditional residual neural network.

[0057] In order to compare the model performance in depth, the present invention plots Figure 5 The confusion matrix in , intuitively shows the classification effect of the model using parameter setting 3 of the present invention and the traditional residual neural network model on a given test set. Figure 5As shown, although the residual neural network model performs well overall, it has obvious deficiencies when dealing with specific categories. Specifically, the residual neural network model shows limited discrimination ability when identifying category 0 (lateral ventricle cross section), category 4 (thalamus cross section) and category 5 (cerebellum cross section), and often misclassifies images of category 0 and category 5 as category 4. The difficulty in distinguishing these three planes mainly stems from their locations in the fetal head region. Anatomically, the head region is full of complex details, which becomes more challenging due to the inherent ambiguity of ultrasound images. In addition, categories 0, 4, and 5 show considerable similarities in shape and size, which further exacerbates the difficulty of classification. Therefore, accurately distinguishing these images requires a model with robust detail extraction capabilities. The traditional global average pooling layer has obvious information loss when extracting detailed information, resulting in its poor performance in distinguishing the above three categories. In contrast, the singular pooling layer introduced in the present invention effectively captures the complex details in the image, not only improves the recognition performance of these three planes, but also maximizes the model's prediction accuracy for other planes. The model of the present invention not only maintains a high level of overall classification performance, but also achieves significant improvements in distinguishing these three key categories. It is worth noting that the performance of the 5th and 1st category predictions is significantly improved, reaching a level close to clinical application. This comparison further demonstrates the effectiveness of the singular pooling layer of the present invention in extracting image detail information.

[0058] (3) Ablation experiments on two parameter setting methods of one-dimensional convolution for strided kernel convolution.

[0059] The relationship between the number of training times and the accuracy of the strided kernel convolution of the present invention under different second-layer one-dimensional convolutional neural network parameter setting methods is as follows: Figure 6 .

[0060] Figure 6 The dotted line in represents the performance of the traditional residual neural network model on the dataset. Figure 6 The parameter setting method (3) uses a convolution kernel size and a step size of 50. Figure 6 The parameter setting method (2) uses a convolution kernel size and a step size of 10. The parameter setting method (1) uses a traditional one-dimensional convolutional neural network parameter setting method with a convolution kernel size of 3 and a step size of 1. It can be easily observed that Figure 6 Parameter setting method (3) and Figure 6 The parameter setting method (2) only needs 1-2 training times to achieve better results, while Figure 6 The parameter setting method (1) requires more training to filter out invalid information. Figure 6 The performance of the model with parameter setting method (3) is higher than Figure 6This further proves the effectiveness of the "large kernel strided convolution" proposed by the present invention as a decoder.

[0061] (4) The performance of the new structure that combines the two large-stride kernel convolution parameter setting methods is shown in Table 5: Table 5 It can be found from Table 5 that the performance of the new structure is much higher than that of the traditional residual neural network model. Compared with the two parameter setting methods of strided convolution, the new structure also has obvious advantages and obtains higher accuracy and F1 score.

[0062] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound, characterized in that: include: Pre-training a residual neural network model based on a second trimester prenatal grade II ultrasound standard fetal planar image set, retaining the convolutional layer of the pre-trained residual neural network model, and generating a feature map of shape [n, FZ, FZ] based on the convolutional layer of the residual neural network model, where n represents the number of feature maps generated, and the two FZs represent the width and height of the feature map, respectively; The singular vectors and singular values ​​after feature map decomposition are combined to construct a matrix gene vector MGV, and the matrix gene vector MGV is decoded by a two-layer one-dimensional convolutional neural network with different strided kernel convolution parameters, or by a feature fusion method, i.e., fusing the convolution kernel length and step length of the second layer of the one-dimensional convolutional neural network with strided kernel convolution. The classification results of the second trimester prenatal grade II ultrasound standard fetal plane images are output.

2. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 1, characterized in that: Before pre-training the residual neural network model, the second-trimester prenatal level II ultrasound standard fetal plane image set needs to adjust the second-trimester prenatal level II ultrasound standard fetal plane images in the second-trimester prenatal level II ultrasound standard fetal plane image set to a uniform size using bilinear interpolation.

3. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 1, characterized in that: The singular vectors and singular values ​​after decomposition of the feature graph are combined to construct the matrix gene vector MGV, which is specifically implemented as follows: Singular value decomposition is used to decompose the feature map into a left singular vector matrix, a right singular vector matrix and a singular value matrix, that is, A=USV T ; where A represents the feature map; U represents the left singular vector matrix, and ,u i is a left singular vector, 1≤i≤FZ; V represents a right singular vector matrix, and , v i are right singular vectors; S represents the singular value matrix, and , diag represents a diagonal matrix, is a singular value, and the singular values ​​are sorted from large to small; Select the top k largest singular values ​​of each feature map The corresponding left singular vector Multiply the corresponding ; After completing the product of the first k singular values ​​of all feature maps and the corresponding left singular vectors, they are spliced ​​into a large vector, namely the matrix gene vector MGV.

4. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 3, characterized in that: The kernel length and stride length of the first layer of the one-dimensional convolutional neural network of the strided kernel convolution are both 1, and the kernel length and stride length of the second layer of the one-dimensional convolutional neural network of the strided kernel convolution are both FZ or k×FZ.

5. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 4, characterized in that: The feature fusion method is specifically implemented as follows: Construct two branches. In branch 1, the convolution kernel length and step size of the first layer of the one-dimensional convolutional neural network are both 1, and the convolution kernel length and step size of the second layer of the one-dimensional convolutional neural network are both FZ. In branch 2, the convolution kernel length and step size of the first layer of the one-dimensional convolutional neural network are both 1, and the convolution kernel length and step size of the second layer of the one-dimensional convolutional neural network are both k×FZ. For branch 1, the matrix gene vector MGV sequentially enters the first layer of one-dimensional convolutional neural network with a kernel length and a step size of 1, the Mish activation function, the second layer of one-dimensional convolutional neural network with a kernel length and a step size of FZ, and the Mish activation function, and then the output enters the one-dimensional maximum pooling layer with a window size and a step size of k, and the one-dimensional average pooling layer with a window size and a step size of k, respectively. Then, the outputs of the one-dimensional maximum pooling layer and the one-dimensional average pooling layer are added to obtain the output of branch 1; For branch 2, the matrix gene vector MGV sequentially enters the attention mechanism, the first layer of one-dimensional convolutional neural network with a kernel length and a step size of 1, the Mish activation function, the second layer of one-dimensional convolutional neural network with a kernel length and a step size of k×FZ, and the Mish activation function to obtain the output of branch 2; The outputs of branch 1 and branch 2 are added to obtain the result of matrix gene vector MGV decoding.

6. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 5, characterized in that: In branch 2, the attention mechanism uses the sum of the first k singular values ​​of each feature map as its attention score, and then inputs the attention scores of all feature maps into the soft maximization layer SoftMax to obtain the corresponding attention weights, and finally multiplies these attention weights by the matrix gene vector MGV of the corresponding feature map.

7. The method for identifying the standard fetal plane of second trimester prenatal grade II ultrasound according to claim 5, characterized in that: The result after matrix gene vector MGV decoding is passed through the flattening layer and the fully connected layer in turn to output the 10 classification results of the second trimester prenatal grade II ultrasound standard fetal plane image.

Images