Method, device and ultrasonic imaging display system for generating standard sections of fetal brain

Abstract

The present invention provides a method, device and ultrasonic imaging display system for generating standard sections of the fetal brain. The method specifically includes collecting ultrasonic three-dimensional volume data, and correcting the 3D volume data so that it is located in the same reference coordinate system. By sampling the three-dimensional volume data, a two-dimensional section image is obtained and regression labels are set. Based on convolutional neural network training, a corresponding function relationship between the two-dimensional section image and the coordinates is obtained. The trained network can simultaneously predict the coordinates of multiple standard planes and automatically generate multiple standard section images. Based on the present invention, there is no need to repeat 2D ultrasonic scanning, and some sections that are difficult to obtain are quickly generated, which is convenient for doctors to diagnose; at the same time, doctors only need to collect 3D volume data, and then they can generate multiple fetal brain diagnostic section images by collecting standard two-dimensional section images once, which reduces the inspection time, improves the utilization rate of 3D data, and reduces the dependence on professional ultrasound doctors. It has very important medical significance.

Description

Technical Field

[0001] The present invention relates to deep learning technology, and more specifically to the field of mid-pregnancy tertiary prenatal examination, and more specifically to a processing technology for generating standard sections for diagnosing fetal cranial malformations using big data and deep learning. Background Art

[0002] Fetal cranial and brain structural malformation is one of the common congenital malformations. Ultrasound is the first choice and important means of screening fetal cranial and brain structure.

[0003] At present, the diagnosis of fetal cranial structural malformations in the second trimester is mainly performed through level III screening using three standard planes (SP) of two-dimensional (2D) ultrasound (US). However, there is a certain risk in diagnosing only from standard sections, and lesions other than standard sections may be missed, so it is necessary to increase sagittal and coronal section observations to reduce the missed diagnosis rate. It is difficult for doctors to scan dozens of ultrasound SP images at a time for diagnosis, which is time-consuming and heavily operator-dependent. Three-dimensional (3D) ultrasound (US) volume data can contain multiple information, more standard planes, and has low user dependence. If the position information of SPs in 3D space can be found, SPs can be automatically generated in 3D data based on the position information, reducing the inspection time.

[0004] This technology requires a high level of experience from ultrasound doctors. Currently, there is a large shortage of experienced prenatal ultrasound diagnostic doctors in hospitals at all levels. The third-level examination in the second trimester is mainly diagnosed through three standard sections, namely, transthalamic (TT), transventricular (TV), and cerebellar (TC). However, lesions other than these three sections may be missed. In order to improve the accuracy of diagnosis, more sections are needed for auxiliary diagnosis. The midsagittal plane can be used to observe the development of the corpus callosum. Coronal sections such as the midsagittal (MS) section, coronal to the body of the Lateralventricles (CLV) section, cerebellar coronal (CC) section, and coronal to the trigone of the Lateral Ventricles (CTLV) section can further diagnose related cranial structures. Therefore, a series of sagittal and coronal sections are added for related diagnosis. With the rapid development of computer image analysis technologies such as neural networks, it has become possible to obtain image information for auxiliary diagnosis of diseases through large-sample machine learning. Through a large amount of training, models trained by deep learning can replace doctors to diagnose and identify medical images. Using deep learning models, multi-parameter regression learning can be performed. In the three-dimensional volume data space, three points can determine a plane. Knowing the coordinates of the three points of the standard section, it is possible to locate and generate the standard section image in the three-dimensional space, and realize the automatic generation of a series of auxiliary diagnosis section images with one click.

[0005] In the existing technology, two-dimensional ultrasound is mainly used for scanning, and the doctor manually scans to obtain three standard sections. However, due to the fetal position, it takes a lot of time to scan the three standard sections. It is possible that three standard sections cannot be obtained in one scan, and multiple scans are required. In addition, it is difficult to obtain a series of section images of the sagittal and coronal planes with two-dimensional ultrasound. 3D US can contain multiple standard planes, and has low user dependence, high efficiency, simple operation process, and fast imaging speed. However, because the position of the standard plane in the three-dimensional data is unknown, the amount of data generated by the three-dimensional volume data is large, and it takes a lot of time to search one by one manually, which lacks feasibility. If multiple standard SPs and corresponding images can be automatically located in 3D ultrasound, it can reduce dependence on professional doctors, shorten prenatal examination time, and improve the accuracy of diagnosis. Therefore, it is very important to find a method that can locate the standard plane of three-dimensional data in any posture and automatically generate standard sections in three-dimensional space with one click. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a method, device and ultrasound imaging display system for generating standard sections of the fetal brain. The main goal of the present invention is to predict 7 standard diagnostic sections, namely, transthalamic horizontal section (TT), transventricular horizontal section (TV), transcerebellar horizontal section (TC), midsagittal (MS) section, coronal to the body of the lateral ventricles (CLV) section, cerebellar coronal (CC) section, and coronal to the trigone of the lateral ventricles (CTLV) section. Among these planes, the three standard planes of the transverse section can be obtained by two-dimensional ultrasound, but other sections are not easy to obtain by two-dimensional ultrasound. The generated series of cross-sectional images may or may not include standard cross-sectional images, because the position of the standard plane in the three-dimensional data is unknown. The amount of data generated by the three-dimensional volume data is large, and it takes a lot of time to search manually one by one, which is not feasible.

[0007] In a first aspect, the present invention provides a method for generating a standard section of a fetal brain, the method comprising the steps of:

[0008] Collect ultrasonic three-dimensional volume data and calibrate the three-dimensional volume data using affine transformation;

[0009] Sampling the calibrated three-dimensional volume data to generate a series of two-dimensional section images;

[0010] Setting regression labels of all the two-dimensional slice images, and randomly and synchronously rotating the two-dimensional slice images and the regression labels of the two-dimensional slice images;

[0011] Using a convolutional neural network to train the two-dimensional slice image and the corresponding regression label, adding an attention matrix to the convolutional neural network to obtain the trained convolutional neural network, and fitting the convolutional neural network to obtain a functional relationship between the regression label and the two-dimensional slice image;

[0012] A two-dimensional ultrasound system scans to obtain a 2D section image, selects correlated continuous frame 2D section images obtained by ultrasound, inputs the 2D section image into the convolutional neural network, and predicts its corresponding position coordinates in three-dimensional space; based on the positional relationship between the 2D section image and other standard planes, the position coordinates of other standard planes are calculated, and other standard section images are generated by sampling the corresponding coordinate positions in the three-dimensional volume data.

[0013] Specifically, in one embodiment, sampling the calibrated three-dimensional volume data to generate a two-dimensional series of slice images specifically includes:

[0014] Using a Fibonacci sphere, the sampling plane normal vectors are evenly distributed on the surface of the sphere, a number of planes are generated at equal intervals in each normal vector direction, and the three-dimensional volume data is sampled based on the several planes to obtain ultrasonic two-dimensional section data;

[0015] The fetal MRI of the standard atlas is mapped to the same coordinate system according to the same radiometric transformation and used as a pixel-level mask label of the fetal ultrasound brain to extract the region of interest ROI of the ultrasound two-dimensional section data to obtain the two-dimensional series of section images.

[0016] Specifically, in one embodiment, the sampling plane normal vectors are uniformly distributed on the surface of the sphere as follows:

[0017] Fibonacci grid sampling achieves uniform distribution of points on the sphere. P(φ, θ) represents the normal vector of the sampling plane, φ represents the azimuth, and θ represents the elevation. If the number of normal vectors of the sampling plane is n, then and It can be calculated by formula (1) and formula (2) respectively:

[0018]

[0019]

[0020] Among them, φ i ,θ i Represents the azimuth and elevation of the current i-th normal vector, i∈(1,n).

[0021] Specifically, in one embodiment, the convolutional neural network is mainly composed of three parts, including a feature extraction module, an attention module and a prediction module; wherein the feature extraction module is a feature extractor based on the VGG16 network, which is used to extract deep features of the two-dimensional cross-sectional image.

[0022] Specifically, in one embodiment, the attention matrix specifically includes:

[0023] Input M two-dimensional slice images are the height and width of the two-dimensional slice image respectively;

[0024] After passing through the feature extractor, a feature vector is generated Send Vi into the attention mechanism model to generate the attention matrix.

[0025] Specifically, in one embodiment, the attention mechanism model is specifically:

[0026] Calculate the correlation between the target value (target) and the input value (source), wherein the correlation calculation specifically includes: calculating the vector dot product of the two, calculating the vector cosine similarity of the two, or evaluating by introducing an additional multilayer perceptron; wherein the multilayer perceptron is composed of two parts, alpha and beta, and the specific calculation method is formula (3):

[0027] Similarity(target i ,source j )=alpha(target i ,source j )·beta(target i ,source j ) T (3)

[0028] Among them, the target and source tables represent the extracted feature vectors Vi and Vj, which are used to calculate the similarity between the two input two-dimensional section images.

[0029] Specifically, in one embodiment, the 2D cross-sectional image obtained by scanning with the two-dimensional ultrasound system is one of the three standard planes acquired in the clinical two-dimensional ultrasound system: the thalamic horizontal cross-section (Transthalamic, TT), the lateral ventricle horizontal cross-section (Transventricular, TV), and the cerebellar horizontal cross-section (Transcerebellar, TC).

[0030] In one embodiment, inputting the 2D section image into the convolutional neural network to predict its corresponding position coordinates in the three-dimensional space includes:

[0031] Collect two consecutive frames of cross-sectional images with correlation at standard plane positions;

[0032] The feature vectors of two consecutive cross-sectional images after passing through the feature extractor are spliced ​​in pairs to form a new feature vector

[0033] The feature vector P ijInput the fully connected layer of the convolutional neural network and transform the feature vector P ij After dimensionality reduction, the output feature vector is

[0034] Calculate the slice image I using the attention model i The attention weights, combined with all I j The value of is weighted summed to get image I i The weighting coefficient S i ,S i ∈R 1×512 , the specific calculation formula is shown in formula (4):

[0035]

[0036] After the convolutional neural network is applied to S i Perform regression prediction to predict the values ​​of three coordinate points

[0037] In a second aspect, the present invention provides a device for generating a standard section of a fetal cranium, which specifically comprises:

[0038] A first input module is used to access the ultrasonic three-dimensional volume data collected by the three-dimensional ultrasonic system;

[0039] A first data processing module calibrates the three-dimensional volume data by using affine transformation, samples the calibrated three-dimensional volume data, and generates a series of two-dimensional section images;

[0040] A first output module, used for outputting the calibrated three-dimensional volume data;

[0041] A neural network preprocessing module, used for setting regression labels of all the two-dimensional section images, and randomly and synchronously rotating the two-dimensional section images and the regression labels of the two-dimensional section images;

[0042] A neural network module, which adds an attention matrix and is used to train the two-dimensional slice image and the corresponding regression label to obtain a functional relationship between the regression label and the two-dimensional slice image;

[0043] The second input module is used to access the 2D section image on a selected standard section scanned by the doctor;

[0044] The second data processing module is used to input the 2D section image into the convolutional neural network to predict its corresponding position coordinates in the three-dimensional space; according to the positional relationship between the 2D section image and other standard planes, the position coordinates of other standard planes are calculated, and the corresponding coordinate positions in the three-dimensional volume data are sampled to generate other standard section images;

[0045] The second output module is used to output the position coordinates of the 2D section image and the position coordinates of other standard planes, and output the 2D section image and other standard section images at the same time.

[0046] In a third aspect, the present invention proposes an ultrasound imaging display system, which includes a probe, a display, and the fetal cranial standard section generation device according to claim 9, wherein the first input module and the second input module of the fetal cranial standard section generation device are electrically connected to the probe, and the display is electrically connected to the first output module and the second output module of the fetal cranial standard section generation device, and is characterized in that it includes:

[0047] The probe comprises a first acquisition mode and a second acquisition mode, wherein the first acquisition mode is used to acquire three-dimensional volume data of the subject, and the second acquisition mode is used to acquire two-dimensional cross-sectional images of the subject in real time;

[0048] The fetal cranial standard section generating device generates a standard three-dimensional model of the fetal cranial brain of the subject according to the three-dimensional volume data, and the position coordinates of each standard section; obtains the position coordinates of the corresponding three-dimensional space and the section images of each standard section according to the two-dimensional section image;

[0049] A display, comprising a first display area, a second display area, and a third display area, wherein the first display area is used to display the standard three-dimensional model, and display marks are set on the standard three-dimensional model to distinguish and mark the positions of various standard sections;

[0050] The second display area is used to display the two-dimensional section image in real time;

[0051] The third display area is used to display the section images of each standard section.

[0052] Specifically, in one embodiment, the first display area is provided with a human-computer interaction interface or interface control; the display mark includes reference lines or reference planes of different colors; the display mark and the standard three-dimensional model are linked, and the human-computer interaction device or the interface control is used to translate or rotate the standard three-dimensional model and the display mark.

[0053] Specifically, in one embodiment, according to the position coordinates of the two-dimensional slice image in three-dimensional space, a display mark corresponding to the two-dimensional slice is set at a corresponding coordinate position in the three-dimensional body model in the first display area.

[0054] Specifically, in one embodiment, the second display area also includes a setting module for setting a target section, characterized in that a movement strategy of the probe is determined based on a two-dimensional section acquired by the probe in the second mode and a coordinate position of the target section in the three-dimensional model; the movement strategy includes a direction and an angle of movement of the probe.

[0055] In a fourth aspect, the present invention proposes an electronic device, characterized in that it includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include a method for executing any one of the fetal cranial standard section generation methods.

[0056] In a fifth aspect, the present invention proposes a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any one of the methods for generating standard sections of the fetal skull and brain are implemented.

[0057] In summary, compared with the prior art, the method, device and ultrasound imaging display system for generating standard sections of the fetal brain based on the present invention achieve the following beneficial technical effects.

[0058] (1) A method for generating standard sections of the fetal brain and skull of the present invention collects ultrasonic three-dimensional volume data, corrects the 3D volume data so that it is located in the same reference coordinate system, samples the three-dimensional volume data to obtain a two-dimensional section image and sets a regression label, and obtains a corresponding function relationship between the two-dimensional section image and the coordinates based on convolutional neural network training. The trained network can simultaneously predict the coordinates of multiple standard planes and automatically generate multiple standard section images. On the one hand, there is no need to repeat 2D ultrasonic scanning, and some sections that are difficult to obtain can also be quickly generated, which is convenient for doctors to diagnose. On the other hand, for doctors, they only need to collect 3D volume data, and then they can generate 7 (or more) standard sections for fetal brain diagnosis by inputting the two-dimensional image of the standard section, which improves the utilization rate of data and reduces the dependence on professional ultrasound doctors. It has very important medical significance.

[0059] (2) The present invention provides a method for generating standard sections of the fetal brain. When obtaining a two-dimensional section image, the present invention uses the fetal MRI of the standard atlas to extract the ROI of the ultrasound image, thereby eliminating background information that causes noise in network training, improving the accuracy of neural network training, and enhancing the contrast of the image, which is helpful for extracting features in the convolutional neural network.

[0060] (3) A method for generating standard sections of the fetal skull and brain according to the present invention performs random rotation within the plane generated by three-dimensional sampling before training the neural network to increase data diversity and avoid overfitting caused by the convolutional neural network; an attention matrix is ​​added to the training neural network to improve the prediction accuracy of the neural network model.

[0061] (4) The present invention provides a method for generating standard sections of the fetal brain. During continuous sampling, there is a certain correlation between the sampled ultrasound images. In order to fully utilize the correlation between continuous ultrasound frame images, the present invention performs pairwise splicing of feature vectors of two, three or more continuous section images to improve the accuracy of the prediction of corresponding coordinate points of 2D section images.

[0062] (5) The ultrasound imaging display system of the present invention can help doctors dynamically know the exact position of the current ultrasound probe acquisition position corresponding to the fetal skull and brain by dynamically displaying the three-dimensional model of the fetal skull and brain and the two-dimensional section image acquired by the ultrasound probe in the current mode, and displaying the position between the two in real time. The system can also set the target section to be acquired through the setting module, determine the movement strategy of the probe based on the two-dimensional section acquired by the current probe and the coordinate position of the target section in the three-dimensional model, and assist in guiding the doctor to adjust the direction and movement angle of the probe, which helps the doctor to accurately approach the acquisition of the two-dimensional image of the target section. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 Schematic diagram of (a) Fibonacci sphere sampling, (b) normal vector section, and (c) three basic coordinate points of the method for generating standard sections of the fetal brain provided by an embodiment of the present invention.

[0065] Figure 2 Schematic diagram of (a) a two-dimensional image generated by sampling - (b) a two-dimensional section image after three-dimensional calibration - (c) a two-dimensional section image after contrast enhancement in the method for generating standard sections of the fetal brain provided by an embodiment of the present invention.

[0066] Figure 3 It is a schematic diagram of the convolutional neural network regression network structure of the method for generating standard sections of the fetal cranium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the features of the methods, devices, systems and embodiments in the following embodiments can be combined with each other without conflict.

[0068] In view of the shortcomings of the prior art, the present invention proposes a method, device and ultrasound imaging display system for generating standard sections of the fetal brain. The main goal of the present invention is to predict 7 standard diagnostic sections, namely, transthalamic horizontal section (TT), transventricular horizontal section (TV), transcerebellar horizontal section (TC), midsagittal (MS) section, coronal to the body of the lateral ventricles (CLV) section, cerebellar coronal (CC) section, and coronal to the trigone of the lateral ventricles (CTLV) section. Among these planes, the three standard planes of the transverse section can be obtained by two-dimensional ultrasound, but other sections are not easy to obtain by two-dimensional ultrasound. The generated series of cross-sectional images may or may not include standard cross-sectional images, because the position of the standard plane in the three-dimensional data is unknown. The amount of data generated by the three-dimensional volume data is large, and it takes a lot of time to search manually one by one, which is not feasible.

[0069] Specifically, the present invention proposes a method for generating a standard section of the fetal brain, collecting ultrasound three-dimensional volume data, further correcting the 3D volume data so that it is located in the same reference coordinate system, and also obtaining a calibrated standard three-dimensional volume model. Then, according to Fibonacci sphere sampling ( Figure 1 a) Sample the 3D volume data and generate 2D slices for training samples. Extract three coordinate points at fixed positions on the 2D slice ( Figure 1c), as a training label (regression label), used to calculate the loss function in regression learning. Finally, a 2D SP that can be obtained by the 2D US doctor is input into the network to predict their coordinate information in 3D space, and the corresponding section image is extracted from the three-dimensional volume data according to the coordinate information. The trained network can simultaneously predict the coordinates of multiple other standard planes and automatically generate multiple standard sections. There is no need to repeat the 2D ultrasound scan to obtain multiple standard section images. For some sections that are not easy to obtain in 2D ultrasound, they can also be quickly generated, which is convenient for doctors to diagnose. For doctors, they only need to collect 3D volume data, and by inputting two-dimensional standard section images, they can generate 7 standard sections for fetal cranial and brain diagnosis with one click, which improves the utilization rate of data and reduces the dependence on professional ultrasound doctors, which has very important medical significance. At the same time, with the increase of standard section settings in the future, the present invention can set more than 7 standard sections. For example, the present invention can set 12 or more standard sections. The method of the present invention can also be used to generate 12 or more standard sections for fetal cranial and brain diagnosis with one click by inputting two-dimensional standard section images.

[0070] In the following embodiments of the present invention, the specific exemplary technical solutions are for the purpose of examples, and the implementation methods of the technical solutions are not limited thereto.

[0071] First embodiment

[0072] In one embodiment, the present invention provides a method for generating a standard section of a fetal brain, the method specifically comprising the following steps:

[0073] Step S100: The three-dimensional ultrasound system collects ultrasound three-dimensional volume data, and calibrates the three-dimensional volume data using affine transformation.

[0074] Specifically, illustratively, the ultrasound probe of the three-dimensional ultrasound system is used to collect three-dimensional ultrasound volume data, and the three-dimensional volume data is calibrated by affine transformation (three-dimensional rotation, three-dimensional translation, three-dimensional scaling), that is, the collected three-dimensional data is transformed into a unified coordinate system, and the x, y, and z axes of the three-dimensional data are three standard planes, namely, the transverse plane, the coronal plane, and the sagittal plane. In the present invention, a standard three-dimensional model of the fetal skull can be obtained by calibrating the three-dimensional volume data under a unified coordinate system, and the relative position relationship between the standard planes is also determined. The relative position relationship can be understood as that when the coordinates of a set standard section change, the position coordinates of other standard sections after the change can be determined according to the relative position relationship.

[0075] Step S200: sampling the calibrated three-dimensional volume data to generate a series of two-dimensional section images.

[0076] Specifically, exemplarily, according to Fibonacci sphere sampling ( Figure 1 a) Sampling the 3D volume data to generate 2D sections for training samples; for example, extracting planes at different angles from the 3D volume data, the sections generated by sampling should be evenly distributed in the 3D volume data and should include all possible sections, that is, generating several planes at equal intervals in each normal vector direction (e.g. Figure 1 b) Collect several planes in all normal vector directions. Consider the 3D volume data as a unit sphere, and the sampling plane normal vectors should be evenly distributed on the surface of the sphere. Fibonacci grid sampling can achieve uniform distribution of points on the sphere, which is represented by P(φ, θ), where φ represents the azimuth and θ represents the elevation. Assuming that the number of normal vectors of the sampling plane is n, then and It can be calculated by formula (1) and formula (2) respectively.

[0077]

[0078]

[0079] Among them, φ i ,θ i Represents the azimuth and elevation of the current i-th normal vector, i∈(1,n).

[0080] For example, before the three-dimensional volume data is sampled, it needs to be preprocessed so that the three-dimensional fetal cranial volume data and its corresponding pixel-level label data are located in the same coordinate system. Figure 2 a to Figure 2 b) so that the size and direction of all two-dimensional section images are in the same coordinate system. Furthermore, since the three-dimensional volume data itself contains a large amount of background information, it is necessary to extract the region of interest (ROI) of all two-dimensional section images before training the neural network. Exemplarily, the preprocessing specifically includes mapping the fetal MRI image of the standard atlas to the same coordinate system according to the same radiometric transformation, as a pixel-level mask label of the fetal ultrasound brain, for extracting the ROI of the ultrasonic two-dimensional section image. The present invention may also use other methods that are helpful for extracting the ROI of the two-dimensional section image. The above exemplary technical solutions are all for illustrative purposes, and the implementation methods of the technical solutions are not limited to this. Through the above processing, on the one hand, by calibrating the three-dimensional ultrasound volume data, the correspondence between all two-dimensional sections and coordinates is established; on the other hand, by extracting the ROI of the ultrasound image, the background information that causes noise in the network training is eliminated, which can improve the accuracy of the network training.

[0081] Step S300: setting regression labels of all the two-dimensional slice images, and randomly and synchronously rotating the two-dimensional slice images and the regression labels of the two-dimensional slice images.

[0082] Specifically, for example, for the generated two-dimensional section image, the size is (L*L). Generally speaking, any point on the plane can be selected as the regression label. Set the center point P of the two-dimensional section c =(L / 2,L / 2), the midpoint of the line connecting the lower left corner and the center point is P l =((L / 4,L / 4)+Pc) / 2, the midpoint of the line connecting the lower right corner and the center point is P r =((L / 2+L / 4,L / 4)+Pc) / 2 as the basic coordinate point, that is, the regression label, that is, the image label corresponding to the two-dimensional section image, such as Figure 1 As shown in c.

[0083] Exemplarily, the three-dimensional volume data and the corresponding pixel-level mask labels are rotated and sampled in the same way, which ensures that the generated two-dimensional section data and mask data are consistent. Since the normal vector of the section is constant, and for the fetal cranial section, if too many sections are generated in the same direction, it is easy to cause overfitting. The present invention uses random generation of countless section images for training. Therefore, after the three-dimensional sampling generates the section plane, random rotation is performed within the plane to increase the diversity of the data, while avoiding overfitting.

[0084] For example, since the boundary between the ROI area and the background area of ​​the ultrasound image is relatively fuzzy and the ultrasound image contains more noise, the present invention may also include enhancing the contrast of the two-dimensional cross-sectional image to help extract features in the convolutional neural network. Commonly used contrast enhancement techniques can be used in this solution. Specifically, Figure 2 As shown in c.

[0085] Step S400, adding an attention matrix to a convolutional neural network, using the convolutional neural network to train the two-dimensional section image and the corresponding regression label to obtain a trained convolutional neural network, and fitting the convolutional neural network to obtain a functional relationship between the regression label and the two-dimensional slice image.

[0086] Specifically, exemplarily, the convolutional neural network is mainly composed of three parts, including a feature extraction module, an attention module and a prediction module. The feature extraction module uses the VGG16 network as a feature extractor to extract deep features of the two-dimensional cross-sectional image. The main framework of the network is as follows: Figure 3 As shown. The attention module mainly consists of two parts. The first part is the multi-layer perceptron module. Figure 3The alpha and beta modules in the figure are as follows: Figure 3 As shown in the lower left corner, a multi-layer perceptron is formed, which forms an attention module. After the feature vector extracted by the feature extractor is input into the Alpha and Beta parts, a matrix multiplication operation is performed to generate an attention matrix. By adding an attention matrix to the neural network, the prediction accuracy of the model can be improved.

[0087] For example, the VGG16 network structure contains 16 network layers with parameters, namely 13 convolutional layers, 5 pooling layers, and 3 fully connected layers. The specific network structure is as follows: Figure 3 As shown. vgg16 is divided into six modules, as shown in the dark and light colors in the figure. Each module contains two parts: convolution and pooling. The size of the convolution kernel is 3*3. The pooling selects the maximum pooling with a size of 2*2. The input image size in the figure is 160*160*1, 160*160 is the length and width of the image, 1 is the number of channels, indicating a grayscale image, 64, 128, 256, 512, 512 are the output channel numbers of each module respectively, and the Relu activation function is selected. Finally, the global average pooling (Avgpool) is used to obtain the final feature vector. The one-dimensional feature vector contains a series of information such as the edge, contour, and key structure of the image, which is used to calculate the corresponding weights in combination with the attention module in the subsequent step.

[0088] Among them, illustratively, the multilayer perceptron (MLP) mainly consists of an input layer, a hidden layer, and an output layer, and outputs a feature vector extracted by a feature extractor. The input layer is represented by a vector x, and the output of the hidden layer is f(wx+b), w is a weight (also called a connection coefficient), b is a bias, and the function f can be a commonly used sigmoid function: the introduction of the sigmoid activation function in the multilayer perceptron can introduce nonlinear factors into neurons, so that the neural network can arbitrarily approximate any nonlinear function, thereby improving the utilization rate of the model.

[0089] Specifically, exemplary, calculating the attention matrix specifically includes, inputting M two-dimensional slice images are the height and width of the two-dimensional cross-sectional image, respectively. After passing through the feature extractor, a feature vector is generated. Vi is sent to the attention mechanism model respectively to generate the attention matrix. Exemplarily, the attention mechanism model can assign different weights to the target value and the non-target value, that is, the closer the predicted value is to the input value, the greater the weight is, and vice versa. The attention model weight is usually calculated by calculating the similarity or correlation between the target value (target) and the input value (source). The most common methods include: finding the dot product of the vectors of the two, finding the Cosine similarity of the vectors of the two, or by introducing an additional neural network (e.g., a multilayer perceptron MLP) to evaluate. Exemplarily, using MLP (i.e. Figure 3 The alpha and beta modules in the equation are used to calculate the similarity between the two. The calculation formula is shown in equation (3).

[0090] Similarity(target i ,source j )=alpha(target i ,source j )·beta(target i ,source j ) T (3)

[0091] Among them, · represents the vector dot product, and the target and source tables represent the extracted feature vectors Vi and Vj, which are used to calculate the similarity between the two input two-dimensional section images.

[0092] Similarity(target i ,source j ) combines the information of two feature vectors, whose size is M×M. In order to facilitate the calculation of weight coefficients, the attention matrix A(i,j) is used to represent Similarity(target i ,source j ).

[0093] Step S500: The doctor uses an ultrasound system to scan and select a standard section, selects continuous frame 2D section images with correlation on the standard section, inputs the 2D section image into the convolutional neural network, and predicts its corresponding position coordinates in three-dimensional space; based on the positional relationship between the 2D section image and other standard planes, the position coordinates of other standard planes are calculated, and other standard section images are generated by sampling the corresponding coordinate positions in the three-dimensional volume data.

[0094] Specifically, the 2D section image used in the present invention is one of the three standard planes: transthalamic horizontal cross section (TT), transventricular horizontal cross section (TV), and transcerebellar horizontal cross section (TC). Due to the fetal position, it takes a lot of time to scan the three standard sections, and it is possible that three sections cannot be obtained in one scan. Therefore, using the solution of the present invention, the doctor can scan and obtain the corresponding 2D section image of one of the three standard planes that is easy to collect in the clinical ultrasound system. By inputting the two-dimensional standard section image, all standard section images can be obtained.

[0095] Specifically, exemplarily, in step S500, the 2D section image is input into the convolutional neural network to predict its corresponding position coordinates in the three-dimensional space, including: when continuously acquiring 2D ultrasound images, there is a certain correlation between the acquired ultrasound images. In order to make full use of the correlation between consecutive ultrasound frame images, exemplarily, the consecutive frames can be two consecutive frames, three frames or more frames of two-dimensional section images; exemplarily, in one embodiment, the present invention uses two consecutive frames for illustration, and the feature vectors of the consecutive frame section images are spliced ​​in pairs, so that a contrast is formed between the two frames of section images, such as Figure 3 As shown in the dotted box in . The two are spliced ​​together to form a new feature vector The feature vector P ij Input the fully connected layer to reduce the dimension of the data, and the output feature vector is In order to further predict the coordinate points, the slice image I is calculated i The attention weights, combined with all I j The value of is weighted summed to get image I i The weighting coefficient S i ,Si∈R 1×512 , the specific calculation formula is shown in formula (4):

[0096]

[0097] Finally, the prediction network of the neural network is S i Perform regression prediction to predict the values ​​of three coordinate points

[0098] Through the above method, on the one hand, the feature vectors of the continuous frame 2D scanning images are spliced ​​in pairs to predict the three-dimensional coordinate points, which makes full use of the continuity between the ultrasonic 2D scanning images and improves the accuracy of the prediction of the corresponding coordinate points of the 2D section image. On the other hand, the functional relationship between the coordinates and the two-dimensional section image is fitted by the convolutional neural network to predict the position coordinates of the two-dimensional section image in the three-dimensional space. The convolutional neural network can input a two-dimensional slice image once, output its coordinates in the three-dimensional space, and link the two-dimensional slice image with the three-dimensional coordinates.

[0099] Further, illustratively, in the method for generating standard sections based on the fetal cranium of the present invention, it also includes: during the training process, the mean square error between the predicted coordinate points and the actual regression label coordinate points is used as the loss function. Since the weight of the edge points accounts for a small proportion, the lower left corner and the lower right corner have a small weight during calculation, so the error values ​​of these two points are divided by the corresponding weight W. The specific calculation is formula (5):

[0100]

[0101] Among them, P cGT ,P cpred ,P lGT ,P lpred ,P rGT ,P rpred The table shows the coordinates of the center of the label plane, the center of the prediction plane, the coordinates of the lower left corner of the label plane, the coordinates of the lower left corner of the prediction plane, the coordinates of the lower right corner of the label plane, and the coordinates of the lower right corner of the prediction plane. Because, in the training of convolutional neural networks, the loss function is very critical, which determines the convergence speed of the convolutional neural network and the performance of the network.

[0102] Further, exemplarily, in the method for generating standard sections based on the fetal cranium of the present invention, in order to evaluate the gap between the predicted plane and the standard plane, the method also includes: using 4 evaluation indicators to evaluate the gap between the predicted plane and the standard plane. The angle between two planes can be a good measure of whether the two planes overlap, and the distance between the two planes can be used to judge the gap between the predicted plane and the standard plane, so the angle between the two planes (PlaneAngle, PA) and the distance between the two planes (Plane Distance, PD) are first introduced to evaluate the effectiveness of the proposed method. They are defined as formula (6) and formula (7), respectively.

[0103]

[0104] PD=|D g -D p | (7)

[0105] Among them, n p ,n g Denote the normal vectors of the predicted plane and the standard plane, respectively, and D g ,D p The separate tables show the distance from the standard plane to the volume data center and the distance from the predicted plane to the volume data center.

[0106] Further, exemplarily, in the method for generating standard sections based on the fetal cranium of the present invention, in order to further evaluate the similarity between the two planes, the present invention introduces two indicators, Structural Similarity (SSIM) and Normalized cross-correlation (NCC), to evaluate the structural similarity between the predicted plane and the standard plane. The closer the value of SSIM is to 1, the more similar the two images are. NCC is used to verify the correlation between the two images from the perspective of vectors. When its value is -1, it indicates negative correlation, when its value is 0, it indicates no correlation, and when its value is 1, it indicates positive correlation.

[0107] Exemplarily, in the method for generating standard sections based on the fetal skull and brain of the present invention, specifically, during the execution of the method, N cases of three-dimensional fetal ultrasound three-dimensional data are trained, and one three-dimensional data generates M two-dimensional section images. When generating data, adjust the direction of the normal vector so that the generated section types cover all possible two-dimensional ultrasound section images as much as possible. Three images are randomly selected from the fetal sagittal series images, coronal series images, and transverse series images, and sent to the network for prediction. Specifically, since the coordinates of the three-dimensional space of the standard section are unknown, two-dimensional ultrasound can only obtain its two-dimensional standard section, and cannot obtain its accurate position in the three-dimensional space. Therefore, the goal of the present invention is to obtain the accurate position information of the standard section and its coordinate information in the three-dimensional space to generate the corresponding standard section image. In the three-point coordinate prediction network, the three coordinate points are finally regressed, which are P and c ,P l ,P r Three coordinate points, using the idea of ​​three points in space to determine a plane, generate the corresponding plane in three-dimensional space. In order to accurately generate the corresponding standard section coordinates, the present invention uses the two-dimensional standard section to predict its position information in three-dimensional space. An experienced ultrasound doctor collects a two-dimensional standard section, sends it to the network, and predicts the corresponding three-dimensional coordinates.

[0108] In addition, different doctors will obtain different results when collecting two-dimensional standard sections, and there will be some deviations. This also shows that the position of the standard section is also relative to the three-dimensional volume data. In order to further generate other related standard sections, the present invention is based on the spatial relationship between different standard sections. For example, the present invention further generates other standard sections based on the horizontal cross-section of the thalamus (TT), namely, the horizontal cross-section of the lateral ventricle (TV), the horizontal cross-section of the cerebellum (TC), the median sagittal section (MS), the coronal section of the body of the lateral ventricle (CLV), the coronal section of the cerebellum (CC), and the coronal section of the trigone of the lateral ventricle (CTLV); at the same time, the number of the above-mentioned standard sections can also be adjusted. The commonly used standard sections include 7. In the present invention, the standard section can be more than 7 standard sections, for example, 12 or more standard sections can be set, and the method of the present invention is also applicable to this.

[0109] Second embodiment

[0110] The present invention also provides another embodiment, which provides a device for generating a standard section of a fetal cranium, which specifically includes:

[0111] A first input module is used to access the ultrasonic three-dimensional volume data collected by the three-dimensional ultrasonic system;

[0112] A first data processing module, which calibrates the three-dimensional volume data by using affine transformation, samples the calibrated three-dimensional volume data, and generates a series of two-dimensional section images;

[0113] A first output module, used for outputting the calibrated three-dimensional volume data;

[0114] A neural network preprocessing module, used for setting regression labels of all the two-dimensional section images, and randomly and synchronously rotating the two-dimensional section images and the regression labels of the two-dimensional section images;

[0115] A neural network module, which adds an attention matrix and is used to train the two-dimensional slice image and the corresponding regression label to obtain a functional relationship between the regression label and the two-dimensional slice image;

[0116] The second input module is used to access the 2D section image of the selected standard section scanned by the doctor with associated continuous frames;

[0117] The second data processing module is used to input the 2D section image into the convolutional neural network to predict its corresponding position coordinates in the three-dimensional space; according to the positional relationship between the 2D section image and other standard planes, the position coordinates of other standard planes are calculated, and the corresponding coordinate positions in the three-dimensional volume data are sampled to generate other standard section images;

[0118] The second output module is used to output the position coordinates of the 2D section image and the position coordinates of other standard planes, and output the 2D section image and other standard section images at the same time.

[0119] Third embodiment

[0120] The present invention also provides another embodiment. The present invention proposes an ultrasound imaging display system, which includes a probe, a display, and the fetal cranial standard section generation device as claimed in claim 9, wherein the first input module and the second input module of the fetal cranial standard section generation device are electrically connected to the probe, and the display is electrically connected to the first output module and the second output module of the fetal cranial standard section generation device, and is characterized in that it includes:

[0121] The probe comprises a first acquisition mode and a second acquisition mode, wherein the first acquisition mode is used to acquire three-dimensional volume data of the subject, and the second acquisition mode is used to acquire two-dimensional cross-sectional images of the subject in real time;

[0122] The fetal cranial standard section generating device generates a standard three-dimensional model of the fetal cranial brain of the subject according to the three-dimensional volume data, and the position coordinates of each standard section; obtains the position coordinates of the corresponding three-dimensional space and the section images of each standard section according to the two-dimensional section image;

[0123] A display, comprising a first display area, a second display area, and a third display area, wherein the first display area is used to display the standard three-dimensional body model, and display marks are set on the standard three-dimensional body model to distinguish and mark the positions of various standard sections;

[0124] The second display area is used to display the two-dimensional section image in real time;

[0125] The third display area is used to display the generated section images of each standard section.

[0126] In an embodiment of the present invention, exemplarily, three standard planes are displayed and set in the standard three-dimensional model, corresponding to the three standard planes of the x, y, and z axes of the three-dimensional data, namely the transverse plane, the coronal plane, and the sagittal plane. The first display area is provided with a human-computer interaction interface, and the display mark includes reference lines and reference planes of different colors, or other presentation methods that can distinguish different standard sections. There is a linkage relationship between the display mark and the standard three-dimensional body model. By using human-computer interaction devices or interface controls such as trackballs, knobs, menu buttons, etc. to translate or rotate the standard three-dimensional body model and the display mark (reference line or reference plane), dynamic observation of various angles of the standard three-dimensional body model can be achieved.

[0127] In an embodiment of the present invention, exemplarily, the first display area and the second area have a linkage relationship, and according to the two-dimensional section image acquired in real time by the probe in the second mode, its coordinate position in the three-dimensional model is obtained in real time, and the display mark corresponding to the section is set according to the coordinate position of the three-dimensional coordinate position in the three-dimensional model in the first display area, that is, the reference line or reference plane corresponding to the real-time two-dimensional section image is dynamically displayed in the three-dimensional model. By dynamically displaying the position of the two-dimensional section image acquired by the current probe position, it can help doctors determine in real time the exact position of the currently acquired section position corresponding to the fetal brain.

[0128] In an embodiment of the present invention, exemplarily, further, the second display area of ​​the ultrasound imaging display system also includes a setting module for setting a target section, the target section is one or more of the three standard planes of the cross-section that the doctor wants to collect, that is, one or more of the thalamic horizontal cross-section (Transthalamic, TT), the lateral ventricle horizontal cross-section (Transventricular, TV), and the cerebellar horizontal cross-section (Transcerebellar, TC), among the 7 or more standard diagnostic sections, including the thalamic horizontal cross-section (Transthalamic, TT), the lateral ventricle horizontal cross-section (Transventricular, TV), the cerebellar horizontal cross-section (Transcerebellar, TC), the midsagittal (Midsagittal, MS) section, the coronal to the body of the Lateralventricles (Coronal to the body of the Lateralventricles, CLV) section, the cerebellar coronal (Cerebellar coronal, CC) section, the trigone of the lateral ventricle coronal (Coronal to the trigone of the Lateral The probe has a plurality of standard sections, including the three standard planes of the cross section, such as the transverse section of the ventricles, the CTLV section, etc., among which the three standard planes of the cross section can be obtained by two-dimensional ultrasound, but it is not easy to obtain other section images through two-dimensional ultrasound. In the second mode of the probe, the doctor can set the target section to be collected through the setting module, and determine the movement strategy of the probe according to the two-dimensional section currently collected by the probe and the coordinate position of the target section in the three-dimensional model. Exemplarily, the movement strategy includes the direction and angle of movement of the probe, and is dynamically displayed in the three-dimensional model in the first display area. Through the implementation of the above embodiments, the doctor can be assisted and guided to adjust the direction and movement angle of the probe, which helps the doctor to accurately approach the image of the target section.

[0129] Fourth embodiment

[0130] The present invention also provides another implementation mode. The present invention proposes an electronic device, which specifically includes: a processor 1 and a memory 2.

[0131] The memory 2 is used to store computer programs.

[0132] The memory 2 includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0133] The processor 1 is connected to the memory 2 and is used to execute the computer program stored in the memory 2 so that the device for generating standard sections of the fetal cranium and brain executes the method for generating standard sections of the fetal cranium and brain of the above embodiment.

[0134] Preferably, the processor 1 may be a central processing unit (CPU for short); or an application specific integrated circuit (ASIC for short).

[0135] Fifth embodiment

[0136] The present invention also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed by at least one processor so that the at least one processor performs the steps of the method for generating standard sections of the fetal cranium as in the above-mentioned embodiment.

[0137] In summary, a method, device and ultrasonic imaging display system for generating standard sections of fetal cranial brain of the present invention, using the idea of ​​convolutional neural network regression, solves the problem of positioning standard sections of fetal cranial brain in three-dimensional space. Convolutional neural network can use two-dimensional standard sections to predict its coordinates in three-dimensional space, and then use it to generate standard sections with one key in the future, which greatly shortens the time of prenatal examination and reduces the dependence on professional doctors. For some grassroots hospitals, only three-dimensional fetal ultrasound data needs to be collected to generate a series of standard section images, which will be more conducive to the analysis of fetal generation and development in clinical practice and reduce the rate of missed diagnosis of malformations. The method of the present invention can also be used for other parts, such as the generation of standard sections of ultrasound in the abdomen and heart, and only relevant network training needs to be carried out. It is proved in the actual experiment of the present invention that the method of the present invention is more accurate than the current regression method and reinforcement learning method in positioning the standard sections of fetal cranial brain, and has certain application value. At the same time, by dynamically displaying the position between the three-dimensional model of the fetal brain and the two-dimensional section image acquired by the ultrasound probe in the current mode on the display, it can help the doctor dynamically know that the current acquisition position corresponds to the exact position in the fetal brain, and set the target section to be acquired through the setting module, and determine the movement strategy of the probe based on the two-dimensional section acquired by the probe in the second mode and the coordinate position of the target section in the three-dimensional model, which can assist and guide the doctor to adjust the direction and movement angle of the probe, and help the doctor accurately acquire the image of the target section.

[0138] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for generating standard sections based on fetal cranial structural malformations, characterized in that: The method comprises the steps of: The three-dimensional ultrasound system collects ultrasound three-dimensional volume data and calibrates the three-dimensional volume data using affine transformation; Sampling the calibrated three-dimensional volume data to generate a series of two-dimensional section images; Setting regression labels of all the two-dimensional slice images, and randomly and synchronously rotating the two-dimensional slice images and the regression labels of the two-dimensional slice images; Adding an attention matrix to a convolutional neural network, using the convolutional neural network to train the two-dimensional slice image and the corresponding regression label to obtain the trained convolutional neural network, and fitting the convolutional neural network to obtain a functional relationship between the regression label and the two-dimensional slice image; The doctor selects a standard section by scanning with an ultrasound system, selects a continuous frame 2D section image with correlation on the standard section, inputs the 2D section image into the convolutional neural network, and predicts its corresponding position coordinates in the three-dimensional space; calculates the position coordinates of other standard planes according to the positional relationship between the 2D section image and other standard planes, and samples the corresponding coordinate positions in the three-dimensional volume data to generate other standard section images; The 2D cross-sectional image is input into the convolutional neural network to predict its corresponding position coordinates in the three-dimensional space, specifically including: Collect two consecutive frames of cross-sectional images with correlation at standard plane positions; The feature vectors of two consecutive cross-sectional images after passing through the feature extraction module are spliced ​​in pairs to form a new feature vector The new feature vector P ij Input the fully connected layer of the convolutional neural network and transform the new feature vector P ij After dimensionality reduction, the output feature vector is The section image I is calculated using the attention module i The attention weights are combined with all T j The value of is weighted summed to get image I i The weighting coefficient S i ; After the convolutional neural network is applied to S i Perform regression prediction to predict the values ​​of three coordinate points 2. The method for generating standard sections based on fetal cranial structural malformation according to claim 1, characterized in that: Sampling the calibrated three-dimensional volume data to generate a two-dimensional series of slice images specifically includes: Using a Fibonacci sphere, the sampling plane normal vectors are evenly distributed on the surface of the sphere, a number of planes are generated at equal intervals in each normal vector direction, and the three-dimensional volume data are sampled on the several planes to obtain ultrasonic two-dimensional section data; The fetal MRI images of the standard atlas are mapped to the same coordinate system according to the same affine transformation, and used as pixel-level mask labels of the fetal ultrasound brain to extract the region of interest ROI of the ultrasound two-dimensional section data to obtain a series of the two-dimensional section images.

3. The method for generating standard sections of fetal cranial and brain structural malformations according to claim 2, characterized in that: The sampling plane normal vector is uniformly distributed on the surface of the sphere as follows: Fibonacci grid sampling achieves uniform distribution of points on the sphere. P(φ, θ) represents the normal vector of the sampling plane, φ represents the azimuth, and θ represents the elevation. If the number of normal vectors of the sampling plane is n, then and It can be calculated by formula (1) and formula (2) respectively: Among them, φ i ,θ i Represents the azimuth and elevation of the current i-th normal vector, i∈(1,n).

4. The method for generating standard sections of fetal cranial structural malformation according to claim 1, characterized in that: The convolutional neural network mainly consists of three parts, including the feature extraction module, the attention module and the prediction module; wherein the feature extraction module is a feature extractor based on the VGG16 network, which is used to extract the deep features of the two-dimensional section image.

5. The method for generating standard sections of fetal cranial and brain structural malformations according to claim 4, characterized in that: The attention matrix specifically includes: M two-dimensional slice images input h and w are the height and width of the two-dimensional slice image respectively; After passing through the feature extractor, a feature vector is generated Send Vi into the attention mechanism model to generate the attention matrix.

6. The method for generating standard sections of fetal cranial structural malformation according to claim 5, characterized in that: The attention mechanism model is specifically: calculating the correlation between the target value (target) and the input value (source), and the correlation calculation specifically includes: calculating the vector dot product of the two, calculating the vector Cosine similarity of the two, or evaluating by introducing an additional multi-layer perceptron; wherein the multi-layer perceptron is composed of two parts, alpha and beta, and the specific calculation method is formula (3): Similarity(target i ,source j )=alpha(target i ,source j )·beta(target i ,source j ) T (3) Among them, the target and source tables represent the extracted feature vectors Vi and Vj, which are used to calculate the similarity between the two input two-dimensional section images.

7. The method for generating standard sections of fetal cranial structural malformation according to any one of claims 4 to 6, characterized in that: The doctor uses an ultrasound system to scan and select a 2D standard section, wherein the selected standard section includes one of a thalamus horizontal cross section, a lateral ventricle horizontal cross section, and a cerebellum horizontal cross section; The other standard section images include one or more of a horizontal cross section of the thalamus, a horizontal cross section of the lateral ventricle, a horizontal cross section of the cerebellum, a mid-sagittal section, a coronal section of the body of the lateral ventricle, a coronal section of the cerebellum, and a coronal section of the trigone of the lateral ventricle.

8. A device for generating standard sections of fetal cranial structural deformities, comprising: A first input module is used to access the ultrasonic three-dimensional volume data collected by the three-dimensional ultrasonic system; A first data processing module, which calibrates the three-dimensional volume data by using affine transformation, samples the calibrated three-dimensional volume data, and generates a series of two-dimensional section images; A first output module, used for outputting the calibrated three-dimensional volume data; A neural network preprocessing module, used for setting regression labels of all the two-dimensional section images, and randomly and synchronously rotating the two-dimensional section images and the regression labels of the two-dimensional section images; A neural network module, which adds an attention matrix and is used to train the two-dimensional slice image and the corresponding regression label to obtain a functional relationship between the regression label and the two-dimensional slice image; The second input module is used to access the associated continuous frame 2D section image of a selected standard section scanned by the doctor; The second data processing module is used to input the 2D section image into the neural network to predict its corresponding position coordinates in the three-dimensional space; according to the positional relationship between the 2D section image and other standard planes, calculate the position coordinates of other standard planes, and sample the corresponding coordinate positions in the three-dimensional volume data to generate other standard section images; The 2D slice image is input into the neural network to predict its corresponding position coordinates in the three-dimensional space, specifically including: Collect two consecutive frames of cross-sectional images with correlation at standard plane positions; The feature vectors of two consecutive cross-sectional images after passing through the feature extraction module of the neural network are spliced ​​in pairs to form a new feature vector The new feature vector P ij Input the fully connected layer of the neural network and transform the new feature vector P ij After dimensionality reduction, the output feature vector is The section image I is calculated using the attention module of the neural network i The attention weights are combined with all T j The value of is weighted summed to get image I i The weighting coefficient S i ; After the neural network is applied to S i Perform regression prediction to predict the values ​​of three coordinate points The second output module is used to output the position coordinates of the 2D section image and the position coordinates of other standard planes, and output the 2D section image and other standard section images at the same time.

9. An ultrasonic imaging display system, comprising a probe, a display, and the device for generating standard sections of fetal cranial and brain structural deformities according to claim 8, wherein the first input module and the second input module of the device for generating standard sections of fetal cranial and brain structural deformities are electrically connected to the probe, and the display is electrically connected to the first output module and the second output module of the device for generating standard sections of fetal cranial and brain structural deformities, characterized in that: include: The probe comprises a first acquisition mode and a second acquisition mode, wherein the first acquisition mode is used to acquire three-dimensional volume data of the subject, and the second acquisition mode is used to acquire two-dimensional cross-sectional images of the subject in real time; The device for generating standard sections of fetal cranial brain structural deformity generates a standard three-dimensional model of the fetal cranial brain of the subject according to the three-dimensional volume data, and position coordinates of each standard section; and obtains position coordinates of the corresponding three-dimensional space and section images of each standard section according to the two-dimensional section image; The display comprises a first display area, a second display area and a third display area, wherein the first display area is used to display the standard three-dimensional body model, and a display mark is set on the standard three-dimensional body model to distinguish and mark the position of each standard section; the second display area is used to display the two-dimensional section image in real time; The third display area is used to display the generated section images of each standard section.

10. The ultrasonic imaging display system according to claim 9, characterized in that: The first display area is provided with a human-computer interaction interface or an interface control; the display mark includes reference lines or reference planes of different colors; the display mark and the standard three-dimensional model are linked, and the human-computer interaction device or the interface control is used to translate or rotate the standard three-dimensional model and the display mark.

11. The ultrasonic imaging display system according to claim 10, characterized in that: According to the position coordinates of the two-dimensional section image in the three-dimensional space, a display mark corresponding to the two-dimensional section is set at a corresponding coordinate position in the three-dimensional volume model in the first display area.

12. The ultrasonic imaging display system according to claim 10 or 11, wherein the second display area further comprises a setting module for setting a target section, characterized in that: The movement strategy of the probe is determined according to the two-dimensional section acquired by the probe in the second acquisition mode and the coordinate position of the target section in the three-dimensional model; the movement strategy includes the direction and angle of movement of the probe.

13. An electronic device, characterized in that: It includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include steps for executing the method for generating standard sections of fetal cranial structural malformations as described in any one of claims 1-7.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of generating a standard section of a fetal cranial structural deformity according to any one of claims 1 to 7.

Images