Fetal cranial ultrasound structure segmentation method and device based on three-dimensional convolutional neural network

Abstract

The present invention discloses a method and device for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network. The method comprises: obtaining original volume data of fetal cranial ultrasound structure; inputting the original volume data of fetal cranial ultrasound structure into a determined three-dimensional convolutional neural network segmentation model for analysis, and obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of fetal cranial ultrasound structure. It can be seen that the implementation of the present invention can realize automatic segmentation of fetal cranial ultrasound structure using the three-dimensional convolutional neural network segmentation model, which is conducive to improving the accuracy of fetal cranial ultrasound structure segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network. Background Art

[0002] In real life, when medical images are used to display the ultrasound structure of the fetal brain, factors such as individual differences, individual movement, high similarity between different soft tissues, and changes in physiological states will lead to large differences between the shape appearance displayed in the medical image and the actual shape appearance. Therefore, the clinical application of medical image analysis of fetal brain ultrasound structure faces great challenges. In order to reduce the difference between the shape appearance displayed in the medical image and the actual shape appearance and obtain more accurate medical images, it is necessary to segment the fetal brain ultrasound structure in the medical image.

[0003] At present, the segmentation of fetal cranial ultrasound structure mainly adopts manual segmentation, that is, by manually outlining the edge of the segmentation object in each fetal cranial ultrasound structure scan slice with the help of software, the segmentation result of each fetal cranial ultrasound structure scan slice is obtained, and then the segmentation results of all fetal cranial ultrasound structure scan slices are synthesized into the segmentation result of the entire fetal cranial ultrasound structure. However, it is found in practice that the accuracy of manual segmentation is often relatively low due to the influence of subjective factors such as the operator's work experience, professional level and working status. It can be seen that how to improve the accuracy of fetal cranial ultrasound structure segmentation is particularly important. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network, which can realize automatic recognition and effective segmentation of fetal cranial ultrasound structure, and is conducive to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a fetal cranial ultrasound structure segmentation method based on a three-dimensional convolutional neural network, the method comprising:

[0006] Obtain the original volume data of fetal cranial ultrasound structure;

[0007] The original volume data of the fetal cranial ultrasound structure is input into the determined three-dimensional convolutional neural network segmentation model for analysis, and the analysis result output by the three-dimensional convolutional neural network segmentation model is obtained as the segmentation result of the fetal cranial ultrasound structure.

[0008] As an optional embodiment, in the first aspect of the present invention, before inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, the method further includes:

[0009] Detecting whether there is sensitive information in the original volume data of the fetal cranial ultrasound structure, wherein the sensitive information includes the privacy information of the individual corresponding to the fetal cranial ultrasound structure, and the privacy information of the individual includes one or more of the individual's name, gender, age, date of birth, and contact information;

[0010] When it is detected that the sensitive information exists in the original volume data of the fetal cranial brain ultrasound structure, the sensitive information is removed, and the original volume data after the sensitive information is removed is updated to the original volume data of the fetal cranial brain ultrasound structure, and the operation of inputting the original volume data of the fetal cranial brain ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered;

[0011] And, the method further comprises:

[0012] When it is detected that the sensitive information does not exist in the original volume data of the fetal cranial ultrasound structure, the operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered.

[0013] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0014] Acquire a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, wherein the first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmentation of the sample original volume data;

[0015] Building an original three-dimensional convolutional neural network segmentation model, training the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determining the trained original three-dimensional convolutional neural network segmentation model as a first training segmentation model;

[0016] Adjusting the first training segmentation model based on the second sample data set, and determining the adjusted first training segmentation model as the second training segmentation model;

[0017] The loss value of the second training segmentation model is calculated based on the determined loss function, and based on the loss value, the parameters of the second training segmentation model are optimized, and the second training segmentation model after the optimized parameters is determined as a three-dimensional convolutional neural network segmentation model.

[0018] As an optional implementation, in the first aspect of the present invention, adjusting the first training segmentation model based on the second sample data set includes:

[0019] Determining a target number of sample data pairs in the second sample data set required for a single adjustment of the first training segmentation model according to the determined number of sample data pairs in the second sample data set and the number of voxels of original volume data of each sample in the second sample data set;

[0020] Inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, and obtaining an analysis result output by the first training segmentation model, adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, and updating the adjusted segmentation model as the first training segmentation model;

[0021] Re-performing the step of inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, obtaining the analysis result output by the first training segmentation model, and adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, until the total number of adjustments of the first training segmentation model reaches the determined adjustment number threshold;

[0022] The step of determining the adjusted first training segmentation model as the second training segmentation model includes:

[0023] The segmentation model obtained after the last adjustment is determined as the second training segmentation model.

[0024] As an optional implementation, in the first aspect of the present invention, the calculating the loss value of the second training segmentation model based on the determined loss function includes:

[0025] Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis;

[0026] Acquire the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample;

[0027] Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set;

[0028] Based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, the loss value of the second training segmentation model is calculated, wherein the loss function is any one of a cross entropy cost function, a hinge loss function, and a mean square error loss function, and the loss value is used to characterize the difference between the predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data.

[0029] As an optional implementation, in the first aspect of the present invention, the determining of the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set includes:

[0030] Determine that the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is an equal weight; or,

[0031] The weight corresponding to each voxel is determined according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel.

[0032] As an optional implementation, in the first aspect of the present invention, optimizing the parameters of the second training segmentation model based on the loss value includes:

[0033] Determining a learning rate of a stochastic gradient descent method, performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, and reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method;

[0034] Re-execute the steps of performing a first preset number of optimization operations on the parameters of the second training segmentation model based on the learning rate of the stochastic gradient descent method using the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method, until the number of executions of the optimization operation reaches the determined optimization number threshold.

[0035] As an optional implementation, in the first aspect of the present invention, before optimizing the parameters of the second training segmentation model based on the loss value, the method further includes:

[0036] Determine whether the loss value is less than or equal to a preset loss threshold;

[0037] When the judgment result is yes, triggering the execution of the operation of optimizing the parameters of the second training segmentation model based on the loss value;

[0038] When the judgment result is no, the operation of adjusting the first training segmentation model based on the second sample data set is re-executed.

[0039] As an optional embodiment, in the first aspect of the present invention, after collecting the first sample data set corresponding to the non-fetal cranial ultrasound structure and the second sample data set corresponding to the sample fetal cranial ultrasound structure, and before building the original three-dimensional convolutional neural network segmentation model, the method further includes:

[0040] Detecting whether there are sample data pairs that do not meet the preset quality condition in the third sample data set, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality condition include at least one of a sample data pair with missing data, a sample data pair with duplicate data, and a sample data pair with incorrect data;

[0041] When the detection result is yes, the sample data pairs in the third sample data set that do not meet the preset quality condition are deleted, or the sample data pairs in the third sample data set that do not meet the preset quality condition are corrected to obtain sample data pairs that meet the preset quality condition, and the operation of building the original three-dimensional convolutional neural network segmentation model is triggered;

[0042] And, the method further comprises:

[0043] When the detection result is negative, the operation of building the original three-dimensional convolutional neural network segmentation model is triggered.

[0044] As an optional implementation, in the first aspect of the present invention, before adjusting the first training segmentation model based on the second sample data set, the method further includes:

[0045] Determine the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set;

[0046] Determine a data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, wherein the data enhancement method is used to increase the number of sample data pairs in the second sample data set;

[0047] Perform a data enhancement operation on the data in the second sample data set according to the determined data enhancement method to obtain a sample data set after data enhancement, update the sample data set after data enhancement to the second sample data set, and trigger the execution of the operation of adjusting the first training segmentation model based on the second sample data set.

[0048] As an optional implementation, in the first aspect of the present invention, after obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure, the method further includes:

[0049] Rendering the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image;

[0050] The rendered image is used as a segmented image of the fetal cranial ultrasound structure, and the segmented image of the fetal cranial ultrasound structure is transmitted to a corresponding image output device.

[0051] The second aspect of the present invention discloses a fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network, the device comprising:

[0052] A data acquisition module, used to obtain the original volume data of the fetal cranial ultrasound structure;

[0053] The segmentation module is used to input the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, and obtain the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure.

[0054] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0055] A first detection module is used to detect whether there is sensitive information in the original volume data of the fetal cranial ultrasound structure after the data acquisition module acquires the original volume data of the fetal cranial ultrasound structure, and when the detection result is no, trigger the segmentation module to perform the operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, wherein the sensitive information includes the privacy information of the individual corresponding to the fetal cranial ultrasound structure, and the privacy information of the individual includes one or more of the name, gender, age, date of birth, and contact information of the individual;

[0056] an information removal module, configured to remove the sensitive information when the first detection module detects that the original volume data of the fetal cranial ultrasound structure contains the sensitive information;

[0057] An updating module is used to update the original volume data after removing the sensitive information to the original volume data of the fetal cranial ultrasound structure, and trigger the segmentation module to execute the operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis.

[0058] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0059] The data acquisition module is further used to acquire a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, wherein the first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmentation of the sample original volume data;

[0060] A training module, used to build an original three-dimensional convolutional neural network segmentation model, train the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determine the trained original three-dimensional convolutional neural network segmentation model as a first training segmentation model;

[0061] an adjustment module, configured to adjust the first training segmentation model based on the second sample data set, and determine the adjusted first training segmentation model as the second training segmentation model;

[0062] An optimization module is used to calculate the loss value of the second training segmentation model based on the determined loss function, and based on the loss value, optimize the parameters of the second training segmentation model, and determine the second training segmentation model after the optimized parameters as a three-dimensional convolutional neural network segmentation model.

[0063] As an optional implementation, in the second aspect of the present invention, the specific manner in which the adjustment module adjusts the first training segmentation model based on the second sample data set is:

[0064] Determining a target number of sample data pairs in the second sample data set required for a single adjustment of the first training segmentation model according to the determined number of sample data pairs in the second sample data set and the number of voxels of original volume data of each sample in the second sample data set;

[0065] Inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, and obtaining an analysis result output by the first training segmentation model, adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, and updating the adjusted segmentation model as the first training segmentation model;

[0066] Re-performing the step of inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, obtaining the analysis result output by the first training segmentation model, and adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, until the total number of adjustments of the first training segmentation model reaches the determined adjustment number threshold;

[0067] The step of determining the adjusted first training segmentation model as the second training segmentation model includes:

[0068] The segmentation model obtained after the last adjustment is determined as the second training segmentation model.

[0069] As an optional implementation, in the second aspect of the present invention, the specific manner in which the optimization module calculates the loss value of the second training segmentation model based on the determined loss function is:

[0070] Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis;

[0071] Acquire the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample;

[0072] Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set;

[0073] Based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, the loss value of the second training segmentation model is calculated, wherein the loss function is any one of a cross entropy cost function, a hinge loss function, and a mean square error loss function, and the loss value is used to characterize the difference between the predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data.

[0074] As an optional implementation, in the second aspect of the present invention, the specific manner in which the optimization module determines the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is:

[0075] Determine that the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is an equal weight; or,

[0076] The weight corresponding to each voxel is determined according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel.

[0077] As an optional implementation, in the second aspect of the present invention, the optimization module optimizes the parameters of the second training segmentation model based on the loss value in a specific manner as follows:

[0078] Determining a learning rate of a stochastic gradient descent method, performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, and reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method;

[0079] Re-execute the steps of performing a first preset number of optimization operations on the parameters of the second training segmentation model based on the learning rate of the stochastic gradient descent method using the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method, until the number of executions of the optimization operation reaches the determined optimization number threshold.

[0080] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0081] A judgment module is used to input the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data of the second training segmentation model and the determined loss function in the optimization module based on the determined second sample data set, calculate the loss value of the second training segmentation model, and then judge whether the loss value is less than or equal to a preset loss threshold; when the judgment result is yes, trigger the optimization module to execute the operation of optimizing the parameters of the second training segmentation model based on the loss value; when the judgment result is no, trigger the adjustment module to re-execute the operation of adjusting the first training segmentation model based on the second sample data set.

[0082] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0083] A second detection module is used for detecting whether there are sample data pairs that do not meet the preset quality conditions in the third sample data set after the data acquisition module acquires the first sample data set corresponding to the non-fetal cranial ultrasound structure and the second sample data set corresponding to the sample fetal cranial ultrasound structure, and when the detection result is no, triggering the training module to perform the operation of building the original three-dimensional convolutional neural network segmentation model, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality conditions include at least one of sample data pairs with missing data, sample data pairs with repeated data, and sample data pairs with incorrect data;

[0084] A data cleaning module is used to delete the sample data pairs that do not meet the preset quality conditions in the third sample data set when the second detection module detects that there are sample data pairs that do not meet the preset quality conditions in the third sample data set, or to correct the sample data pairs that do not meet the preset quality conditions in the third sample data set, so as to obtain sample data pairs that meet the preset quality conditions, and trigger the training module to perform the operation of building the original three-dimensional convolutional neural network segmentation model.

[0085] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0086] a determination module, configured to determine the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, and determine a data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, wherein the data enhancement method is used to increase the number of sample data pairs in the second sample data set;

[0087] A data enhancement module is used to perform a data enhancement operation on the data in the second sample data set according to the determined data enhancement method after the determination module determines the data enhancement method for the second sample data set, to obtain a sample data set after data enhancement, and to update the sample data set after data enhancement to the second sample data set, and to trigger the adjustment module to perform the operation of adjusting the first training segmentation model based on the second sample data set.

[0088] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0089] A rendering module is used to render the segmentation result of the fetal brain ultrasound structure after the segmentation module obtains the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal brain ultrasound structure to obtain a rendered image.

[0090] A transmission module is used to use the rendered image as a segmented image of the fetal cranial ultrasound structure, and transmit the segmented image of the fetal cranial ultrasound structure to a corresponding image output device.

[0091] The third aspect of the present invention discloses another fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network, the device comprising:

[0092] A memory storing executable program code;

[0093] a processor coupled to the memory;

[0094] The processor calls the executable program code stored in the memory to execute the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network disclosed in the first aspect of the present invention.

[0095] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network disclosed in the first aspect of the present invention.

[0096] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0097] In an embodiment of the present invention, the original volume data of the fetal cranial ultrasound structure is obtained, the original volume data of the fetal cranial ultrasound structure is input into a determined three-dimensional convolutional neural network segmentation model for analysis, and the analysis result output by the three-dimensional convolutional neural network segmentation model is obtained as the segmentation result of the fetal cranial ultrasound structure. It can be seen that the implementation of the present invention can realize the automatic recognition and effective segmentation of the fetal cranial ultrasound structure using the three-dimensional convolutional neural network segmentation model, which is beneficial to improve the accuracy of the fetal cranial ultrasound structure segmentation, and is beneficial to help the operator quickly locate the location of the fetal cranial ultrasound structure, and improve the inspection efficiency and accuracy of the operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0099] Figure 1 It is a flow chart of a method for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention;

[0100] Figure 2 It is a flow chart of another method for fetal cranial ultrasound structure segmentation based on three-dimensional convolutional neural network disclosed in an embodiment of the present invention;

[0101] Figure 3 It is a structural schematic diagram of a fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention;

[0102] Figure 4 It is a structural schematic diagram of another fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention;

[0103] Figure 5 It is a structural schematic diagram of another fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0104] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0105] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0106] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0107] The present invention discloses a fetal cranial ultrasound structure segmentation method and device based on a three-dimensional convolutional neural network, which can realize automatic recognition and effective segmentation of fetal cranial ultrasound structure by using a three-dimensional convolutional neural network segmentation model, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation, and is beneficial to helping operators quickly locate the location of fetal cranial ultrasound structure, thereby improving the inspection efficiency and accuracy of operators. The following are detailed descriptions.

[0108] Embodiment 1

[0109] See also Figure 1 , Figure 1 1 is a flow chart of a method for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention. Figure 1 The described fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network can be applied to a fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network to achieve the segmentation of fetal cranial ultrasound structure, such as the segmentation of fetal cranial ultrasound structure in an image during prenatal ultrasound examination. Figure 1 As shown, the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network may include the following operations:

[0110] 101. Obtain the original volume data of the fetal brain ultrasound structure.

[0111] The original volume data may be obtained by electronic computed tomography (CT) scanning or positron emission tomography (PET) scanning, or by magnetic resonance imaging (MRI) or ultrasonic imaging technology, which is not limited in the embodiments of the present invention. The original volume data of the fetal cranial ultrasound structure may be the original volume data of the complete fetal cranial ultrasound structure, or the original volume data of a part of the fetal cranial ultrasound structure, which is not limited in the embodiments of the present invention.

[0112] 102. The original volume data of the fetal cranial ultrasound structure is input into the determined three-dimensional convolutional neural network segmentation model for analysis.

[0113] In an embodiment of the present invention, the three-dimensional convolutional neural network segmentation model may include any one of a 3D-FCN model, a 3D-UNet model, a VNet model, and a DenseVoxNet model. Preferably, the three-dimensional convolutional neural network segmentation model is a 3D-UNet model.

[0114] The three-dimensional convolutional neural network segmentation model can be used to determine the grayscale value of each voxel in the original volume data, and based on the determined grayscale value of each voxel in the original volume data, downsample the original volume data to obtain a downsampled result, perform feature recognition on the downsampled result to obtain a feature recognition result, and upsample the downsampled result to obtain an upsampled result, and determine the voxel category to which each voxel in the original volume data belongs and the segmentation boundary of the central nervous system according to the feature recognition result and the upsampled result as an analysis result, wherein the resolution of the upsampled result is greater than the resolution of the downsampled result.

[0115] 103. Obtain the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure.

[0116] In the embodiment of the present invention, optionally, the segmentation result of the fetal brain ultrasound structure includes a normal structure and / or a deformed structure.

[0117] Furthermore, the method may also include: outputting prompt information of the segmentation result to the segmentation operator corresponding to the fetal cranial ultrasound structure.

[0118] It can be seen that the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network described in the embodiment of the present invention can use the three-dimensional convolutional neural network segmentation model to realize automatic recognition and effective segmentation of fetal cranial ultrasound structure, which is beneficial to improve the accuracy of fetal cranial ultrasound structure segmentation, and is beneficial to help the operator quickly locate the three-dimensional structure of the fetal cranial brain, and give the operator a prompt whether the structure is normal or abnormal, thereby improving the operator's inspection efficiency and accuracy, and reducing the misdiagnosis and missed diagnosis rate. In addition, the present invention can also be applied to experimental teaching, thereby improving the teaching effect.

[0119] In an optional embodiment, before inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, the method may further include:

[0120] Detecting whether there is sensitive information in the original volume data of the fetal cranial ultrasound structure, wherein the sensitive information includes the privacy information of the individual corresponding to the fetal cranial ultrasound structure, and the privacy information of the individual may include one or more of the individual's name, gender, age, date of birth, and contact information;

[0121] When sensitive information is detected in the original volume data of the fetal cranial ultrasound structure, the sensitive information is removed, and the original volume data after the sensitive information is removed is updated to the original volume data of the fetal cranial ultrasound structure, and an operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered;

[0122] And, the method may further include:

[0123] When it is detected that no sensitive information exists in the original volume data of the fetal cranial ultrasound structure, an operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered.

[0124] In this optional embodiment, the individual corresponding to the fetal cranial ultrasound structure may be the fetus corresponding to the fetal cranial ultrasound structure or a relative of the fetus, such as a mother and a father.

[0125] In this optional embodiment, after detecting that sensitive information exists in the original volume data of the fetal cranial ultrasound structure, and before removing the sensitive information, the method may further include: determining the location of the sensitive information in the original volume data, and triggering the operation of removing the sensitive information. It can be seen that this can quickly and accurately remove the sensitive information in the original volume data.

[0126] It can be seen that the implementation of this optional embodiment can remove sensitive information when the original volume data contains sensitive information, protect the personal privacy of the individual corresponding to the fetal cranial ultrasound structure, and reduce the occurrence of the analysis accuracy and efficiency of the original volume data being reduced due to the presence of sensitive information in the original volume data, thereby improving the accuracy and efficiency of the fetal cranial ultrasound structure segmentation.

[0127] In another optional embodiment, after obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure, the method may further include:

[0128] Rendering the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image;

[0129] The rendered image is used as a segmented image of the fetal cranial ultrasound structure, and the segmented image of the fetal cranial ultrasound structure is transmitted to a corresponding image output device (such as a computer display).

[0130] In this optional embodiment, optionally, rendering the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image may include:

[0131] Determine the range that needs to be rendered in the segmentation result of the fetal cranial ultrasound structure, the type of virtual light source required for rendering, and the parameters of the virtual light source; based on the type of virtual light source, the parameters of the virtual light source, and the voxel category to which each voxel in the range that needs to be rendered in the segmentation result belongs, color the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image. It can be seen that this can improve the accuracy and reliability of rendering, obtain a precise rendered image, and facilitate observation by the operator.

[0132] It can be seen that implementing this optional embodiment can render and display the segmentation results of the fetal cranial ultrasound structure, process the model data of the fetal cranial ultrasound structure into a real picture, and make the segmentation results of the fetal cranial ultrasound structure visually closer to the actual original.

[0133] Embodiment 2

[0134] See also Figure 2 , Figure 2 1 is a flow chart of another method for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention. Figure 2 The described fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network can be applied to a fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network to achieve the segmentation of fetal cranial ultrasound structure, such as the segmentation of fetal cranial ultrasound structure in an image during prenatal ultrasound examination. Figure 2 As shown, the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network may include the following operations:

[0135] 201. Collect a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure.

[0136] The first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmenting the sample original volume data.

[0137] In an embodiment of the present invention, the first sample data set corresponding to the non-fetal cranial ultrasound structure may include a sample data set corresponding to a natural scene image, a sample data set corresponding to an animal body structure, and a sample data set corresponding to other human body structures except the fetal cranial ultrasound structure, which is not limited in the embodiment of the present invention. Preferably, the first sample data set is a sample data set corresponding to other cranial ultrasound structures except the fetal cranial ultrasound structure.

[0138] In the embodiment of the present invention, optionally, based on the object of the three-dimensional convolutional neural network segmentation model, a sample data set corresponding to the sample fetal cranial ultrasound structure of the category population corresponding to the object is selected as the second sample data set. For example, if the three-dimensional convolutional neural network segmentation model needs to be used to segment the original volume data of the fetal cranial ultrasound structure of a female, the second sample data set can be a sample data set corresponding to the sample fetal cranial ultrasound structure of the female category. It can be seen that this can improve the quality of the data set, which is conducive to improving the accuracy of model training and adjustment, and then is conducive to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0139] 202. Build an original three-dimensional convolutional neural network segmentation model, train the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determine the trained original three-dimensional convolutional neural network segmentation model as the first training segmentation model.

[0140] As an optional implementation, training the original three-dimensional convolutional neural network segmentation model based on the first sample data set may include:

[0141] The sample data pairs in the first sample data set are input into the original three-dimensional convolutional neural network segmentation model for analysis to obtain the analysis results output by the original three-dimensional convolutional neural network segmentation model; based on the analysis results output by the original three-dimensional convolutional neural network segmentation model, the original three-dimensional convolutional neural network segmentation model is trained.

[0142] It can be seen that when the data amount of the second sample data set corresponding to the sample fetal cranial ultrasound structure is limited, the original three-dimensional convolutional neural network segmentation model can be first trained based on the first sample data set corresponding to the non-fetal cranial ultrasound structure, so as to meet the model training requirements for sample data volume and improve the accuracy and reliability of model training, which is conducive to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0143] 203. Adjust the first training segmentation model based on the second sample data set, and determine the adjusted first training segmentation model as the second training segmentation model.

[0144] In the embodiment of the present invention, as an optional implementation, adjusting the first training segmentation model based on the second sample data set may include:

[0145] Determining a target number of sample data pairs in the second sample data set required for a single adjustment of the first training segmentation model according to the determined number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set;

[0146] Inputting a target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, and obtaining an analysis result output by the first training segmentation model, adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, and updating the adjusted segmentation model to the first training segmentation model;

[0147] Re-execute the steps of inputting a target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, obtaining the analysis result output by the first training segmentation model, and adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, until the total number of adjustments of the first training segmentation model reaches the determined adjustment number threshold;

[0148] And, determining the adjusted first training segmentation model as the second training segmentation model, comprising:

[0149] The segmentation model obtained after the last adjustment is determined as the second training segmentation model.

[0150] For example, if the number of sample data pairs in the second sample data set is 1000, and the determined target number is 40, and the adjustment number threshold is 20, then 40 sample data pairs in the second sample data set are input into the first training segmentation model for adjustment each time, and the adjusted first training segmentation model is used as the object of the next adjustment until the total number of adjustments reaches 20 times. That is, when the total number of sample data pairs in the second sample data set input into the first training segmentation model reaches 800, the adjustment process ends.

[0151] It can be seen that in this way, the model adjustment method can be determined according to the characteristics of the second sample data set corresponding to the sample fetal cranial ultrasound structure, the second sample data set can be fully and effectively utilized, and the accuracy and reliability of the adjustment of the first training segmentation model can be improved, which is conducive to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0152] 204. Calculate the loss value of the second training segmentation model based on the determined loss function, optimize the parameters of the second training segmentation model based on the loss value, and determine the second training segmentation model after the optimized parameters as a three-dimensional convolutional neural network segmentation model.

[0153] In the embodiment of the present invention, as an optional implementation, calculating the loss value of the second training segmentation model based on the determined loss function may include:

[0154] Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis;

[0155] Obtaining the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample;

[0156] Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set;

[0157] Based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, the loss value of the second training segmentation model is calculated, wherein the loss function is any one of a cross entropy cost function, a hinge loss function, and a mean square error loss function, and the loss value is used to characterize the difference between the predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data.

[0158] In this optional implementation, further optionally, determining the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set may include:

[0159] Determine that the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is an equal weight; or,

[0160] The weight corresponding to each voxel is determined according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel.

[0161] For example, the fetal cranial ultrasound structure includes the cerebrum, cerebellum, brainstem, and diencephalon. If the cerebellum is to be highlighted when segmenting the fetal cranial ultrasound structure, the weight corresponding to the voxels belonging to the cerebellum in the sample labeled volume data may be greater than the weights corresponding to the voxels belonging to other parts; if the number of voxels belonging to the background part in the sample labeled volume data is greater than the number of voxels not belonging to the background part, the weight corresponding to the voxels belonging to the background part in the sample labeled volume data may be less than the weight corresponding to the voxels not belonging to the background part; if the number of voxels in the sample labeled volume data is less than the target number of voxels, that is, the size of the sample labeled volume data is smaller than the target size, then the weight corresponding to each voxel of the sample labeled volume data may be greater than the target weight.

[0162] It can be seen that in this way, reasonable weights can be determined for the voxels of the sample labeled volume data according to the data characteristics of the second sample data set and / or the requirements for the three-dimensional convolutional neural network segmentation model, especially when the number of voxels in different sample labeled volume data is unbalanced or the voxel categories are unbalanced. This can improve the accuracy of calculating the loss value, reduce the occurrence of overfitting in the model optimization process, and improve the optimization effect of the model, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0163] It can be seen that implementing this optional implementation method can improve the accuracy of calculating the loss value, so that the segmentation effect of the second training segmentation model can be effectively evaluated based on the loss value, and appropriate evaluation criteria are provided for the model optimization effect, thereby improving the efficiency of model optimization.

[0164] In the embodiment of the present invention, as another optional implementation, optimizing the parameters of the second training segmentation model based on the loss value may include:

[0165] Determining a learning rate of the stochastic gradient descent method, performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method;

[0166] Re-execute the steps of performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method until the number of executions of the optimization operation reaches the determined optimization number threshold.

[0167] For example, if the determined learning rate of the stochastic gradient descent method is 0.001, the first preset number is 20, and the optimization number threshold is 100 times, then the stochastic gradient descent method is used to perform multiple optimization operations on the parameters of the second training segmentation model based on the learning rate, and the learning rate is reduced once every 20 optimization operations, and the reduced learning rate is updated to the learning rate of the stochastic gradient descent method until the number of optimization operations performed reaches 100 times.

[0168] It can be seen that the implementation of this optional implementation method can use the stochastic gradient descent method to optimize the parameters of the second training segmentation model. Since the stochastic gradient descent method only needs to calculate the first-order derivative of the loss function, the optimization speed is faster, and multiple optimization operations are performed on the parameters of the second training segmentation model, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0169] 205. Obtain the original volume data of the fetal brain ultrasound structure.

[0170] 206. Input the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis.

[0171] 207. The analysis result output by the three-dimensional convolutional neural network segmentation model is obtained as the segmentation result of the fetal cranial ultrasound structure.

[0172] In the embodiment of the present invention, for other descriptions of step 205 to step 207, please refer to the detailed description of step 101 to step 103 in the first embodiment, and the embodiment of the present invention will not be repeated.

[0173] It should be noted that steps 202 to 204 can occur in any deep learning framework of PaddlePaddle, Tensorflow, Caffe, Theano, MXNet, Torch, and Pytorch. Preferably, steps 202 and 204 both occur in the deep learning framework Pytorch. There is no order relationship between any of steps 201 to 204 and step 205. They can occur before step 205, after step 205, or at the same time as step 205.

[0174] It can be seen that the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network described in the embodiment of the present invention obtains an accurate three-dimensional convolutional neural network segmentation model by building, training, adjusting and optimizing the three-dimensional convolutional neural network segmentation model, and can use the accurate three-dimensional convolutional neural network segmentation model to realize automatic recognition and effective segmentation of fetal cranial ultrasound structure, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation, helping the operator to quickly locate the position of the fetal cranial ultrasound structure, and improving the operator's inspection efficiency and accuracy.

[0175] In an optional embodiment, before optimizing the parameters of the second training segmentation model based on the loss value, the method may further include:

[0176] Determine whether the loss value is less than or equal to the preset loss threshold;

[0177] When the judgment result is yes, triggering the execution of an operation of optimizing the parameters of the second training segmentation model based on the loss value;

[0178] When the judgment result is no, the operation of adjusting the first training segmentation model based on the second sample data set is re-executed.

[0179] It can be seen that the implementation of this optional embodiment can optimize the parameters of the model only when the loss value of the second training segmentation model meets the preset conditions, which can improve the effects of model adjustment and model optimization, thereby helping to improve the accuracy of fetal cranial ultrasound structure segmentation.

[0180] In another optional embodiment, after acquiring a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, and before building an original three-dimensional convolutional neural network segmentation model, the method may further include:

[0181] Detecting whether there are sample data pairs that do not meet the preset quality condition in the third sample data set, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality condition include at least one of a sample data pair with missing data, a sample data pair with duplicate data, and a sample data pair with incorrect data;

[0182] When the detection result is yes, the sample data pairs in the third sample data set that do not meet the preset quality conditions are deleted, or the sample data pairs in the third sample data set that do not meet the preset quality conditions are corrected to obtain sample data pairs that meet the preset quality conditions, and trigger the execution of the operation of building the original three-dimensional convolutional neural network segmentation model;

[0183] And, the method may further include:

[0184] When the detection result is negative, the operation of building the original 3D convolutional neural network segmentation model is triggered.

[0185] Optionally, correcting the sample data pairs that do not meet the preset quality conditions in the third sample data set may include: filling the missing data in the sample data pairs with missing data, and / or replacing the erroneous data in the sample data pairs with erroneous data, and / or performing any one of flipping, rotating, and cropping operations on the sample data pairs with repeated data, and replacing the sample data pairs with repeated data with the sample data pairs after performing any one of flipping, rotating, and cropping operations. Specifically: if a sample data pair is missing sample labeled volume data, the corresponding sample labeled volume data is supplemented according to the sample original volume data of the sample data pair; if the sample labeled volume data in a sample data pair is not the sample labeled volume corresponding to the sample original volume data of the sample data pair, the sample labeled volume data corresponding to the sample original volume data replaces the original sample labeled volume data. It can be seen that in this way, different correction operations can be taken according to the type of sample data pairs that do not meet the preset quality conditions to improve the quality of the sample data set.

[0186] It can be seen that implementing this optional embodiment can delete or correct sample data pairs in the sample data set that do not meet the preset quality conditions, thereby improving the quality of the sample data set, which is beneficial to improving the effects of model training, adjustment and optimization, and thus is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0187] In yet another optional embodiment, before adjusting the first training segmentation model based on the second sample data set, the method may further include:

[0188] Determine the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set;

[0189] Determine a data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, where the data enhancement method is used to increase the number of sample data pairs in the second sample data set;

[0190] According to the determined data enhancement method, a data enhancement operation is performed on the data in the second sample data set to obtain a data-enhanced sample data set, and the data-enhanced sample data set is updated to the second sample data set, and an operation of adjusting the first training segmentation model based on the second sample data set is triggered.

[0191] Optionally, the data enhancement method may include at least one of random flipping, random rotation, random cropping, and deformation scaling.

[0192] In this optional embodiment, optionally, before determining the data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, the method may also include: judging whether the number of sample data pairs in the second sample data set is greater than or equal to a second preset number, and when the judgment result is yes, triggering the operation of adjusting the first training segmentation model based on the second sample data set; when the judgment result is no, triggering the operation of determining the data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set. It can be seen that this can reduce unnecessary steps, reduce the consumption of running memory, and improve the efficiency of model adjustment and model optimization.

[0193] It can be seen that this optional embodiment can increase the number of sample data pairs in the second sample data set, reduce the overfitting during model adjustment and model optimization due to the limited number of sample data pairs, improve the effect of model adjustment and model optimization, and thus help to improve the accuracy of fetal cranial ultrasound structure segmentation.

[0194] In another optional embodiment, after acquiring a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, and before building an original three-dimensional convolutional neural network segmentation model, the method may further include:

[0195] Normalization processing of spatial scale and / or gray value is performed on all sample original volume data and all sample marked volume data in the first sample data set and / or the second sample data set.

[0196] It can be seen that this can ensure the uniformity of the measurement units of the data in the first sample data set and / or the second sample data set, which is conducive to improving the calculation speed and accuracy during model training, adjustment, and optimization.

[0197] Embodiment 3

[0198] See also Figure 3 , Figure 3 : is a schematic diagram of a fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention. Figure 3 The described fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network is used to segment the fetal cranial ultrasound structure, for example, to segment the fetal cranial ultrasound structure in an image during prenatal ultrasound examination. Figure 3 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may include:

[0199] The data acquisition module 301 is used to obtain the original volume data of the fetal cranial ultrasound structure;

[0200] The segmentation module 302 is used to input the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, and obtain the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure.

[0201] It can be seen that implementation Figure 3 The described device can use a three-dimensional convolutional neural network segmentation model to realize automatic recognition and effective segmentation of fetal cranial ultrasound structures, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation, helping operators to quickly locate the location of fetal cranial ultrasound structures, and improving the operator's inspection efficiency and accuracy.

[0202] In an optional embodiment, if Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0203] The first detection module 303 is used to detect whether there is sensitive information in the original volume data of the fetal cranial ultrasound structure after the data acquisition module 301 obtains the original volume data of the fetal cranial ultrasound structure. When the detection result is no, the segmentation module 302 is triggered to perform an operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, wherein the sensitive information includes the privacy information of the individual corresponding to the fetal cranial ultrasound structure, and the privacy information of the individual may include one or more of the individual's name, gender, age, date of birth, and contact information;

[0204] The information removal module 304 is used to remove the sensitive information when the first detection module 303 detects that there is sensitive information in the original volume data of the fetal cranial ultrasound structure;

[0205] The updating module 305 is used to update the original volume data after removing sensitive information to the original volume data of the fetal cranial ultrasound structure, and trigger the segmentation module 302 to execute the operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis.

[0206] It can be seen that implementation Figure 4 The described device can remove sensitive information when the original volume data contains sensitive information, protect the personal privacy of the individual corresponding to the fetal cranial ultrasound structure, and reduce the occurrence of the reduction of the analysis accuracy and efficiency of the original volume data due to the presence of sensitive information in the original volume data, thereby improving the accuracy and efficiency of the fetal cranial ultrasound structure segmentation.

[0207] In another optional embodiment, Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0208] The data acquisition module 301 is further used to acquire a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, wherein the first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmentation of the sample original volume data;

[0209] A training module 306 is used to build an original three-dimensional convolutional neural network segmentation model, train the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determine the trained original three-dimensional convolutional neural network segmentation model as the first training segmentation model;

[0210] An adjustment module 307, configured to adjust the first training segmentation model based on the second sample data set, and determine the adjusted first training segmentation model as the second training segmentation model;

[0211] The optimization module 308 is used to calculate the loss value of the second training segmentation model based on the determined loss function, and optimize the parameters of the second training segmentation model based on the loss value, and determine the second training segmentation model after the optimized parameters as a three-dimensional convolutional neural network segmentation model.

[0212] It can be seen that implementation Figure 4 The described device can build, train, adjust and optimize a three-dimensional convolutional neural network segmentation model to obtain an accurate three-dimensional convolutional neural network segmentation model, and can use the accurate three-dimensional convolutional neural network segmentation model to realize automatic recognition and effective segmentation of fetal cranial ultrasound structure, which is beneficial to improve the accuracy of fetal cranial ultrasound structure segmentation, help operators quickly locate the location of fetal cranial ultrasound structure, and improve the operator's inspection efficiency and accuracy.

[0213] In this optional embodiment, as an optional implementation, the specific manner in which the adjustment module 307 adjusts the first training segmentation model based on the second sample data set is:

[0214] Determining a target number of sample data pairs in the second sample data set required for a single adjustment of the first training segmentation model according to the determined number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set;

[0215] Inputting a target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, and obtaining an analysis result output by the first training segmentation model, adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, and updating the adjusted segmentation model to the first training segmentation model;

[0216] Re-execute the steps of inputting a target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, obtaining the analysis result output by the first training segmentation model, and adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, until the total number of adjustments of the first training segmentation model reaches the determined adjustment number threshold;

[0217] The step of determining the adjusted first training segmentation model as the second training segmentation model includes:

[0218] The segmentation model obtained after the last adjustment is determined as the second training segmentation model.

[0219] It can be seen that implementation Figure 4 The described device can determine the model adjustment method according to the characteristics of the second sample data set corresponding to the sample fetal cranial ultrasound structure, make full and effective use of the second sample data set, which is conducive to improving the accuracy and reliability of the first training segmentation model, thereby improving the accuracy of fetal cranial ultrasound structure segmentation.

[0220] In this optional embodiment, as another optional implementation, the specific manner in which the optimization module 308 calculates the loss value of the second training segmentation model based on the determined loss function is:

[0221] Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis;

[0222] Obtaining the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample;

[0223] Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set;

[0224] Based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, the loss value of the second training segmentation model is calculated, wherein the loss function is any one of a cross entropy cost function, a hinge loss function, and a mean square error loss function, and the loss value is used to characterize the difference between the predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data.

[0225] It can be seen that implementation Figure 4 The described device can improve the accuracy of calculating the loss value, so that the segmentation effect of the second training segmentation model can be effectively evaluated based on the loss value, and an appropriate evaluation criterion is provided for the model optimization effect, thereby improving the efficiency of model optimization.

[0226] In this optional implementation, further optionally, the optimization module 308 determines the specific manner in which the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is:

[0227] Determine that the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is an equal weight; or,

[0228] The weight corresponding to each voxel is determined according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel.

[0229] It can be seen that implementation Figure 4 The described device can determine reasonable weights for voxels of sample labeled volume data based on data features of the second sample data set and / or requirements for a three-dimensional convolutional neural network segmentation model, especially when the number of voxels in different sample labeled volume data is unbalanced or the voxel categories are unbalanced. This can improve the accuracy of calculating loss values, reduce overfitting during model optimization, and improve the optimization effect of the model, thereby facilitating improving the accuracy of fetal cranial ultrasound structure segmentation.

[0230] In this optional embodiment, as another optional implementation, the optimization module 308 optimizes the parameters of the second training segmentation model based on the loss value in the following specific manner:

[0231] Determining a learning rate of a stochastic gradient descent method, performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, and reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method;

[0232] Re-execute the steps of performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method until the number of executions of the optimization operation reaches the determined optimization number threshold.

[0233] It can be seen that implementation Figure 4 The described device can use the stochastic gradient descent method to optimize the parameters of the second training segmentation model. Since the stochastic gradient descent method only needs to calculate the first-order derivative of the loss function, the optimization speed is faster, and multiple optimization operations are performed on the parameters of the second training segmentation model, which is beneficial to improving the accuracy of fetal cranial ultrasound structure segmentation.

[0234] In this optional embodiment, as another optional implementation, as Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0235] The judgment module 309 is used to determine whether the loss value is less than or equal to a preset loss threshold after calculating the loss value of the second training segmentation model based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data of the second training segmentation model input in the optimization module 308 based on the determined second sample data set and the determined loss function; when the judgment result is yes, the optimization module 308 is triggered to execute the operation of optimizing the parameters of the second training segmentation model based on the loss value; when the judgment result is no, the adjustment module 307 is triggered to re-execute the operation of adjusting the first training segmentation model based on the second sample data set.

[0236] It can be seen that implementation Figure 4 The described device can optimize the parameters of the model only when the loss value of the second training segmentation model meets the preset conditions, which can improve the effects of model adjustment and model optimization, thereby facilitating improving the accuracy of fetal cranial ultrasound structure segmentation.

[0237] In this optional embodiment, as another optional implementation, as Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0238] The second detection module 310 is used for detecting whether there are sample data pairs that do not meet the preset quality conditions in the third sample data set after the data acquisition module 301 acquires the first sample data set corresponding to the non-fetal cranial ultrasound structure and the second sample data set corresponding to the sample fetal cranial ultrasound structure, and when the detection result is no, triggering the training module 306 to execute the operation of building the original three-dimensional convolutional neural network segmentation model, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality conditions include at least one of sample data pairs with missing data, sample data pairs with repeated data, and sample data pairs with wrong data;

[0239] The data cleaning module 311 is used to delete the sample data pairs that do not meet the preset quality conditions in the third sample data set when the second detection module 310 detects that there are sample data pairs that do not meet the preset quality conditions in the third sample data set, or to correct the sample data pairs that do not meet the preset quality conditions in the third sample data set, so as to obtain the sample data pairs that meet the preset quality conditions, and trigger the training module 306 to execute the operation of building the original three-dimensional convolutional neural network segmentation model.

[0240] It can be seen that implementation Figure 4 The described device can delete or correct sample data pairs that do not meet preset quality conditions in the sample data set, increase the number of sample data sets, and help improve the effects of model training, adjustment and optimization, thereby helping to improve the accuracy of fetal cranial ultrasound structure segmentation.

[0241] In this optional embodiment, as another optional implementation, as Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0242] A determination module 312 is used to determine the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, and determine a data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, where the data enhancement method is used to increase the number of sample data pairs in the second sample data set;

[0243] The data enhancement module 313 is used to perform a data enhancement operation on the data in the second sample data set according to the determined data enhancement method after the determination module 312 determines the data enhancement method for the second sample data set, to obtain a sample data set after data enhancement, and to update the sample data set after data enhancement to the second sample data set, and to trigger the adjustment module 307 to perform an operation of adjusting the first training segmentation model based on the second sample data set.

[0244] It can be seen that implementation Figure 4 The described device can increase the number of sample data pairs in the second sample data set, reduce the overfitting during model adjustment and model optimization caused by the small number of sample data pairs, improve the effect of model adjustment and model optimization, and thus help improve the accuracy of fetal cranial ultrasound structure segmentation.

[0245] In yet another optional embodiment, Figure 4 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may also include:

[0246] The rendering module 314 is used to obtain the analysis result output by the three-dimensional convolutional neural network segmentation model in the segmentation module 302 as the segmentation result of the fetal cranial ultrasound structure, and then render the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image.

[0247] The transmission module 315 is used to use the rendered image as a segmented image of the fetal cranial ultrasound structure, and transmit the segmented image of the fetal cranial ultrasound structure to a corresponding image output device.

[0248] It can be seen that implementation Figure 4 The described device can render and display the segmentation results of the fetal cranial ultrasound structure, process the three-dimensional model of the fetal cranial ultrasound structure into a real picture, and make the segmentation results of the fetal cranial ultrasound structure visually closer to the actual original.

[0249] Embodiment 4

[0250] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of another fetal cranial ultrasound structure segmentation device based on a three-dimensional convolutional neural network disclosed in an embodiment of the present invention. Figure 5 As shown, the fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network may include:

[0251] A memory 401 storing executable program codes;

[0252] a processor 402 coupled to the memory 401;

[0253] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the fetal cranial ultrasound structure segmentation method based on the three-dimensional convolutional neural network described in the first embodiment of the present invention or the second embodiment of the present invention.

[0254] Embodiment 5

[0255] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the fetal cranial ultrasound structure segmentation method based on a three-dimensional convolutional neural network described in Embodiment 1 or Embodiment 2 of the present invention.

[0256] Embodiment 6

[0257] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the fetal cranial ultrasound structure segmentation method based on a three-dimensional convolutional neural network described in Example 1 or Example 2.

[0258] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0259] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0260] Finally, it should be noted that the method and device for fetal cranial ultrasound structure segmentation based on a three-dimensional convolutional neural network disclosed in the embodiment of the present invention discloses only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network. It is characterized in that The method comprises: Obtain the original volume data of fetal cranial ultrasound structure; Inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, and obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure; After obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure, the method further includes: Rendering the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image; Using the rendered image as a segmented image of the fetal cranial ultrasound structure, and transmitting the segmented image of the fetal cranial ultrasound structure to a corresponding image output device; The rendering of the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image includes: Determining the range that needs to be rendered in the segmentation result of the fetal cranial ultrasound structure, the type of virtual light source required for rendering, and the parameters of the virtual light source; Based on the type of the virtual light source, the parameters of the virtual light source and the voxel category to which each voxel within the range to be rendered in the segmentation result belongs, coloring the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image; And, the method further comprises: Acquire a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, wherein the first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmentation of the sample original volume data; Building an original three-dimensional convolutional neural network segmentation model, training the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determining the trained original three-dimensional convolutional neural network segmentation model as a first training segmentation model; Adjusting the first training segmentation model based on the second sample data set, and determining the adjusted first training segmentation model as the second training segmentation model; Calculating a loss value of the second training segmentation model based on the determined loss function, optimizing parameters of the second training segmentation model based on the loss value, and determining the second training segmentation model after the optimized parameters as a three-dimensional convolutional neural network segmentation model; The calculating the loss value of the second training segmentation model based on the determined loss function includes: Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis; Acquire the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample; Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set; Calculating a loss value of the second training segmentation model based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, wherein the loss function is any one of a hinge loss function and a mean square error loss function, and the loss value is used to characterize a difference between a predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data; The determining of the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set includes: Determine a weight corresponding to each voxel according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel; Furthermore, after acquiring the first sample data set corresponding to the non-fetal cranial ultrasound structure and the second sample data set corresponding to the sample fetal cranial ultrasound structure, and before building the original three-dimensional convolutional neural network segmentation model, the method further includes: Detecting whether there are sample data pairs that do not meet the preset quality condition in the third sample data set, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality condition include at least one of a sample data pair with missing data, a sample data pair with duplicate data, and a sample data pair with incorrect data; When the detection result is yes, the sample data pairs in the third sample data set that do not meet the preset quality condition are deleted, or the sample data pairs in the third sample data set that do not meet the preset quality condition are corrected to obtain sample data pairs that meet the preset quality condition, and the operation of building the original three-dimensional convolutional neural network segmentation model is triggered; When the detection result is no, triggering the execution of the operation of building the original three-dimensional convolutional neural network segmentation model; The step of correcting the sample data pairs in the third sample data set that do not meet the preset quality condition includes: Fill in the missing data in the sample data pair with missing data, and / or replace the erroneous data in the sample data pair with erroneous data, and / or perform any one of flipping, rotating and cropping operations on the sample data pair with repeated data, and replace the sample data pair with repeated data with the sample data pair after performing any one of the flipping, rotating and cropping operations.

2. The fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network according to claim 1, It is characterized in that Before inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, the method further includes: Detecting whether there is sensitive information in the original volume data of the fetal cranial ultrasound structure, wherein the sensitive information includes the privacy information of the individual corresponding to the fetal cranial ultrasound structure, and the privacy information of the individual includes one or more of the individual's name, gender, age, date of birth, and contact information; When it is detected that the sensitive information exists in the original volume data of the fetal cranial brain ultrasound structure, the sensitive information is removed, and the original volume data after the sensitive information is removed is updated to the original volume data of the fetal cranial brain ultrasound structure, and the operation of inputting the original volume data of the fetal cranial brain ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered; And, the method further comprises: When it is detected that the sensitive information does not exist in the original volume data of the fetal cranial ultrasound structure, the operation of inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis is triggered.

3. The fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network according to claim 1, It is characterized in that The adjusting the first training segmentation model based on the second sample data set includes: Determining a target number of sample data pairs in the second sample data set required for a single adjustment of the first training segmentation model according to the determined number of sample data pairs in the second sample data set and the number of voxels of each of the sample original volume data in the second sample data set; Inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, and obtaining an analysis result output by the first training segmentation model, adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, and updating the adjusted segmentation model as the first training segmentation model; Re-performing the step of inputting the target number of sample data pairs selected from the second sample data set into the first training segmentation model for analysis, obtaining the analysis result output by the first training segmentation model, and adjusting the first training segmentation model based on the analysis result to obtain an adjusted segmentation model, until the total number of adjustments of the first training segmentation model reaches the determined adjustment number threshold; And, determining the adjusted first training segmentation model as the second training segmentation model includes: The segmentation model obtained after the last adjustment is determined as the second training segmentation model.

4. The fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network according to claim 1, It is characterized in that The optimizing the parameters of the second training segmentation model based on the loss value includes: Determining a learning rate of a stochastic gradient descent method, performing a first preset number of optimization operations on the parameters of the second training segmentation model using the stochastic gradient descent method based on the learning rate of the stochastic gradient descent method, and reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method; Re-execute the steps of performing a first preset number of optimization operations on the parameters of the second training segmentation model based on the learning rate of the stochastic gradient descent method using the stochastic gradient descent method, reducing the learning rate of the stochastic gradient descent method to obtain a reduced learning rate, and updating the reduced learning rate to the learning rate of the stochastic gradient descent method, until the number of executions of the optimization operation reaches the determined optimization number threshold.

5. The fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network according to claim 3, It is characterized in that Before adjusting the first training segmentation model based on the second sample data set, the method further includes: Determine the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set; Determine a data enhancement method for the second sample data set according to the number of sample data pairs in the second sample data set and the number of voxels of each sample original volume data in the second sample data set, wherein the data enhancement method is used to increase the number of sample data pairs in the second sample data set; Perform a data enhancement operation on the data in the second sample data set according to the determined data enhancement method to obtain a sample data set after data enhancement, update the sample data set after data enhancement to the second sample data set, and trigger the execution of the operation of adjusting the first training segmentation model based on the second sample data set.

6. A fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network, It is characterized in that The device comprises: A data acquisition module, used to obtain the original volume data of the fetal cranial ultrasound structure; A segmentation module, used for inputting the original volume data of the fetal cranial ultrasound structure into the determined three-dimensional convolutional neural network segmentation model for analysis, and obtaining the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure; The device also includes: A rendering module, configured to render the segmentation result of the fetal cranial ultrasound structure after the segmentation module obtains the analysis result output by the three-dimensional convolutional neural network segmentation model as the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image; A transmission module, used for using the rendered image as a segmented image of the fetal cranial ultrasound structure, and transmitting the segmented image of the fetal cranial ultrasound structure to a corresponding image output device; The device is also used for: The method of rendering the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image specifically includes: Determining the range that needs to be rendered in the segmentation result of the fetal cranial ultrasound structure, the type of virtual light source required for rendering, and the parameters of the virtual light source; Based on the type of the virtual light source, the parameters of the virtual light source and the voxel category to which each voxel within the range to be rendered in the segmentation result belongs, coloring the segmentation result of the fetal cranial ultrasound structure to obtain a rendered image; The data acquisition module is further used to acquire a first sample data set corresponding to a non-fetal cranial ultrasound structure and a second sample data set corresponding to a sample fetal cranial ultrasound structure, wherein the first sample data set and the second sample data set each include at least one sample data pair, and each sample data pair includes a sample original volume data and sample labeled volume data obtained after segmentation of the sample original volume data; The device also includes: A training module, used to build an original three-dimensional convolutional neural network segmentation model, train the original three-dimensional convolutional neural network segmentation model based on the first sample data set, and determine the trained original three-dimensional convolutional neural network segmentation model as a first training segmentation model; an adjustment module, configured to adjust the first training segmentation model based on the second sample data set, and determine the adjusted first training segmentation model as the second training segmentation model; An optimization module, configured to calculate a loss value of the second training segmentation model based on the determined loss function, optimize parameters of the second training segmentation model based on the loss value, and determine the second training segmentation model after the optimized parameters as a three-dimensional convolutional neural network segmentation model; The specific manner in which the optimization module calculates the loss value of the second training segmentation model based on the determined loss function is: Inputting the original volume data of the samples in the second sample data set into the second training segmentation model for analysis; Acquire the analysis result output by the second training segmentation model as the predicted segmentation result of the original volume data of the sample; Determine a weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set; Calculating a loss value of the second training segmentation model based on the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the determined second sample data set and the determined loss function, wherein the loss function is any one of a hinge loss function and a mean square error loss function, and the loss value is used to characterize a difference between a predicted segmentation result of the sample original volume data and the sample labeled volume data corresponding to the sample original volume data; The specific manner in which the optimization module determines the weight corresponding to each voxel of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set is: Determine a weight corresponding to each voxel according to at least one of the number of voxels of the sample labeled volume data corresponding to the sample original volume data input into the second training segmentation model in the second sample data set, the position of each voxel, and the category of each voxel; The device also includes: A second detection module is used to detect whether there are sample data pairs that do not meet the preset quality conditions in the third sample data set after the data acquisition module acquires the first sample data set corresponding to the non-fetal cranial ultrasound structure and the second sample data set corresponding to the sample fetal cranial ultrasound structure, and before the training module builds the original three-dimensional convolutional neural network segmentation model. When the detection result is no, the training module is triggered to perform the operation of building the original three-dimensional convolutional neural network segmentation model, wherein the third sample data set includes the first sample data set and / or the second sample data set, and the sample data pairs that do not meet the preset quality conditions include at least one of sample data pairs with missing data, sample data pairs with repeated data, and sample data pairs with incorrect data; A data cleaning module, configured to, when the second detection module detects that there are sample data pairs that do not meet the preset quality conditions in the third sample data set, delete the sample data pairs that do not meet the preset quality conditions in the third sample data set, or correct the sample data pairs that do not meet the preset quality conditions in the third sample data set, to obtain sample data pairs that meet the preset quality conditions, and trigger the training module to perform the operation of building the original three-dimensional convolutional neural network segmentation model; The device is also used for: When the detection result of the second detection module is no, triggering the training module to execute the operation of building the original three-dimensional convolutional neural network segmentation model; The method of correcting the sample data pairs in the third sample data set that do not meet the preset quality condition specifically includes: Fill in the missing data in the sample data pair with missing data, and / or replace the erroneous data in the sample data pair with erroneous data, and / or perform any one of flipping, rotating and cropping operations on the sample data pair with repeated data, and replace the sample data pair with repeated data with the sample data pair after performing any one of the flipping, rotating and cropping operations.

7. A fetal cranial ultrasound structure segmentation device based on three-dimensional convolutional neural network, It is characterized in that The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the fetal cranial ultrasound structure segmentation method based on three-dimensional convolutional neural network as described in any one of claims 1-5.

Images