Fetal ultrasound key section recognition method and device based on convolutional neural network

Abstract

The present invention discloses a fetal ultrasound key section recognition method and device based on a convolutional neural network, wherein the method comprises: obtaining original ultrasound image data including fetal structural section images; determining target ultrasound image data according to the original ultrasound image data; inputting the target ultrasound image data into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is obtained by training a training data set including a plurality of training ultrasound image data labeled with section types. It can be seen that the present invention accurately classifies the section information in the ultrasound image data based on the trained neural network model, which is conducive to helping doctors quickly classify and judge the biological structure sections, improve the inspection efficiency, reduce the misdiagnosis and missed diagnosis rate, and promote the effective use of medical resources at the same time, and has great practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and device for identifying key sections of fetal ultrasound based on a convolutional neural network. Background Art

[0002] With the development of medical imaging technology, the existing medical field has begun to introduce more and more medical images, especially ultrasound images, into the diagnosis and treatment process when combining biological structure section information for diagnosis and treatment. As a result, the efficiency requirements for the type recognition of biological structure sections in ultrasound images have become increasingly heavy. In the prior art, the type of biological structure section is generally recognized by the naked eye of the physician, and the introduction of neural network technology is not considered to realize automatic recognition of the type of biological structure section in ultrasound image data. Therefore, the recognition efficiency of the prior art is low, and it is easy to misdiagnose and miss the diagnosis. It can be seen that the prior art has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device for identifying key sections of fetal ultrasound based on a convolutional neural network, which can accurately classify the section information in the ultrasound image data based on the trained neural network model, which is helpful to help doctors quickly classify and judge the sections of biological structures, improve inspection efficiency, reduce the misdiagnosis and missed diagnosis rate, and at the same time promote the effective use of medical resources, and has great practical value.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a fetal ultrasound key section recognition method based on a convolutional neural network, the method comprising:

[0005] Acquiring original ultrasound image data including cross-sectional images of fetal structures;

[0006] Determining target ultrasonic image data according to the original ultrasonic image data;

[0007] The target ultrasound image data is input into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types.

[0008] As an optional implementation, in the first aspect of the present invention, determining the target ultrasonic image data according to the original ultrasonic image data includes:

[0009] Determining sensitive information in the original ultrasound image data;

[0010] The sensitive information is removed to obtain target ultrasound image data.

[0011] As an optional implementation, in the first aspect of the present invention, determining the sensitive information in the original ultrasound image data includes:

[0012] Determining a purpose of capturing the original ultrasound image data;

[0013] According to the photographing purpose, determining irrelevant data in the original ultrasonic image data that is irrelevant to the photographing purpose;

[0014] determining the irrelevant data as sensitive information in the original ultrasound image data;

[0015] and / or,

[0016] Determining the privacy data in the original ultrasound image data; the privacy data includes at least one of private disease information, private body part information and patient personal information;

[0017] The privacy data is determined as sensitive information in the original ultrasound image data.

[0018] As an optional embodiment, in the first aspect of the present invention, the classification convolutional neural network model is a convolutional neural network classification model developed and tested under the deep learning framework Pytorch.

[0019] As an optional implementation, in the first aspect of the present invention, the classification convolutional neural network model is trained according to the following steps:

[0020] Determine a training data set including a plurality of training ultrasound image data labeled with section types and a training classification model; the training classification model includes the classification convolutional neural network model and a loss function calculation layer;

[0021] The training ultrasound image data and the section type annotations in the training data set are input into the training classification model in the form of paired labels for training, and the stochastic gradient descent method is used during the training process to continuously reduce the loss function value calculated by the loss function layer, and the training network model is optimized until convergence to obtain the trained classification convolutional neural network model.

[0022] As an optional implementation, in the first aspect of the present invention, the loss function of the loss function calculation layer is a cross entropy cost function.

[0023] As an optional implementation, in the first aspect of the present invention, the stochastic gradient descent method is used during the training process to make the loss function value calculated by the loss function layer continuously decrease, and the training network model is optimized until convergence, including:

[0024] The model parameters of the training network model were optimized using a stochastic gradient descent algorithm, with a total of 1000 iterations, an initial learning rate of 0.002, and a decrease in the learning rate every 500 rounds, so that the loss function value calculated by the loss function layer continued to decrease so that the training network model reached the optimal value.

[0025] The second aspect of the present invention discloses a fetal ultrasound key section recognition device based on a convolutional neural network, comprising:

[0026] An acquisition module, used for acquiring original ultrasound image data including fetal structural section images;

[0027] A determination module, configured to determine target ultrasonic image data according to the original ultrasonic image data;

[0028] The recognition module is used to input the target ultrasound image data into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types.

[0029] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the target ultrasonic image data according to the original ultrasonic image data includes:

[0030] Determining sensitive information in the original ultrasound image data;

[0031] The sensitive information is removed to obtain target ultrasound image data.

[0032] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the sensitive information in the original ultrasound image data includes:

[0033] Determining a purpose of capturing the original ultrasound image data;

[0034] According to the photographing purpose, determining irrelevant data in the original ultrasonic image data that is irrelevant to the photographing purpose;

[0035] determining the irrelevant data as sensitive information in the original ultrasound image data;

[0036] and / or,

[0037] Determining the privacy data in the original ultrasound image data; the privacy data includes at least one of private disease information, private body part information and patient personal information;

[0038] The privacy data is determined as sensitive information in the original ultrasound image data.

[0039] As an optional embodiment, in the second aspect of the present invention, the classification convolutional neural network model is a convolutional neural network classification model developed and tested under the deep learning framework Pytorch.

[0040] As an optional implementation, in the second aspect of the present invention, the device further includes: a training module, configured to perform the following steps to train and obtain the classification convolutional neural network model:

[0041] Determine a training data set including a plurality of training ultrasound image data labeled with section types and a training classification model; the training classification model includes the classification convolutional neural network model and a loss function calculation layer;

[0042] The training ultrasound image data and the section type annotations in the training data set are input into the training classification model in the form of paired labels for training, and the stochastic gradient descent method is used during the training process to continuously reduce the loss function value calculated by the loss function layer, and the training network model is optimized until convergence to obtain the trained classification convolutional neural network model.

[0043] As an optional implementation, in the second aspect of the present invention, the loss function of the loss function calculation layer is a cross entropy cost function.

[0044] As an optional implementation, in the second aspect of the present invention, the training module uses the stochastic gradient descent method during the training process to continuously reduce the loss function value calculated by the loss function layer, and optimizes the training network model until convergence, including:

[0045] The model parameters of the training network model were optimized using a stochastic gradient descent algorithm, with a total of 1000 iterations, an initial learning rate of 0.002, and a decrease in the learning rate every 500 rounds, so that the loss function value calculated by the loss function layer continued to decrease so that the training network model reached the optimal value.

[0046] The third aspect of the present invention discloses another fetal ultrasound key section recognition device based on a convolutional neural network, the device comprising:

[0047] A memory storing executable program code;

[0048] a processor coupled to the memory;

[0049] The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for identifying key sections of fetal ultrasound based on a convolutional neural network disclosed in the first aspect of an embodiment of the present invention.

[0050] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0051] In an embodiment of the present invention, original ultrasound image data including fetal structural section images are obtained; target ultrasound image data are determined based on the original ultrasound image data; the target ultrasound image data is input into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is trained using a training data set including a plurality of training ultrasound image data labeled with section types. It can be seen that the present invention accurately classifies section information in ultrasound image data based on a trained neural network model, which is helpful for helping doctors to quickly classify and judge biological structure sections, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and promote the effective use of medical resources, which has great practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a flow chart of a method for identifying key sections of fetal ultrasound based on a convolutional neural network disclosed in an embodiment of the present invention;

[0054] Figure 2 It is a structural schematic diagram of a fetal ultrasound key section recognition device based on a convolutional neural network disclosed in an embodiment of the present invention;

[0055] Figure 3 It is a structural schematic diagram of another fetal ultrasound key section recognition device based on convolutional neural network disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0058] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] The present invention discloses a fetal ultrasound key section recognition method and device based on a convolutional neural network, which can accurately classify section information in ultrasound image data based on a trained neural network model, which is helpful for helping doctors to quickly classify and judge biological structure sections, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and promote the effective use of medical resources, which has great practical value. The following are detailed descriptions.

[0060] Embodiment 1

[0061] See also Figure 1 , Figure 1 1 is a flow chart of a method for identifying key sections of fetal ultrasound based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 1 The described method can be applied to a corresponding image processing terminal, an image processing device or an image processing server, and the server can be a local server or a cloud server. Figure 1 As shown, the fetal ultrasound key section recognition method based on convolutional neural network may include the following operations:

[0062] 101. Acquire original ultrasound image data including cross-sectional images of fetal structures.

[0063] Optionally, the ultrasound image data of the present invention may be a dynamic image or a static image, which is not limited by the present invention. Optionally, the ultrasound image data may be two-dimensional image data, which may include not only grayscale ultrasound images but also Doppler images, which may be obtained by transmitting ultrasound to a certain part of the internal tissue structure of the fetus through an ultrasound imaging device, receiving and processing ultrasound echo signals, and may include at least one fetal structural section image.

[0064] 102. Determine target ultrasound image data based on the original ultrasound image data.

[0065] Optionally, the target ultrasound image data can be obtained by data expansion methods, specifically by enlarging, translating, rotating and contrast enhancing the original image. The ultrasound image enlargement operation should be performed within an appropriate range to prevent the image from being enlarged too much, thereby losing the effective ultrasound image area; the translation operation should also pay attention to the size of the translation to prevent the loss of the effective area; the rotation operation is mainly mirror rotation along the vertical axis, rotation of 90° and 180°, and the grayscale range of the image is widened by various methods, thereby performing contrast enhancement operations on the ultrasound image. Using the above method to expand the ultrasound image data can enrich the diversity of ultrasound image data features and improve the universality and stability of the algorithm.

[0066] 103. Input the target ultrasound image data into the trained classification convolutional neural network model to obtain the section type classification result.

[0067] Optionally, the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types. Optionally, the labeling of the training ultrasound image data can be completed by an operator such as an experienced physician. Optionally, the classification convolutional neural network model is a convolutional neural network classification model, which is developed and tested under the deep learning framework Pytorch. Specifically, the classification convolutional neural network model can be built based on the deep convolutional neural network VGGNet, which proposes the idea of ​​replacing network layers with basic blocks, while better maintaining the image properties, increasing the network depth to ensure learning of more complex patterns to improve the precision and accuracy of image classification and recognition.

[0068] It can be seen that the method described in the embodiment of the present invention can accurately classify the section information in the ultrasound image data based on the trained neural network model, which is helpful to help doctors quickly classify and judge the sections of biological structures, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and at the same time promote the effective use of medical resources, which has great practical value.

[0069] As an optional implementation, in the above step 102, determining the target ultrasound image data according to the original ultrasound image data includes:

[0070] Determining sensitive information in raw ultrasound image data;

[0071] Remove sensitive information and obtain target ultrasound image data.

[0072] Optionally, sensitive information can be removed by replacing the true value with a set fixed value, or by desensitizing the sensitive information by truncating, encrypting, hiding, etc. the data value so that it is no longer of use value, or by replacing the true value with random data and maintaining the randomness of the replacement value to simulate the authenticity of the sample.

[0073] It can be seen that through this optional implementation, sensitive information in the original ultrasound image data can be removed to obtain ultrasound image data including target biological structure sections, so that the target ultrasound image data can be calculated more accurately, reducing the risk of patient privacy leakage, and improving the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0074] As an optional implementation, in the above step, determining the sensitive information in the original ultrasound image data includes:

[0075] Determine the purpose of capturing the original ultrasound image data;

[0076] According to the purpose of shooting, irrelevant data irrelevant to the purpose of shooting in the original ultrasound image data is determined;

[0077] The irrelevant data is determined as sensitive information in the original ultrasound image data.

[0078] Optionally, the purpose of the shooting can be determined by retrieving the medical files corresponding to the original ultrasound image data. Optionally, the purpose of the shooting can be a treatment purpose, a diagnosis purpose, a research purpose, or a diagnosis purpose for a specific disease.

[0079] Optionally, irrelevant data in the original ultrasound image data that is irrelevant to the purpose of shooting can be judged by an algorithm model or by an operator. Optionally, a classification network model can be trained by a training data set including a plurality of ultrasound image data labeled with irrelevant data to judge irrelevant data.

[0080] It can be seen that through this optional implementation method, sensitive information in the original medical image data that is irrelevant to the shooting purpose can be determined according to the shooting purpose, so as to facilitate the subsequent removal of sensitive information, thereby being able to more accurately calculate the target ultrasound image data, reduce the risk of patient privacy leakage, and improve the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0081] As an optional implementation, in the above step, determining the sensitive information in the original ultrasound image data includes:

[0082] Determining the privacy data in the original ultrasound image data;

[0083] The private data is identified as the sensitive information in the original ultrasound image data.

[0084] Optionally, the private data includes at least one of private disease information, private body part information and patient personal information. Optionally, the private disease information and patient personal information can be judged by the data label or the text content of the data, for example: manually establish a small set of keywords including private disease information or patient personal information, then perform word segmentation preprocessing on the text data in the historical ultrasound image data to obtain multiple word data, calculate the Euclidean distance between each word data and the keywords in the small set respectively, and by setting the Euclidean distance threshold, determine the word data with a Euclidean distance less than the threshold as a keyword, and add it to the small set to form a private information word library. Further, the private disease information and patient personal information in the original ultrasound image data can be identified and determined based on the private information word library. For example, in the subsequent, the text data in the original ultrasound image data can be segmented to obtain multiple word data, and the similarity of each word data and all the keywords in the private information word library is calculated. When the similarity between any word data and any keyword exceeds the preset similarity threshold, the word data is determined to be private disease information or patient personal information, so as to be cleared in the subsequent.

[0085] Optionally, the private part information can be used to refer to the image data belonging to the private part in the original ultrasound image data, and the determination method can be through a trained image recognition algorithm model for private part identification, for example: obtaining cross-sectional images of different angles of different private parts to form a pre-stored image database, and inputting the original ultrasound image data into the image recognition model to calculate the similarity between all or any part of the original ultrasound image data and each cross-sectional image data in the pre-stored image database, and judging the data belonging to the private part information in the original ultrasound image data based on the similarity. For example, when the similarity between a part of the original ultrasound image data and any cross-sectional image data in the pre-stored image database exceeds a preset threshold, it is judged that the part of the data belongs to the private part information.

[0086] It can be seen that through this optional implementation, the privacy data can be determined as sensitive information in the original ultrasound image data, so that the sensitive information can be removed later, so that the target ultrasound image data can be calculated more accurately, reducing the risk of patient privacy leakage, and improving the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0087] As an optional implementation, the classification convolutional neural network model is trained according to the following steps:

[0088] Determine a training data set including a plurality of training ultrasound image data labeled with section types and a training classification model; the training classification model includes a classification convolutional neural network model and a loss function calculation layer;

[0089] The training ultrasound image data and section type annotations in the training data set are input into the training classification model in the form of paired labels for training. During the training process, the stochastic gradient descent method is used to continuously reduce the loss function value calculated by the loss function layer, and the training network model is optimized until convergence to obtain a trained classification convolutional neural network model.

[0090] Optionally, the loss function of the loss function calculation layer is a cross entropy cost function to measure the similarity between the distribution of the slice type annotations of the training ultrasound image data and the distribution of the slice type prediction results output by the training network model.

[0091] It can be seen that through this optional implementation, the training ultrasound image data and section type annotations in the training data set can be input into the training classification model in the form of paired labels, and the training optimization can be performed until convergence, so as to accurately train the classification convolutional neural network model and classify the section information in the ultrasound image data more accurately and efficiently.

[0092] As an optional implementation, in the above steps, the stochastic gradient descent method is used during the training process to continuously reduce the loss function value calculated by the loss function layer, and the training network model is optimized until convergence, including:

[0093] The stochastic gradient descent algorithm was used to optimize the model parameters of the training network model. A total of 1000 iterations were performed, with an initial learning rate of 0.002 and a decrease in the learning rate every 500 rounds, so that the loss function value calculated by the loss function layer continued to decrease, so that the training network model reached the optimal value.

[0094] It can be seen that through this optional implementation, the training classification model can be fully optimized until convergence, so that the classification convolutional neural network model can be trained more accurately and the section information in the ultrasound image data can be classified more accurately and efficiently.

[0095] As an optional implementation, in the above step 103, the target ultrasound image data is input into the trained classification convolutional neural network model to obtain the section type classification result, including:

[0096] The target ultrasound image data is transformed in different ways to obtain a plurality of transformed image data;

[0097] Inputting multiple transformed image data into the trained classification convolutional neural network model respectively to obtain multiple section type classification results;

[0098] According to the multiple slice type classification results, a final slice type classification result of the target ultrasound image data is determined.

[0099] Optionally, the transformation of the target ultrasonic image data should be a transformation while retaining the original size of the target ultrasonic image data to prevent the transformed image data from losing too many original details. Optionally, the transformation can be to perform denoising and smoothing processing on the target ultrasonic image, and then based on the wavelet threshold denoising method, perform wavelet decomposition, quantize the decomposition layer coefficients, wavelet inverse transformation and other operations on the signal corresponding to the processed image through a preset number of decomposition layers and wavelet basis functions to obtain the transformed image data. This transformation method can effectively suppress speckle noise and enhance image data to more accurately extract the features of the ultrasonic image data.

[0100] Optionally, the transformation of the target ultrasound image data may also be to calculate the grayscale histogram of the target ultrasound image and crop it to obtain a new grayscale histogram, and then map each grayscale level in the new grayscale histogram to a new grayscale level through the grayscale cumulative frequency and grayscale mapping function to obtain the transformed image data.

[0101] Optionally, for multiple slice type classification results of multiple transformed image data, a corresponding weight percentage can be assigned to each transformed image data, and the multiple slice type classification results are weighted and summed according to the weight percentages of their respective corresponding transformed image data to determine the slice type classification result of the final target ultrasound image data. Generally speaking, the sum of the weight percentages of all transformed image data should be 1. For example, the classification convolutional neural network model can output a predicted slice type and a corresponding prediction score for each transformed image data, then the prediction scores corresponding to all transformed image data can be weightedly calculated, and then all weighted prediction scores corresponding to different predicted slice types are added, and finally the slice type with the highest added prediction score is determined as the final slice type classification result of the target ultrasound image data. Specifically, the weight percentage of each transformed image data can be proportional to the similarity between the transformed image data and the target ultrasound image data, and the weight percentage can be determined after calculating the similarity between each transformed image data and the target ultrasound image data.

[0102] It can be seen that through this optional implementation method, the section information in the target ultrasound image data can be accurately classified based on the trained neural network model, which is helpful to help doctors quickly classify and judge the sections of biological structures, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and at the same time promote the effective use of medical resources, which has great practical value.

[0103] Embodiment 2

[0104] See also Figure 2 , Figure 2: is a schematic diagram of a fetal ultrasound key section recognition device based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 2 The described apparatus can be applied to a corresponding image processing terminal, an image processing device or an image processing server, and the server can be a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 2 As shown, the device may include:

[0105] The acquisition module 201 is used to acquire original ultrasound image data including fetal structural cross-sectional images.

[0106] Optionally, the ultrasound image data of the present invention may be a dynamic image or a static image, which is not limited by the present invention. Optionally, the ultrasound image data may be two-dimensional image data, which may include not only grayscale ultrasound images but also Doppler images, which may be obtained by transmitting ultrasound to a certain part of the internal tissue structure of the fetus through an ultrasound imaging device, receiving and processing ultrasound echo signals, and may include at least one fetal structural section image.

[0107] The determination module 202 is used to determine the target ultrasound image data according to the original ultrasound image data.

[0108] Optionally, the target ultrasound image data can be obtained by data expansion methods, specifically by enlarging, translating, rotating and contrast enhancing the original image. The ultrasound image enlargement operation should be performed within an appropriate range to prevent the image from being enlarged too much, thereby losing the effective ultrasound image area; the translation operation should also pay attention to the size of the translation to prevent the loss of the effective area; the rotation operation is mainly mirror rotation along the vertical axis, rotation of 90° and 180°, and the grayscale range of the image is widened by various methods, thereby performing contrast enhancement operations on the ultrasound image. Using the above method to expand the ultrasound image data can enrich the diversity of ultrasound image data features and improve the universality and stability of the algorithm.

[0109] The recognition module 203 is used to input the target ultrasound image data into the trained classification convolutional neural network model to obtain the section type classification result.

[0110] Optionally, the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types. Optionally, the labeling of the training ultrasound image data can be completed by an operator such as an experienced physician. Optionally, the classification convolutional neural network model is a convolutional neural network classification model, which is developed and tested under the deep learning framework Pytorch. Specifically, the classification convolutional neural network model can be built based on the deep convolutional neural network VGGNet, which proposes the idea of ​​replacing network layers with basic blocks, while better maintaining the image properties, increasing the network depth to ensure learning of more complex patterns to improve the precision and accuracy of image classification and recognition.

[0111] It can be seen that the device described in the embodiment of the present invention can accurately classify the section information in the ultrasound image data based on the trained neural network model, which is helpful to help doctors quickly classify and judge the sections of biological structures, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and at the same time promote the effective use of medical resources, which has great practical value.

[0112] As an optional implementation, the specific manner in which the determination module 202 determines the target ultrasound image data according to the original ultrasound image data includes:

[0113] Determining sensitive information in raw ultrasound image data;

[0114] Remove sensitive information and obtain target ultrasound image data.

[0115] Optionally, sensitive information can be removed by replacing the true value with a set fixed value, or by desensitizing the sensitive information by truncating, encrypting, hiding, etc. the data value so that it is no longer of use value, or by replacing the true value with random data and maintaining the randomness of the replacement value to simulate the authenticity of the sample.

[0116] It can be seen that through this optional implementation, sensitive information in the original ultrasound image data can be removed to obtain ultrasound image data including target biological structure sections, so that the target ultrasound image data can be calculated more accurately, reducing the risk of patient privacy leakage, and improving the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0117] As an optional implementation, the specific manner in which the determination module 202 determines the sensitive information in the original ultrasound image data includes:

[0118] Determine the purpose of capturing the original ultrasound image data;

[0119] According to the purpose of shooting, irrelevant data irrelevant to the purpose of shooting in the original ultrasound image data is determined;

[0120] The irrelevant data is determined as sensitive information in the original ultrasound image data.

[0121] Optionally, the purpose of the shooting can be determined by retrieving the medical files corresponding to the original ultrasound image data. Optionally, the purpose of the shooting can be a treatment purpose, a diagnosis purpose, a research purpose, or a diagnosis purpose for a specific disease.

[0122] Optionally, irrelevant data in the original ultrasound image data that is irrelevant to the purpose of shooting can be judged by an algorithm model or by an operator. Optionally, a classification network model can be trained by a training data set including a plurality of ultrasound image data labeled with irrelevant data to judge irrelevant data.

[0123] It can be seen that through this optional implementation method, sensitive information in the original medical image data that is irrelevant to the shooting purpose can be determined according to the shooting purpose, so as to facilitate the subsequent removal of sensitive information, thereby being able to more accurately calculate the target ultrasound image data, reduce the risk of patient privacy leakage, and improve the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0124] As an optional implementation, the specific manner in which the determination module 202 determines the sensitive information in the original ultrasound image data includes:

[0125] Determining the privacy data in the original ultrasound image data;

[0126] The private data is identified as the sensitive information in the original ultrasound image data.

[0127] Optionally, the private data includes at least one of private disease information, private body part information and patient personal information. Optionally, the private disease information and patient personal information can be judged by the data label or the text content of the data, for example: manually establish a small set of keywords including private disease information or patient personal information, then perform word segmentation preprocessing on the text data in the historical ultrasound image data to obtain multiple word data, calculate the Euclidean distance between each word data and the keywords in the small set respectively, and by setting the Euclidean distance threshold, determine the word data with a Euclidean distance less than the threshold as a keyword, and add it to the small set to form a private information word library. Further, the private disease information and patient personal information in the original ultrasound image data can be identified and determined based on the private information word library. For example, in the subsequent, the text data in the original ultrasound image data can be segmented to obtain multiple word data, and the similarity of each word data and all the keywords in the private information word library is calculated. When the similarity between any word data and any keyword exceeds the preset similarity threshold, the word data is determined to be private disease information or patient personal information, so as to be cleared in the subsequent.

[0128] Optionally, the private part information can be used to refer to the image data belonging to the private part in the original ultrasound image data, and the determination method can be through a trained image recognition algorithm model for private part identification, for example: obtaining cross-sectional images of different angles of different private parts to form a pre-stored image database, and inputting the original ultrasound image data into the image recognition model to calculate the similarity between all or any part of the original ultrasound image data and each cross-sectional image data in the pre-stored image database, and judging the data belonging to the private part information in the original ultrasound image data based on the similarity. For example, when the similarity between a part of the original ultrasound image data and any cross-sectional image data in the pre-stored image database exceeds a preset threshold, it is judged that the part of the data belongs to the private part information.

[0129] It can be seen that through this optional implementation, the privacy data can be determined as sensitive information in the original ultrasound image data, so that the sensitive information can be removed later, so that the target ultrasound image data can be calculated more accurately, reducing the risk of patient privacy leakage, and improving the quality of image data used for subsequent processing, which is conducive to the classification convolutional neural network model to achieve better section type recognition effect.

[0130] As an optional implementation, the device further includes: a training module, configured to perform the following steps to train a classification convolutional neural network model:

[0131] Determine a training data set including a plurality of training ultrasound image data labeled with section types and a training classification model; the training classification model includes a classification convolutional neural network model and a loss function calculation layer;

[0132] The training ultrasound image data and section type annotations in the training data set are input into the training classification model in the form of paired labels for training. During the training process, the stochastic gradient descent method is used to continuously reduce the loss function value calculated by the loss function layer, and the training network model is optimized until convergence to obtain a trained classification convolutional neural network model.

[0133] Optionally, the loss function of the loss function calculation layer is a cross entropy cost function to measure the similarity between the distribution of the slice type annotations of the training ultrasound image data and the distribution of the slice type prediction results output by the training network model.

[0134] It can be seen that through this optional implementation, the training ultrasound image data and section type annotations in the training data set can be input into the training classification model in the form of paired labels, and the training optimization can be performed until convergence, so as to accurately train the classification convolutional neural network model and classify the section information in the ultrasound image data more accurately and efficiently.

[0135] As an optional implementation, the training module uses the stochastic gradient descent method during the training process to continuously reduce the loss function value calculated by the loss function layer, and optimizes the training network model until convergence, including:

[0136] The stochastic gradient descent algorithm was used to optimize the model parameters of the training network model. A total of 1000 iterations were performed, with an initial learning rate of 0.002 and a decrease in the learning rate every 500 rounds, so that the loss function value calculated by the loss function layer continued to decrease, so that the training network model reached the optimal value.

[0137] It can be seen that through this optional implementation, the training classification model can be fully optimized until convergence, so that the classification convolutional neural network model can be trained more accurately and the section information in the ultrasound image data can be classified more accurately and efficiently.

[0138] As an optional implementation, the recognition module 203 inputs the target ultrasound image data into the trained classification convolutional neural network model to obtain the specific method of the section type classification result, including:

[0139] The target ultrasound image data is transformed in different ways to obtain a plurality of transformed image data;

[0140] Inputting multiple transformed image data into the trained classification convolutional neural network model respectively to obtain multiple section type classification results;

[0141] According to the multiple slice type classification results, a final slice type classification result of the target ultrasound image data is determined.

[0142] Optionally, the transformation of the target ultrasonic image data should be a transformation while retaining the original size of the target ultrasonic image data to prevent the transformed image data from losing too many original details. Optionally, the transformation can be to perform denoising and smoothing processing on the target ultrasonic image, and then based on the wavelet threshold denoising method, perform wavelet decomposition, quantize the decomposition layer coefficients, wavelet inverse transformation and other operations on the signal corresponding to the processed image through a preset number of decomposition layers and wavelet basis functions to obtain the transformed image data. This transformation method can effectively suppress speckle noise and enhance image data to more accurately extract the features of the ultrasonic image data.

[0143] Optionally, the transformation of the target ultrasound image data may also be to calculate the grayscale histogram of the target ultrasound image and crop it to obtain a new grayscale histogram, and then map each grayscale level in the new grayscale histogram to a new grayscale level through the grayscale cumulative frequency and grayscale mapping function to obtain the transformed image data.

[0144] Optionally, for multiple slice type classification results of multiple transformed image data, a corresponding weight percentage can be assigned to each transformed image data, and the multiple slice type classification results are weighted and summed according to the weight percentages of their respective corresponding transformed image data to determine the slice type classification result of the final target ultrasound image data. Generally speaking, the sum of the weight percentages of all transformed image data should be 1. For example, the classification convolutional neural network model can output a predicted slice type and a corresponding prediction score for each transformed image data, then the prediction scores corresponding to all transformed image data can be weightedly calculated, and then all weighted prediction scores corresponding to different predicted slice types are added, and finally the slice type with the highest added prediction score is determined as the final slice type classification result of the target ultrasound image data. Specifically, the weight percentage of each transformed image data can be proportional to the similarity between the transformed image data and the target ultrasound image data, and the weight percentage can be determined after calculating the similarity between each transformed image data and the target ultrasound image data.

[0145] It can be seen that through this optional implementation method, the section information in the target ultrasound image data can be accurately classified based on the trained neural network model, which is helpful to help doctors quickly classify and judge the sections of biological structures, improve inspection efficiency, reduce misdiagnosis and missed diagnosis rates, and at the same time promote the effective use of medical resources, which has great practical value.

[0146] Embodiment 3

[0147] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of another fetal ultrasound key section recognition device based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 3 As shown, the device may include:

[0148] A memory 301 storing executable program codes;

[0149] a processor 302 coupled to the memory 301;

[0150] The processor 302 calls the executable program code stored in the memory 301 to execute part or all of the steps in the method for identifying key sections of fetal ultrasound based on a convolutional neural network disclosed in the first embodiment of the present invention.

[0151] Embodiment 4

[0152] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the fetal ultrasound key section recognition method based on convolutional neural network disclosed in Example 1 of the present invention.

[0153] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0154] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0155] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be repeated here.

[0156] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0157] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0158] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0159] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0160] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0161] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0164] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0165] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0166] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0167] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0168] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0169] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] Finally, it should be noted that the ultrasonic volume imaging positioning method and device for a biological structure disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fetal ultrasound key section recognition method based on convolutional neural network, characterized in that: The method comprises: Acquiring original ultrasound image data including cross-sectional images of fetal structures; Determining target ultrasonic image data according to the original ultrasonic image data; Inputting the target ultrasound image data into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types; The step of inputting the target ultrasound image data into a trained classification convolutional neural network model to obtain a section type classification result includes: The target ultrasonic image data is transformed in different ways to obtain a plurality of transformed image data; Inputting the plurality of transformed image data into the trained classification convolutional neural network model respectively to obtain a plurality of section type classification results; Determining a final section type classification result of the target ultrasound image data according to the multiple section type classification results; Each of the transformed image data has a corresponding weight percentage, and the section type classification result corresponding to each of the transformed image data includes the predicted section type of the transformed image data and the corresponding predicted score; Wherein, determining the final section type classification result of the target ultrasound image data according to the multiple section type classification results includes: For any of the transformed image data, calculating a weighted score between the weight percentage of the transformed image data and the prediction score of the transformed image data; According to the weighted score corresponding to each predicted section type, all weighted scores of the same section type are added together to obtain the weighted scores of all predicted section types of the same section type; According to the weighted scores of the predicted section types with the same section type, the predicted section types with the same section type are screened, and the predicted section type with the highest weighted score is used as the final section type classification result of the target ultrasound image data.

2. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 1, characterized in that: Determining target ultrasonic image data according to the original ultrasonic image data includes: Determining sensitive information in the original ultrasound image data; The sensitive information is removed to obtain target ultrasound image data.

3. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 2, characterized in that: The determining of sensitive information in the original ultrasound image data includes: Determining a purpose of capturing the original ultrasound image data; According to the photographing purpose, determining irrelevant data in the original ultrasonic image data that is irrelevant to the photographing purpose; determining the irrelevant data as sensitive information in the original ultrasound image data; and / or, Determining the privacy data in the original ultrasound image data; the privacy data includes at least one of private disease information, private body part information and patient personal information; The privacy data is determined as sensitive information in the original ultrasound image data.

4. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 1, characterized in that: The classification convolutional neural network model is a convolutional neural network classification model developed and tested under the deep learning framework Pytorch.

5. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 1, characterized in that: The classification convolutional neural network model is trained according to the following steps: Determine a training data set including a plurality of training ultrasound image data labeled with section types and a training classification model; the training classification model includes the classification convolutional neural network model and a loss function calculation layer; The training ultrasound image data and the section type annotations in the training data set are input into the training classification model in the form of paired labels for training, and the stochastic gradient descent method is used during the training process to continuously reduce the loss function value calculated by the loss function calculation layer, and the classification convolutional neural network model is optimized until convergence to obtain the trained classification convolutional neural network model.

6. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 5, characterized in that: The loss function of the loss function calculation layer is a cross entropy cost function.

7. The method for identifying key sections of fetal ultrasound based on convolutional neural network according to claim 5, characterized in that: The method of using the stochastic gradient descent method in the training process to continuously reduce the loss function value calculated by the loss function calculation layer and optimizing the classification convolutional neural network model until convergence includes: The model parameters of the classification convolutional neural network model were optimized using the stochastic gradient descent algorithm, with a total of 1000 iterations, an initial learning rate of 0.002, and a decrease in the learning rate every 500 rounds so that the loss function value calculated by the loss function calculation layer continued to decrease, so that the classification convolutional neural network model reached the optimal value.

8. A fetal ultrasound key section recognition device based on convolutional neural network, characterized in that: The device comprises: An acquisition module, used for acquiring original ultrasound image data including fetal structural section images; A determination module, configured to determine target ultrasonic image data according to the original ultrasonic image data; A recognition module, used for inputting the target ultrasound image data into a trained classification convolutional neural network model to obtain a section type classification result; the classification convolutional neural network model is trained by a training data set including a plurality of training ultrasound image data labeled with section types; The recognition module inputs the target ultrasound image data into the trained classification convolutional neural network model, and the specific method of obtaining the section type classification result includes: The target ultrasonic image data is transformed in different ways to obtain a plurality of transformed image data; Inputting the plurality of transformed image data into the trained classification convolutional neural network model respectively to obtain a plurality of section type classification results; Determining a final section type classification result of the target ultrasound image data according to the multiple section type classification results; Each of the transformed image data has a corresponding weight percentage, and the section type classification result corresponding to each of the transformed image data includes the predicted section type of the transformed image data and the corresponding predicted score; The specific manner in which the recognition module determines the final section type classification result of the target ultrasound image data according to the multiple section type classification results includes: For any of the transformed image data, calculating a weighted score between the weight percentage of the transformed image data and the prediction score of the transformed image data; According to the weighted score corresponding to each predicted section type, all weighted scores of the same section type are added together to obtain the weighted scores of all predicted section types of the same section type; According to the weighted scores of the predicted section types with the same section type, the predicted section types with the same section type are screened, and the predicted section type with the highest weighted score is used as the final section type classification result of the target ultrasound image data.

9. A fetal ultrasound key section recognition device based on convolutional neural network, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the fetal ultrasound key section recognition method based on convolutional neural network as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the fetal ultrasound key section recognition method based on convolutional neural network as described in any one of claims 1-7.

Images