Ultrasonic image recognition method and related device

Abstract

The present application discloses an ultrasonic image recognition method, comprising: acquiring an ultrasonic image to be recognized; performing tissue recognition on the ultrasonic image to be recognized according to the tissue detection frame recognition information interacted during the recognition process using a tissue recognition model to obtain a recognition result; wherein the recognition result includes result data with a tissue structure segmentation boundary marked; the tissue recognition model is obtained by training according to training data, and the training data is training data marked with a tissue detection frame and a tissue structure segmentation boundary. The tissue recognition model is used to perform tissue recognition on the ultrasonic image to be recognized to obtain a recognition result with a tissue structure segmentation boundary, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundary and improve the speed of ultrasonic examination. The present application also discloses an ultrasonic image recognition device, a computing device, and a computer-readable storage medium, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an ultrasonic image recognition method, an ultrasonic image recognition device, a computing device, and a computer-readable storage medium. Background Art

[0002] With the continuous development of medical detection technology, ultrasound technology is currently commonly used in the medical field for the inspection and diagnosis of diseases. It mainly uses the waves generated by ultrasound to propagate in the human body, and displays ultrasound images containing the reflection and attenuation rules of various organs and tissues in the body to reflect the situation in the body through the oscilloscope screen. Among them, ultrasound has good directionality. When it propagates in the human body, it encounters tissues and organs with different densities, that is, reflection, refraction and absorption phenomena occur. According to the distance, strength and number of echoes of the ultrasound image displayed on the oscilloscope screen, as well as whether the attenuation is obvious, the activity functions of certain organs in the body can be displayed, and it can accurately identify whether the tissues and organs contain liquid or gas, or are substantial tissues. It can be seen that the tissue conditions in the body can be easily displayed through ultrasound images for inspection.

[0003] In the prior art, medical staff generally manually click or press buttons to screen out ultrasound cross-sectional images that meet their needs based on their clinical experience and knowledge when conducting ultrasound examinations, and they are required to identify the tissues and organs in the ultrasound cross-sectional images themselves for marking or measurement. This method requires medical staff to make their own judgments and to manually confirm or adjust repeatedly, which is inefficient and may introduce subjective errors, reducing the accuracy of the examination process.

[0004] Therefore, how to improve the efficiency of finding tissue structures in ultrasound images is a key issue that technicians in this field are concerned about. Summary of the invention

[0005] The purpose of the present application is to provide an ultrasound image recognition method, an ultrasound image recognition device, a computing device and a computer-readable storage medium, which perform tissue recognition on an ultrasound image to be recognized through a tissue recognition model to obtain a recognition result with a tissue detection frame and a structure segmentation boundary, so that medical personnel can quickly determine the position of the tissue and its structure through the tissue detection frame and the structure segmentation boundary, thereby improving the speed of ultrasound examination.

[0006] In order to solve the above technical problems, the present application provides an ultrasonic image recognition method, comprising:

[0007] Acquiring an ultrasound image to be identified;

[0008] Using a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the recognition information of the tissue detection frame interacted during the recognition process, to obtain a recognition result;

[0009] The recognition result includes result data with tissue structure segmentation boundaries marked; the tissue recognition model is obtained by training according to training data, and the training data is training data marked with tissue detection boxes and tissue structure segmentation boundaries.

[0010] Optionally, using a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the recognition information of the tissue detection frame exchanged during the recognition process to obtain a recognition result, including:

[0011] Using the feature extraction network of the tissue recognition model to extract features from the ultrasound image to be recognized, to obtain target features;

[0012] The target feature is detected and identified using the detection network of the tissue identification model to obtain tissue detection frame identification information; wherein the tissue detection frame identification information includes a tissue detection frame that defines the scope of the corresponding tissue.

[0013] The target feature is segmented and identified using the segmentation network of the tissue identification model based on the tissue detection frame identification information to obtain the identification result.

[0014] Optionally, the segmentation network using the tissue recognition model performs segmentation and recognition on the target feature based on the tissue detection frame recognition information to obtain the recognition result, including:

[0015] The segmentation network is used to segment and identify the target feature within the tissue detection frame range corresponding to the tissue detection frame identification information to obtain the identification result.

[0016] Optionally, the step of extracting features from the ultrasound image to be identified using the feature extraction network of the tissue recognition model to obtain target features includes:

[0017] The feature extraction network of the tissue recognition model is used to extract the underlying features of the ultrasound image to be recognized, so as to obtain a plurality of underlying features; the plurality of underlying features are combined to obtain a high-level semantic feature as the target feature.

[0018] Optionally, after detecting and identifying the target feature using the detection network of the tissue recognition model to obtain tissue detection frame recognition information, the method further includes:

[0019] The target feature is classified and identified using the classification network of the tissue identification model based on the tissue detection frame identification information to obtain a tissue classification and identification result.

[0020] Optionally, after the step of using the tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information exchanged during the recognition process to obtain the recognition result, the step further includes:

[0021] According to the recognition result, the recognized tissue is automatically measured and measurement data is output; wherein the measurement data includes measurement values ​​and / or measurement lines.

[0022] Optionally, the tissue recognition model is a deep learning neural network model; wherein the detection network includes a convolutional layer.

[0023] Optionally, before the step of training the tissue recognition model according to the training data, the method further includes:

[0024] Performing image transformation processing on the original training image to obtain the training data; wherein the image transformation processing includes any one or any combination of affine transformation, image splicing, mosaic blur, and Gaussian blur.

[0025] The present application also provides an ultrasonic image recognition device, comprising:

[0026] An image acquisition module, used for acquiring an ultrasound image to be identified;

[0027] An image recognition module is used to use a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information interacted during the recognition process to obtain a recognition result; wherein the recognition result includes result data marked with tissue structure segmentation boundaries; the tissue recognition model is obtained by training based on training data, and the training data is training data marked with tissue detection frames and tissue structure segmentation boundaries.

[0028] The present application also provides a computing device, including:

[0029] Memory for storing computer programs;

[0030] A processor is used to implement the steps of the ultrasound image recognition method as described above when executing the computer program.

[0031] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the ultrasound image recognition method as described above are implemented.

[0032] An ultrasound image recognition method provided in the present application includes: acquiring an ultrasound image to be recognized; performing tissue recognition on the ultrasound image to be recognized according to tissue detection frame recognition information exchanged during the recognition process using a tissue recognition model to obtain a recognition result; wherein the recognition result includes result data with tissue structure segmentation boundaries marked; the tissue recognition model is obtained by training based on training data, and the training data is training data marked with tissue detection frames and tissue structure segmentation boundaries.

[0033] The tissue recognition model is used to perform tissue recognition on the ultrasound image to be recognized to obtain the recognition result, thereby realizing automatic recognition. Since the training data used by the tissue recognition model is training data marked with tissue detection boxes and tissue structure segmentation boundaries, the obtained recognition result contains tissue structure segmentation boundaries, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundaries, thereby improving the speed of ultrasound examination, avoiding subjective errors, and improving the accuracy of ultrasound examination.

[0034] The present application also provides an ultrasonic image recognition device, a computing device, and a computer-readable storage medium, which have the above beneficial effects and are not elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0036] Figure 1 A flowchart of an ultrasonic image recognition method provided in an embodiment of the present application;

[0037] Figure 2 A flowchart of a recognition process of the ultrasonic image recognition method provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the structure of an ultrasonic image recognition device provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The core of the present application is to provide an ultrasound image recognition method, an ultrasound image recognition device, a computing device and a computer-readable storage medium, which perform tissue recognition on an ultrasound image to be recognized through a tissue recognition model to obtain a recognition result with a tissue structure segmentation boundary, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundary and improve the speed of ultrasound examination.

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] In the related technology, medical staff generally manually examine the ultrasound images directly to screen different tissue conditions. However, the manual examination process first requires medical staff to determine the tissue and structure in the ultrasound image. That is, medical staff need to determine the location of the tissue and then determine the structure corresponding to the tissue in the tissue, which not only takes a lot of time, but also may introduce subjective errors and reduce the accuracy of the examination process.

[0043] Therefore, the present application provides an ultrasound image recognition method, which performs tissue recognition on an ultrasound image to be recognized through a tissue recognition model to obtain a recognition result, thereby realizing automatic recognition. Since the training data used by the tissue recognition model is training data marked with a tissue detection frame and a tissue structure segmentation boundary, the obtained recognition result contains a tissue structure segmentation boundary, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundary, thereby improving the speed of ultrasound examination, avoiding subjective errors, and improving the accuracy of ultrasound examination.

[0044] An ultrasonic image recognition method provided by the present application is described below through an embodiment.

[0045] Please refer to Figure 1 , Figure 1 A flowchart of an ultrasonic image recognition method provided in an embodiment of the present application.

[0046] In this embodiment, the method may include:

[0047] S101, obtaining an ultrasound image to be identified;

[0048] This step aims to obtain the ultrasound image to be identified.

[0049] Among them, the ultrasonic image to be identified can be an ultrasonic image to be identified obtained in real time from an ultrasonic system, so that the ultrasonic image to be identified can be identified in real time during the operation and the corresponding identification result can be displayed, providing medical staff with fast and convenient auxiliary functions.

[0050] It can also be the stored ultrasound images to be identified obtained from the database of the ultrasound system, so as to perform batch identification on the ultrasound images to be identified stored in the database, thereby improving the efficiency and speed of medical staff in performing disease examinations with reference to ultrasound images.

[0051] It can be seen that the method of obtaining the ultrasound image to be identified in this step is not unique and is not specifically limited here.

[0052] S102, using a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information exchanged during the recognition process, to obtain a recognition result; wherein the recognition result includes result data with tissue structure segmentation boundaries marked; the tissue recognition model is obtained by training based on training data, and the training data is training data marked with tissue detection frames and tissue structure segmentation boundaries.

[0053] It can be seen that this step aims to use the tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the recognition information of the tissue detection frame interacted during the recognition process to obtain the recognition result.

[0054] The recognition result includes result data with tissue structure segmentation boundaries marked. That is, in the process of recognizing the ultrasound image to be recognized, the tissue structure segmentation boundaries are marked for the tissue structure of interest in the ultrasound image to be recognized. In the recognition result, the position of the tissue structure is directly determined by the tissue structure segmentation boundaries.

[0055] In other words, medical staff can directly check the tissue structure segmentation boundary on the ultrasound image to be identified to determine the tissue structure, and further perform corresponding diagnostic analysis operations, avoiding manual identification on the ultrasound image by medical staff, improving the speed of ultrasound examination, and also avoiding subjective errors. At the same time, it can avoid the tedious operations such as repeated freezing and adjustment by medical staff in order to identify the tissue structure that meets the requirements when scanning and obtaining ultrasound images, thereby avoiding misdiagnosis and missed diagnosis caused by personal reasons such as fatigue of doctors.

[0056] The structure of the tissue recognition model may include two recognition networks, namely a detection network and a segmentation network. The detection network is used to identify the range of tissue in the ultrasound image and frame it with a tissue detection frame, and the segmentation network is used to identify the structure in the ultrasound image based on the tissue detection frame information and mark it with the tissue structure segmentation boundary. The detection network may interact with the tissue detection frame recognition information with the segmentation network so that the segmentation network can perform recognition based on the tissue detection frame recognition information.

[0057] Therefore, the identification process in this step is to use the tissue identification model to perform tissue identification according to the tissue detection frame identification information exchanged during the identification process. That is to say, in the identification process, the tissue detection frame identification information obtained by identification is exchanged between the detection network and the segmentation network so that other networks receive the tissue detection frame information and complete tissue identification. Furthermore, the detection network and the segmentation network in the tissue identification model can be connected to each other so that information can be exchanged during the processing process, and a feedback mechanism between the detection network and the segmentation network is realized to improve the accuracy of the detection network and the segmentation network identification.

[0058] Then, the training data of the corresponding tissue recognition model is the training data labeled with the tissue detection frame and the tissue structure segmentation boundary. That is, the tissue recognition model is obtained by using the training data labeled with the tissue detection frame and the tissue structure segmentation boundary at the same time or separately for deep learning model training.

[0059] In order to further improve the accuracy and recognition effect of the tissue recognition model, the structure of the tissue recognition model can be that the feature extraction network is connected to the detection network and the segmentation network respectively. Among them, the feature extraction network is used to extract high-level semantic features from the ultrasound image in order to improve the data processing effect of the detection network and the segmentation network. Furthermore, the feature extraction network can be specifically used to extract low-level features such as edges, contours, grayscale, contrast, etc., and then all low-level features are systematically combined to obtain high-level semantic features.

[0060] It should be noted that in a specific application environment, the detection network introduced above is used to locate and intercept tissues and organs that medical staff are concerned about during pathological examinations, and the segmentation network is used to segment specific structures in the tissue from the diagnosed tissues and organs.

[0061] Furthermore, in an actual application environment, after determining each tissue structure in an ultrasound image, it is also necessary to measure the size or important measurement items of the corresponding tissue structure in order to diagnose each tissue structure. In order to avoid manual measurement by medical staff and improve the degree of automation, this embodiment may further include:

[0062] According to the recognition result, the recognized tissue is automatically measured and the measurement data is output; wherein the measurement data includes measurement values ​​and / or measurement lines.

[0063] It can be seen that, based on the identification results, the identified tissue can be automatically measured to obtain and output the measurement data. Instead of manual measurement by medical staff, the efficiency of measurement is improved. The measurement data includes measurement values ​​and / or measurement lines. When both the measurement values ​​and the measurement lines are presented, the measurement values ​​and the measurement lines correspond to each other. For example, the measurement value is the length value of the corresponding measurement line. The automatic measurement method can be automatic measurement through a preset script, or it can be measurement based on a trained measurement model, or it can be measurement using any automatic measurement method provided by the prior art, which is not specifically limited here.

[0064] Furthermore, in order to increase the number of training samples so as to enhance the recognition effect of the tissue recognition model, this embodiment may also include:

[0065] Perform image transformation processing on the original training image to obtain training data; wherein the image transformation processing includes any one or any combination of affine transformation, image splicing, mosaic blur, and Gaussian blur.

[0066] It can be seen that the main purpose of this optional solution is to increase the number of training samples and perform image processing on the original training images in order to increase the amount of training data. The image processing methods include but are not limited to any one or any combination of affine transformation, image splicing, mosaic blur, and Gaussian blur. The affine transformation of the image may include transformation operations such as translation, scaling, flipping, and rotation of the image.

[0067] Furthermore, in order to improve the accuracy of tissue identification in a disease examination environment, this embodiment may also include:

[0068] Ultrasonic image data of multiple time periods and / or multiple regions are acquired as training data.

[0069] It can be seen that in order to include richer information in the training data, this optional solution obtains ultrasound image data of multiple time periods and / or multiple regions as training data. Among them, multiple time periods refer to multiple time periods from the beginning to the end of a certain disease. For example, when the training data obtained is obstetric data, training data of multiple gestational weeks is obtained.

[0070] Furthermore, in the recognition result corresponding to the ultrasound image to be recognized, multiple structures in the ultrasound image to be recognized are generally identified by the tissue structure segmentation boundary. In some application scenarios, the types of the multiple structures are different, and medical staff are required to identify different structures, which reduces the inspection efficiency. Therefore, in this embodiment, the tissue structure classification information corresponding to each type of structure can also be identified.

[0071] Therefore, optionally, this embodiment may further include:

[0072] Step 1, when performing tissue recognition on the ultrasound image to be recognized according to the tissue recognition model, obtaining a tissue classification recognition result of the ultrasound image to be recognized in the tissue recognition process;

[0073] Step 2: according to the tissue classification recognition result, corresponding tissue structure classification information is added to the tissue structure segmentation boundary in the recognition result.

[0074] Because, in the process of identifying the structure in the ultrasound image to be identified, the tissue recognition model first classifies the tissue structure to be identified, and then identifies the tissue structure segmentation boundary. Therefore, in this optional solution, the tissue classification recognition result in the classification process can be directly obtained, and then, according to the tissue classification recognition result, the corresponding tissue structure classification information is added to the tissue structure segmentation boundary in the recognition result. For example, in the obstetric examination, facing the recognition result of the fetal ultrasound image, the tissue structure segmentation boundary respectively segments the fetal gastric bubble, nuchal transparent layer (NT) and placenta and other tissues or organs. Furthermore, the corresponding tissue structure classification information can be added to each tissue structure segmentation boundary through this optional solution, that is, the tissue structure classification information such as "placenta", "nuchal transparent layer", "gastric bubble" can be added. It is convenient for medical staff to quickly determine the type of each tissue structure.

[0075] In addition, the tissue recognition model of this embodiment may also include a regression branch network. Specifically, the regression branch network may include a convolutional layer and a fully connected layer to achieve the positioning of the recognized tissue structure.

[0076] In summary, this embodiment uses a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized to obtain a recognition result, thereby achieving automatic recognition. Since the training data used by the tissue recognition model is training data labeled with a tissue detection frame and a tissue structure segmentation boundary, the obtained recognition result contains a tissue structure segmentation boundary, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundary, thereby improving the speed of ultrasound examination, avoiding subjective errors, and improving the accuracy of ultrasound examination. The recognition method of this embodiment can be applied to obstetric ultrasound image screening.

[0077] The ultrasonic image recognition method provided by the present application is further described below through another embodiment.

[0078] In this embodiment, the tissue recognition model includes a feature extraction network, a detection network and a segmentation network; wherein the segmentation network can adopt a fully convolutional network. wherein the feature extraction network is connected to the detection network and the segmentation network respectively, and the detection network is connected to the segmentation network. wherein the feature extraction network is mainly used to extract features from the input data to obtain high-level semantic features in order to improve the effect of feature extraction. based on this, the detection network and the segmentation network process the received high-level semantic features respectively, and realize the interaction of information data between the detection network and the segmentation network during the processing. that is, the segmentation network uses the detection frame information of the detection network during the detection process to adjust the segmentation process, which can narrow the recognition range during the segmentation process, thereby achieving faster and more accurate segmentation recognition.

[0079] The feature extraction network includes convolutional layers and feature pyramid networks. The purpose of the convolutional layer is to extract different features of the input image data. The first convolutional layer may only extract some low-level features such as edges, lines, and corners. When more layers of networks are set, more complex features can be iteratively extracted from low-level features. The feature pyramid network uses the high resolution of low-level features and the high semantic information of high-level features to achieve the purpose of prediction by fusing the features of these different layers, thereby improving the effect of feature extraction.

[0080] In specific implementation, the feature extraction network can use ResNet50 and FPN (i.e., the feature pyramid network mentioned above) as the backbone network to obtain more accurate target features. Among them, Resnet is the abbreviation of Residual Network. This series of networks is widely used in fields such as target classification and as part of the backbone classical neural network for computer vision tasks. Typical networks include resnet50, resnet101, etc. Therefore, ResNet50 can be used to implement feature extraction.

[0081] Please refer to Figure 2 , Figure 2 A flowchart of a recognition process of the ultrasound image recognition method provided in an embodiment of the present application.

[0082] In this embodiment, the identification process may include:

[0083] S201, extracting features of the ultrasound image to be identified using a feature extraction network of a tissue recognition model to obtain target features;

[0084] S202, using a detection network of a tissue recognition model to detect and recognize target features, and obtain tissue detection frame recognition information; wherein the tissue detection frame recognition information includes a tissue detection frame that defines a range of a corresponding tissue;

[0085] S203, using the segmentation network of the tissue recognition model to segment and recognize the target features based on the tissue detection frame recognition information to obtain a recognition result.

[0086] It can be seen that, through S203, in the recognition process, the segmentation network of the tissue recognition model is used to segment and recognize the target features based on the tissue detection frame recognition information.

[0087] Furthermore, the detection network sends the organization detection box identification information to the segmentation network during the recognition process. Including information such as position and size, the segmentation network can narrow the scope of segmentation recognition in the high-level semantic features to be recognized, and then perform segmentation recognition processing. Not only does it reduce the amount of data processed by the segmentation network, it can achieve segmentation recognition more quickly, but it also eliminates erroneous features and improves the accuracy of the segmentation network.

[0088] The method of realizing data interaction between the detection network and the segmentation network can be reflected in the structural design of the tissue recognition model. The structural design of data interaction can realize data interaction by adopting the tissue detection frame information transmission structure, or by adopting any design method provided by the prior art, which is not specifically limited here.

[0089] The structure of the feature extraction network may adopt any feature extraction network provided by the prior art, and is not specifically limited here.

[0090] Further, in order to improve the efficiency and accuracy of segmentation and recognition, S203 in this embodiment may include:

[0091] The segmentation network is used to segment and identify the target features within the tissue detection frame range corresponding to the tissue detection frame recognition information to obtain the recognition result.

[0092] It can be seen that in this optional solution, the detection frame is organized to narrow the scope of segmentation and recognition of the segmentation network, reduce the amount of data, eliminate the error range, improve the efficiency of segmentation and recognition, and also improve the accuracy of segmentation and recognition.

[0093] Further, in order to improve the effect of feature extraction, S202 in this embodiment may include:

[0094] The feature extraction network of the tissue recognition model is used to extract the underlying features of the ultrasound image to be recognized, and multiple underlying features are obtained; the multiple underlying features are combined to obtain high-level semantic features as target features.

[0095] Among them, the feature extraction network is used to extract high-level semantic features from ultrasound images in order to improve the data processing effect of the detection network and the segmentation network. Furthermore, the feature extraction network can be used to extract low-level features such as edges, contours, grayscale, and contrast, and then all low-level features are combined to obtain high-level semantic features of the target features, thereby maintaining the effect of the target features.

[0096] Furthermore, in the recognition result corresponding to the ultrasound image to be recognized, multiple structures in the ultrasound image to be recognized are generally identified by the tissue structure segmentation boundary. In some application scenarios, the types of the multiple structures are different, and medical staff are required to recognize different structures, which reduces the inspection efficiency. Therefore, in order to improve the inspection efficiency, S203 in this embodiment may also include:

[0097] The classification network of the tissue recognition model is used to classify and recognize the target features based on the tissue detection frame recognition information to obtain the tissue classification and recognition results.

[0098] The classification network may include a convolutional layer and a fully connected layer. The segmentation network and the classification network share a feature extraction network, and thus the correlation between the segmentation task and the classification task may be utilized to improve the performance of the tissue recognition model and the efficiency of classification and segmentation.

[0099] Accordingly, the training process also needs to include training data labeled with classification labels in order to train the classification network.

[0100] In this optional solution, the tissue classification recognition results in the classification network can be directly obtained. For example, in the face of the recognition results of fetal ultrasound images in obstetric examinations, the tissue structure segmentation boundaries respectively segment the fetal gastric bubble, nuchal translucency (NT) and placenta and other tissues or organs. Furthermore, the classification network in this optional solution can be used to obtain the corresponding tissue classification recognition results for each tissue structure segmentation boundary, that is, to add structural classification information such as "placenta", "nuchal translucency", "gastric bubble", etc., to facilitate medical staff to quickly determine the type of each tissue structure.

[0101] In this embodiment, the above training data can be used to train the initial model corresponding to the tissue recognition model, and then the tissue recognition model serving as the target model can be obtained.

[0102] It is conceivable that, after the training data is determined, any model training method provided by the prior art can be used without specific limitation. It can be seen that by training the initial model, a tissue recognition model for recognizing the ultrasound image to be recognized can be obtained. Among them, the accuracy and recognition effect of the tissue recognition model can be improved by the information interaction between the detection network and the segmentation network.

[0103] It can be seen that this embodiment improves the detection efficiency and accuracy through information interaction between the detection network and the segmentation network, and then obtains the recognition result and realizes automatic recognition. Since the training data used by the tissue recognition model is training data marked with tissue detection boxes and tissue structure segmentation boundaries, the obtained recognition results contain tissue structure segmentation boundaries, so that medical staff can quickly determine the tissue structure through the tissue structure segmentation boundaries, thereby improving the speed of ultrasonic examination, avoiding subjective errors, and improving the accuracy of ultrasonic examination.

[0104] The ultrasonic image recognition device provided in an embodiment of the present application is introduced below. The ultrasonic image recognition device described below and the ultrasonic image recognition method described above can be referenced to each other.

[0105] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of an ultrasonic image recognition device provided in an embodiment of the present application.

[0106] In this embodiment, the device may include:

[0107] An image acquisition module 100 is used to acquire an ultrasound image to be identified;

[0108] The image recognition module 200 is used to use a tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information interacted during the recognition process to obtain a recognition result; wherein the recognition result is result data marked with a tissue detection frame and a tissue structure segmentation boundary; the tissue recognition model is obtained by training based on training data, and the training data is training data marked with a tissue detection frame and a tissue structure segmentation boundary.

[0109] Optionally, the image recognition module 200 may include:

[0110] A feature extraction unit, used for extracting features from the ultrasound image to be identified by using a feature extraction network of a tissue recognition model to obtain target features;

[0111] A tissue detection unit, used to detect and identify target features using a detection network of a tissue recognition model to obtain tissue detection frame recognition information; wherein the tissue detection frame recognition information includes a tissue detection frame that defines a range of a corresponding tissue;

[0112] The structure segmentation unit is used to use the segmentation network of the tissue recognition model to segment and recognize the target features based on the tissue detection frame recognition information to obtain the recognition result.

[0113] Optionally, the structure segmentation unit is specifically used to use a segmentation network to segment and identify target features within a tissue detection frame range corresponding to the tissue detection frame identification information to obtain a recognition result.

[0114] Optionally, the feature extraction unit is specifically used to use a feature extraction network of a tissue recognition model to extract underlying features of an ultrasound image to be recognized, thereby obtaining a plurality of underlying features; and combining the plurality of underlying features to obtain a high-level semantic feature as a target feature.

[0115] Optionally, the tissue detection unit may further include:

[0116] The classification unit is used to classify and identify the target features based on the tissue detection frame recognition information using the classification network of the tissue recognition model to obtain a tissue classification recognition result.

[0117] Optionally, the device may further include:

[0118] The automatic measurement module is used to automatically measure the identified tissue according to the identification result and output the measurement data; wherein the measurement data includes the measurement value and / or the measurement line.

[0119] Optionally, the tissue recognition model is a deep learning neural network model; wherein the detection network includes a convolutional layer.

[0120] Optionally, the feature extraction network may include a convolutional layer and a feature pyramid network.

[0121] Optionally, the device may further include:

[0122] The image processing module is used to perform image transformation processing on the original training image to obtain training data; wherein the image transformation processing includes any one or any combination of affine transformation, image splicing, mosaic blur, and Gaussian blur.

[0123] Please refer to Figure 4 , Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0124] The present application also provides a computing device, including:

[0125] A memory 11, used for storing computer programs;

[0126] The processor 12 is used to implement the steps of the ultrasound image recognition method described in the above embodiment when executing the computer program.

[0127] Specifically, the computing device may be an ultrasonic diagnostic device.

[0128] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the ultrasound image recognition method described in the above embodiment are implemented.

[0129] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0130] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0131] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0132] The above is a detailed introduction to an ultrasonic image recognition method, an ultrasonic image recognition device, a computing device and a computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. An ultrasonic image recognition method, characterized in that: include: Acquiring an ultrasound image to be identified; The tissue recognition model is used to perform tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information interacted during the recognition process to obtain a recognition result; wherein the structure of the tissue recognition model includes a feature extraction network, a detection network and a segmentation network, the feature extraction network is used to extract underlying features including edges, contours, grayscale and contrast, and then all underlying features are combined to obtain high-level semantic features; the feature extraction network is connected to the detection network and the segmentation network respectively; the detection network is connected to the segmentation network, and the segmentation network uses the detection frame information of the detection network during the detection process to adjust the segmentation process to narrow the recognition range during the segmentation process; The recognition result includes result data with tissue structure segmentation boundaries marked; the tissue recognition model is obtained by training according to training data, and the training data is ultrasound image data of multiple time periods and / or multiple regions, and marked with tissue detection frames and tissue structure segmentation boundaries; The identified tissue is automatically measured using a trained measurement model according to the identification result, and measurement data is output; wherein the measurement data includes measurement values ​​and / or measurement lines.

2. The ultrasonic image recognition method according to claim 1, characterized in that: Using the tissue recognition model to perform tissue recognition on the ultrasound image to be recognized according to the recognition information of the tissue detection frame interacted during the recognition process, and obtaining a recognition result, including: Using the feature extraction network of the tissue recognition model to extract features from the ultrasound image to be recognized, to obtain target features; Detecting and identifying the target feature using the detection network of the tissue recognition model to obtain the tissue detection frame recognition information; wherein the tissue detection frame recognition information includes a tissue detection frame that defines the scope of the corresponding tissue; The target feature is segmented and identified using the segmentation network of the tissue identification model based on the tissue detection frame identification information to obtain the identification result.

3. The ultrasonic image recognition method according to claim 2, characterized in that: The segmentation network using the tissue recognition model performs segmentation and recognition on the target feature based on the tissue detection frame recognition information to obtain the recognition result, including: The segmentation network is used to segment and identify the target feature within the tissue detection frame range corresponding to the tissue detection frame identification information to obtain the identification result.

4. The ultrasonic image recognition method according to claim 2, characterized in that: After the detection network of the tissue recognition model is used to detect and recognize the target feature to obtain tissue detection frame recognition information, the method further includes: The target feature is classified and identified using the classification network of the tissue identification model based on the tissue detection frame identification information to obtain a tissue classification and identification result.

5. The ultrasonic image recognition method according to claim 2, characterized in that: The tissue recognition model is a deep learning neural network model; wherein the detection network includes a convolutional layer.

6. The ultrasonic image recognition method according to claim 1, characterized in that: Before the step of obtaining the tissue recognition model through training according to the training data, the method further includes: Performing image transformation processing on the original training image to obtain the training data; wherein the image transformation processing includes any one or any combination of affine transformation, image splicing, mosaic blur, and Gaussian blur.

7. An ultrasonic image recognition device, characterized in that: include: An image acquisition module, used for acquiring an ultrasound image to be identified; An image recognition module, used for performing tissue recognition on the ultrasound image to be recognized according to the tissue detection frame recognition information exchanged during the recognition process using a tissue recognition model to obtain a recognition result; wherein the structure of the tissue recognition model includes a feature extraction network, a detection network and a segmentation network, wherein the feature extraction network is used to extract underlying features including edges, contours, grayscale and contrast, and then all underlying features are combined to obtain high-level semantic features; the feature extraction network is connected to the detection network and the segmentation network respectively; the detection network is connected to the segmentation network, and the segmentation network uses the detection frame information of the detection network during the detection process to adjust the segmentation process to reduce the recognition range during the segmentation process; The recognition result includes result data with tissue structure segmentation boundaries marked; the tissue recognition model is obtained by training according to training data, and the training data is ultrasound image data of multiple time periods and / or multiple regions, and marked with tissue detection frames and tissue structure segmentation boundaries; The automatic measurement module is used to automatically measure the identified tissue using a trained measurement model according to the identification result and output measurement data; wherein the measurement data includes measurement values ​​and / or measurement lines.

8. A computing device, characterized in that include: Memory for storing computer programs; A processor, configured to implement the steps of the ultrasound image recognition method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the ultrasound image recognition method according to any one of claims 1 to 6 are implemented.

Images