Ultrasonic image processing method and device, electronic device and storage medium

By using ultrasound image processing methods for multiple pregnancy identification and measurement, the problem of time-consuming and labor-intensive multiple pregnancy detection in existing technologies has been solved, realizing automated identification and measurement in multiple pregnancy scenarios, and improving examination efficiency and accuracy.

CN115919367BActive Publication Date: 2024-11-12SONOSCAPE MEDICAL (WUHAN) CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211586750.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-09
Publication Date
2024-11-12
Estimated Expiration
2042-12-09

AI Technical Summary

Technical Problem

Existing ultrasound image processing technology cannot effectively identify multiple pregnancy scenarios, resulting in time-consuming and inaccurate prenatal testing, and is not applicable to monitoring fetal growth and development in multiple pregnancies.

Method used

Using ultrasound image processing methods, a multiple pregnancy identification model is used to identify and label the initial ultrasound image, obtain the ultrasound image of the fetus to be tested, perform standard section recognition and image segmentation, measure relevant items, including crown-rump length, placental thickness, etc., calculate the differences in measurement results and output prompt information.

Benefits of technology

It enables automated identification and measurement in multiple pregnancy scenarios, reducing user workload, improving examination efficiency and accuracy, and providing timely alerts for abnormalities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115919367B_ABST
    Figure CN115919367B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides an ultrasound image processing method and device, an electronic device and a storage medium. The method includes: obtaining an initial ultrasound image; performing multiple fetus identification on the initial ultrasound image to obtain a multiple fetus identification result; marking multiple fetuses in the initial ultrasound image based on the multiple fetus identification result; obtaining an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses; and measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested. The technical solution can assist users in identifying and measuring multiple fetus scenarios, greatly reducing the workload of users, thereby improving the inspection efficiency and accuracy of users.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an ultrasonic image processing method, an ultrasonic image processing device, an electronic device and a storage medium. Background Art

[0002] Multiple pregnancies are high-risk pregnancies, which increase the morbidity and mortality of perinatal-related diseases such as hypertension, premature birth, and dystocia for both the mother and the fetus. In recent years, as the incidence of multiple pregnancies has increased year by year, the series of clinical problems brought about have attracted the attention of more and more obstetricians. Therefore, strengthening prenatal testing, early diagnosis and early treatment of the fetus can reduce maternal and infant-related complications. Prenatal ultrasound examinations can determine the type of multiple pregnancies and the condition of the amniotic membrane, which is helpful for monitoring fetal growth and development.

[0003] In the existing technology, doctors mainly use the method of checking the number of chorionic sacs and amniotic sacs by human eyes during ultrasound examination to determine whether it is a multiple pregnancy, which is time-consuming. In addition, the existing ultrasound image automatic recognition and measurement technology is only applicable to single-birth scenarios and cannot be applied to multiple-birth scenarios.

[0004] Therefore, a new ultrasound image processing method is urgently needed to solve the above problems. Summary of the invention

[0005] In order to at least partially solve the problems existing in the prior art, an ultrasonic image processing method, an ultrasonic image processing device, an electronic device and a storage medium are provided.

[0006] According to one aspect of the present invention, there is provided an ultrasound image processing method, comprising: acquiring an initial ultrasound image; performing multiple-fetus identification on the initial ultrasound image to obtain a multiple-fetus identification result; marking multiple fetuses in the initial ultrasound image based on the multiple-fetus identification result; acquiring an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses; and measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested.

[0007] Exemplarily, measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested includes: performing standard section recognition on the ultrasound image to be tested to determine whether the ultrasound image to be tested belongs to one of at least one preset type of standard section;

[0008] In the case of a standard section of a specific preset type, the ultrasound image to be tested is segmented to segment at least one target structure from the ultrasound image to be tested, and the at least one target structure belongs to the specific

[0009] A structure related to a standard section of a predetermined type; and based on the segmentation result of at least one target structure, measuring at least one measurement item, wherein the at least one measurement item is a measurement item related to the at least one target structure.

[0010] Exemplarily, the number of at least one fetus to be tested is greater than or equal to two. After measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested, the method also includes: for a first measurement item in the at least one measurement item, calculating a first difference between the measurement results of at least one fetus to be tested on the first measurement item; when the first difference is greater than or equal to a first difference threshold, outputting a first prompt message.

[0011] Exemplarily, the first measurement item includes the head-to-hip length.

[0012] 5. Exemplarily, at least one measurement of at least one fetus to be tested based on the ultrasound image to be tested

[0013] After the items are measured, the method further includes: for a second measurement item in at least one measurement item, calculating a second difference between a measurement result of any fetus to be measured on the second measurement item and a standard measurement result of a standard fetus on the second measurement item; when the second difference is greater than or equal to a second difference threshold, outputting a second prompt information.

[0014] 0 Exemplarily, the second measurement item includes one or more of the following: biparietal diameter length, placental thickness

[0015] degree, maximum depth of amniotic fluid.

[0016] Exemplarily, after measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested, the method further includes: for a third measurement item in the at least one measurement item

[0017] Purpose: From the measurement results of at least one fetus to be measured on the third measurement item, select the largest measurement result to output.

[0018] Exemplarily, obtaining an ultrasound image to be tested that includes at least one of the marked multiple fetuses includes: based on the fetal marking result, performing target tracking on the ultrasound image sequence acquired in real time during the user's operation of the ultrasound probe to determine the trajectory of any target fetus to be tested among the at least one fetus to be tested in the ultrasound image sequence; if it is found that the trajectory of the target fetus to be tested is lost, outputting a third prompt message; after the trajectory of the target fetus to be tested is lost for a preset period of time, or when an inspection end instruction is received, determining that the ultrasound image including the target fetus to be tested in the ultrasound image sequence is the ultrasound image to be tested including the target fetus to be tested.

[0019] Exemplarily, marking multiple fetuses in an initial ultrasound image based on multiple fetus identification results includes: identifying the position of the cervical os from the initial ultrasound image; determining a first relative position relationship between the multiple fetuses and the cervical os based on the multiple fetus identification results and the identification results of the cervical os; sorting the multiple fetuses based on the first relative position relationship; and marking the multiple fetuses in the sorted order.

[0020] Exemplarily, marking multiple fetuses in the initial ultrasound image based on the multiple fetus identification result includes: receiving marking information input by a user; and marking the multiple fetuses based on the marking information.

[0021] Exemplarily, performing multiple fetus recognition on an initial ultrasound image to obtain a multiple fetus recognition result includes: performing multiple fetus recognition on the initial ultrasound image through a target detection algorithm to obtain a multiple fetus recognition result, the multiple fetus recognition result includes a target detection frame for indicating the position of each fetus. After performing multiple fetus recognition on the initial ultrasound image to obtain the multiple fetus recognition result, the method also includes: determining a second relative position relationship of at least some of the multiple fetuses based on the multiple fetus recognition result, the second relative position relationship includes an orientation relationship between two fetuses and / or a distance between target detection frames corresponding to the two fetuses; and outputting position information used to represent the second relative position relationship.

[0022] According to another aspect of the present invention, there is provided an ultrasound image processing device, comprising: a first acquisition module for acquiring an initial ultrasound image; an identification module for performing multiple-fetus identification on the initial ultrasound image to obtain a multiple-fetus identification result; a marking module for marking a plurality of fetuses in the initial ultrasound image based on the multiple-fetus identification result; a second acquisition module for acquiring an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses; and a measurement module for measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested.

[0023] According to another aspect of the present invention, an electronic device is provided, including a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the above-mentioned ultrasound image processing method.

[0024] Exemplarily, the electronic device is an ultrasound diagnostic device or an ultrasound workstation.

[0025] According to another aspect of the present invention, a storage medium is provided, storing a computer program / instruction, wherein the computer program / instruction implements the above-mentioned ultrasound image processing method when executed by a processor.

[0026] According to the ultrasonic image processing method, ultrasonic image processing device, electronic device and storage medium of the embodiments of the present invention, a multiple-pregnancy recognition result can be obtained based on the initial ultrasonic image, and at least one measurement item of at least one fetus to be tested can be measured based on the multiple-pregnancy recognition result. The method can assist the user in the recognition and measurement of multiple-pregnancy scenes, greatly reducing the workload of the user, thereby improving the user's inspection efficiency and accuracy.

[0027] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0029] Figure 1 A schematic flow chart of a method for processing an ultrasound image according to an embodiment of the present invention is shown;

[0030] Figure 2 A schematic flow chart of a method for processing an ultrasound image according to an embodiment of the present invention is shown;

[0031] Figure 3 A schematic block diagram showing an apparatus for processing ultrasound images according to an embodiment of the present invention; and

[0032] Figure 4 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0034] In order to at least partially solve the above technical problem, an embodiment of the present invention provides an ultrasound image processing method. Figure 1FIG. 1 is a schematic diagram of an ultrasound image processing method 100 according to an embodiment of the present invention. Figure 1 As shown, the ultrasound image processing method 100 may include the following steps S110, S120, S130, S140 and S150.

[0035] Step S110, acquiring an initial ultrasound image.

[0036] Exemplarily, an ultrasound scanning video (which may be referred to as "ultrasound video") for the object to be tested may be collected by an image acquisition device. The object to be tested may be a biological tissue, such as the uterine part of a human body. The ultrasound scanning video may include multiple frames, each frame may correspond to an ultrasound image, and then multiple frames of ultrasound images may be obtained according to the obtained ultrasound scanning video. For example, a 200-frame ultrasound scanning video may obtain 200 ultrasound images, and the 200 ultrasound images may be optionally sorted according to the acquisition order of the ultrasound scanning video segment. Any one or more ultrasound images may be selected from the ultrasound scanning video as the initial ultrasound image. In one embodiment, the ultrasound image located in the middle position of the 200 ultrasound images (for example, the 100th ultrasound image) may be used as the initial ultrasound image. In another embodiment, part of the ultrasound images located at the front and back ends of the ultrasound scanning video may be removed, and the remaining part of the ultrasound images may be used as the initial ultrasound image. For example, the ultrasound images from the 1st to the 50th frames and the ultrasound images from the 150th to the 200th frames are removed, and the ultrasound images from the 51st to the 149th frames are used as the initial ultrasound images.

[0037] Step S120, performing multiple-pregnancy recognition on the initial ultrasound image to obtain a multiple-pregnancy recognition result.

[0038] For any initial ultrasound image, the multiple fetus identification result may include position information indicating the positions of the multiple fetuses in the initial ultrasound image. The positions of the multiple fetuses in the initial ultrasound image may include the positions of the multiple fetuses as a whole in the initial ultrasound image and / or the positions of the multiple fetuses respectively in the initial ultrasound image. Preferably, the multiple fetus identification result includes position information related to the positions of the multiple fetuses respectively in the initial ultrasound image.

[0039] Exemplarily, the multiple-pregnancy recognition model can be used to perform multiple-pregnancy recognition on the initial ultrasound image. Exemplarily, the multiple-pregnancy recognition model can be implemented using any suitable neural network model, such as any suitable existing or future target detection network model. For example, the multiple-pregnancy recognition model may include one or more of the following: You Only Look Once (YOLO) series, Region-Convolutional Neural Networks (RCNN) series, Retina-Net and other network models. Of course, the above-mentioned target detection network model is only an example, and the multiple-pregnancy recognition model can also be implemented using any suitable existing or future image segmentation network model. For example, the multiple-pregnancy recognition model may include one or more of the following: Fully Convolutional Networks (FCN), U-type network (Unet), DeepLab (DeepLab) series, V-type network (Vnet) and other network models. It can be understood that the image segmentation network model can also identify the location of the target object (such as the fetus described herein). The target detection network model can obtain a target detection frame for indicating the location of the fetus, and the target detection frame can be any suitable shape, preferably a rectangle. The image segmentation network model can be used to obtain a mask or envelope indicating the location of the fetus.

[0040] In addition, by way of example but not limitation, the multiple fetus identification result may also include category information for indicating the category to which the multiple fetuses in the initial ultrasound image belong. For example, in the case where the initial ultrasound image contains twins, they can be divided into dichorionic twins and monochorionic twins. Dichorionic twins and monochorionic twins can be divided according to the number of chorions. Dichorionic twins are twins formed by the fertilization of two eggs respectively. When two gestational sacs are observed, each with a single embryo and fetal heart beat in the sac, it can be diagnosed as dichorionic twins. The twins are separated by a thick layer of fused chorion and two layers of amnion, one on each side. Monochorionic twins are twins formed by the fertilization of one egg. When only one gestational sac is observed, two embryos and fetal heart beats are detected in the sac, it is monochorionic twins, and the twins are separated by two thinner layers of amnion. In one embodiment, when the above-mentioned target detection network model or image segmentation network model is used for multiple fetus identification, the target detection network model or image segmentation network model can output the category information of multiple fetuses in addition to the position information of multiple fetuses. For example, in the above-mentioned twin case, the category information may include information about whether the multiple fetuses belong to dichorionic twins or monochorionic twins. Those skilled in the art can understand the way in which the target detection network model or the image segmentation network model outputs category information, which is not repeated herein. In another embodiment, the multiple fetus recognition model may include a position detection model and a classification model that are completely independent or only partially parameter-shared. The position detection model may be the above-mentioned target detection network model or the image segmentation network model. The classification model may be implemented using any suitable existing or future classification network model, including but not limited to one or more of the following: Visual Geometry Group (VGG) network, Inception network, Residual network (Resnet) and other network models. The position information of multiple fetuses can be obtained through the position detection model, and the category information of multiple fetuses can be obtained through the classification model.

[0041] The above-mentioned multiple-pregnancy recognition model adopted in step S120 can be obtained by training with a first training data set. The first training data set may include multiple first sample ultrasound images and first annotation information (Ground truth) corresponding to the multiple first sample ultrasound images one by one. The first sample ultrasound image may include multiple sample fetuses. The first annotation information may include position information for indicating the positions of the multiple sample fetuses in the corresponding first sample ultrasound image. The multiple first sample ultrasound images are respectively input into the initial multiple-pregnancy recognition model, and the corresponding predicted multiple-pregnancy recognition results can be obtained. The network structure of the initial multiple-pregnancy recognition model is consistent with that of the multiple-pregnancy recognition model adopted in step S120, but the parameters may be inconsistent. After training the parameters in the initial multiple-pregnancy recognition model, the multiple-pregnancy recognition model adopted in step S120 is obtained. The predicted multiple-pregnancy recognition result and the first annotation information of the multiple first sample ultrasound images can be substituted into the first preset loss function for loss calculation to obtain a first loss value. Subsequently, the parameters in the initial multiple-pregnancy recognition model can be optimized according to the first loss value using the back propagation and gradient descent algorithms. The optimization of the parameters can be iteratively performed until the multiple-pregnancy recognition model reaches a convergence state. After the training is completed, the obtained multiple birth recognition model can be used for subsequent multiple birth recognition. This stage can be called the inference stage of the model.

[0042] Step S130, marking multiple fetuses in the initial ultrasound image based on the multiple fetus identification result.

[0043] The marking operation of step S130 is mainly used to distinguish multiple fetuses. The marking method can be set arbitrarily as needed. For example, multiple fetuses can be marked as fetus 1, fetus 2, fetus 3, etc., or marked as fetus A, fetus B, fetus C, etc. Exemplarily, for any initial ultrasound image, the user can manually mark multiple fetuses in the initial ultrasound image. For example, the user can input marking information, and the device for executing the ultrasound image processing method 100 (such as an ultrasound imaging system) can mark the fetus based on the marking information and combined with the multiple fetus identification results. Optionally, the device for executing the ultrasound image processing method 100 can also automatically mark multiple fetuses in the initial ultrasound image based on the multiple fetus identification results. For example, two fetuses are identified in the initial ultrasound image, and the device for executing the ultrasound image processing method 100 can mark the two fetuses as fetus A and fetus B, respectively. Optionally, the marking can be arbitrarily marked as long as the multiple fetuses can be distinguished. Optionally, the marking can also be marked according to a certain rule, for example, the multiple fetuses are marked in order from near to far from the cervical opening.

[0044] Step S140, acquiring an ultrasound image of at least one fetus to be tested among the marked multiple fetuses.

[0045] Exemplarily, according to the marked multiple fetuses, any one or more of the fetuses can be used as the fetuses to be tested, and the ultrasonic images to be tested of these fetuses to be tested can be obtained. For example, the ultrasonic image to be tested of fetus A can be obtained first, and then the ultrasonic image to be tested of fetus B can be obtained. For each fetus to be tested, one or more ultrasonic images to be tested containing the fetus can be obtained. The ultrasonic image to be tested corresponding to each fetus to be tested can only contain the fetus, or can also contain other fetuses other than the fetus. It is more desirable that the ultrasonic image to be tested corresponding to each fetus to be tested only contains the fetus, and the influence of other fetuses is excluded as much as possible. This can be achieved in the following ways. For example, when collecting the ultrasonic image to be tested, image acquisition can be performed for the physical area where the corresponding fetus to be tested exists alone, and after the original ultrasonic image to be tested is collected, the fetus other than the corresponding fetus to be tested can be detected and removed from the ultrasonic image by target detection and other methods, so as to obtain the ultrasonic image to be tested containing only the corresponding fetus to be tested. The ultrasonic image to be tested can be at least a part of the ultrasonic image in the initial ultrasonic image, or it can be one or more ultrasonic images different from the initial ultrasonic image that are re-collected by the user using an image acquisition device for the physical area where the fetus to be tested is located.

[0046] Step S150, measuring at least one measurement item of at least one fetus to be tested based on the ultrasonic image to be tested.

[0047] For example, based on the acquired one or more ultrasound images to be measured containing the fetus A, the fetus A can be measured. The measurement items may include but are not limited to the head-to-hip length, the placental thickness, the maximum depth of the amniotic fluid, etc. When there are structures corresponding to one or more measurement items in the ultrasound image to be measured, the one or more measurement items can be measured on the ultrasound image to be measured to obtain the corresponding measurement results. For example, when the ultrasound image to be measured is an ultrasound image of a horizontal cross-section of the thalamus, the device for executing the ultrasound image processing method 100 can automatically determine that the measurement items matching the ultrasound image to be measured are the head circumference, the biparietal diameter, etc. The device for executing the ultrasound image processing method 100 can automatically measure the size of the head circumference and the biparietal diameter on the ultrasound image to be measured.

[0048] For example, all the ultrasound images to be tested corresponding to any fetus to be tested may be collected and then the measurement items therein may be measured, or they may be collected and measured at the same time (i.e., real-time collection and measurement). For example, when the corresponding measurement items are found in the current real-time ultrasound image to be tested for the fetus A to be tested, the current real-time ultrasound image to be tested may be measured to obtain the corresponding measurement results, so as to realize the automatic measurement of the current real-time ultrasound image to be tested, and further improve the efficiency of ultrasound scanning.

[0049] According to the ultrasound image processing method of the embodiment of the present invention, a multiple pregnancy recognition result can be obtained based on the initial ultrasound image, and at least one measurement item of at least one fetus to be tested can be measured based on the multiple pregnancy recognition result. The method can assist the user in the recognition and measurement of multiple pregnancy scenes, greatly reducing the user's workload, thereby improving the user's inspection efficiency and accuracy.

[0050] Exemplarily, at least one measurement item of at least one fetus to be tested is measured based on the ultrasound image to be tested, including: performing standard section recognition on the ultrasound image to be tested to determine whether the ultrasound image to be tested belongs to one of at least one preset type of standard section; when the ultrasound image to be tested belongs to a standard section of a specific preset type, performing image segmentation on the ultrasound image to be tested to segment at least one target structure from the ultrasound image to be tested, the at least one target structure belonging to a structure related to a standard section of the specific preset type; based on the segmentation result of the at least one target structure, measuring at least one measurement item, the at least one measurement item is a measurement item related to the at least one target structure.

[0051] In one embodiment, the preset types of standard sections described in this article may include but are not limited to one or more of the following: standard head-to-hip length section, thalamus horizontal cross-section, umbilical blood flow section, upper abdominal cross-section, femoral long axis section, placental section, amniotic fluid section, and cervical canal sagittal section.

[0052] Standard section recognition for the ultrasound image to be tested can be achieved through a standard section recognition model. The standard section recognition model can be any suitable neural network model, for example, any suitable existing or future target detection network model and / or classification network model. For example, the standard section recognition model can include one or more of the following: YOLO series, RCNN series, Retina-Net, VGG network, Inception network, Resnet and other network models. It is preferable that the standard section recognition model can be a target detection network model. Through the target detection network model, in addition to determining the category information of the ultrasound image to be tested, the position information of the target structure in the ultrasound image to be tested can also be determined. The category information of the ultrasound image to be tested is information about which type of standard section the ultrasound image to be tested belongs to. The position information of the target structure in the ultrasound image to be tested is information used to indicate the location of the target structure in the ultrasound image to be tested. Similar to target detection for the fetus, in this embodiment, a target detection frame for indicating the location of the target structure can be obtained through the target detection network model. When the target structure is subsequently segmented, the position information output by the target detection network model can be optionally input into the image segmentation network model to assist in the segmentation of the target structure. Compared with the classification network model, the target detection network model has a higher accuracy rate in standard section recognition of the ultrasound image to be tested.

[0053] The above-mentioned standard section recognition model can be obtained by training with a second training data set. The second training data set may include a plurality of second sample ultrasound images and second annotation information corresponding to the plurality of second sample ultrasound images one by one. The plurality of second sample ultrasound images may include at least one group of sample ultrasound images corresponding to at least one preset type of standard section, each group of sample ultrasound images includes one or more second sample ultrasound images, and the one or more second sample ultrasound images belong to the same preset type of standard section. The second annotation information may include a standard section indicating which preset type the corresponding second sample ultrasound image belongs to. The plurality of second sample ultrasound images are respectively input into the initial standard section recognition model to obtain the corresponding predicted standard section recognition results. The predicted standard section recognition results and the second annotation information of the plurality of second sample ultrasound images can be substituted into the second preset loss function for loss calculation to obtain the second loss value. Subsequently, the parameters in the initial standard section recognition model can be optimized using the back propagation and gradient descent algorithms according to the second loss value. The optimization of the parameters can be iteratively performed until the standard section recognition model reaches a convergence state. When the training is completed, the obtained standard section recognition model can be used for subsequent standard section recognition.

[0054] It is understandable that the ultrasound image to be tested may belong to a standard section or may not belong to a standard section. In the case where the standard section recognition result of the ultrasound image to be tested is a standard section belonging to a specific preset type, the ultrasound image to be tested can be segmented. It is understandable that the standard section of a specific preset type is one of at least one preset type of standard section. At least one preset type of standard section can be a pre-set standard section. Different types of standard sections may contain different target structures. After determining the type of standard section to which the current ultrasound image to be tested belongs, the corresponding target structure contained therein can be segmented. For example, if the standard section recognition result of the ultrasound image to be tested is a horizontal cross section of the thalamus, the ultrasound image to be tested can be segmented to segment the thalamus structure therefrom. The target structures described herein may include, but are not limited to: one or more of structures such as amniotic fluid, placenta, thalamus, and four-chamber heart.

[0055] The image segmentation operation for the ultrasound image to be tested can be obtained by using a pre-trained image segmentation model. The image segmentation model can be implemented by any suitable existing or future neural network model. For example, the image segmentation model can include one or more of the following: FCN, Unet, DeepLab series, Vnet and other network models.

[0056] The above-mentioned image segmentation model can be obtained by training with a third training data set. The third training data set may include a plurality of third sample ultrasound images and third annotation information corresponding to the plurality of third sample ultrasound images. The third sample ultrasound images may include at least one group of sample ultrasound images corresponding to at least one preset type of standard section, each group of sample ultrasound images includes one or more third sample ultrasound images, and the one or more third sample ultrasound images belong to the same preset type of standard section. The third annotation information may include location information for indicating the location of at least one target structure in the corresponding third sample ultrasound image. The plurality of third sample ultrasound images are respectively input into the initial image segmentation model, and the corresponding predicted image segmentation results can be obtained. The predicted image segmentation results and the third annotation information of the plurality of third sample ultrasound images can be substituted into the third loss function for loss calculation to obtain a third loss value. Subsequently, the parameters in the initial image segmentation model can be optimized using the back propagation and gradient descent algorithms according to the third loss value. The optimization of the parameters can be iteratively performed until the image segmentation model reaches a convergence state. When the training is completed, the obtained image segmentation model can be used for subsequent image segmentation of the target structure.

[0057] The segmentation result of the target structure may include a mask or envelope for indicating the location of each target structure. According to the segmentation result of the target structure, a relatively accurate location of at least one target structure may be determined. Each target structure may correspond to its own measurement item, and the measurement item corresponding to each target structure may be one or more. For example, in the standard section of the head-buttock length, the target structure may include the head-buttock position, and the measurement item corresponding to the position may include the head-buttock length. Therefore, according to the segmentation result of the head-buttock position, the head-buttock length of the head-buttock position may be measured, and then the measurement result related to the head-buttock length may be obtained.

[0058] According to the above technical solution, the standard section of the ultrasonic image to be measured is firstly identified, and then, when the ultrasonic image to be measured belongs to a standard section of a specific preset type, the ultrasonic image to be measured is segmented, and at least one measurement item of at least one target structure obtained by segmentation is measured. The measurement efficiency and accuracy of this solution are both high.

[0059] Exemplarily, the number of at least one fetus to be tested is greater than or equal to two, and at least one measurement item of at least one fetus to be tested is measured based on the ultrasound image to be tested, including: for a first measurement item in the at least one measurement item, calculating a first difference between the measurement results of at least one fetus to be tested on the first measurement item; when the first difference is greater than or equal to a first difference threshold, outputting a first prompt message.

[0060] In one embodiment, the number of fetuses to be tested in the ultrasound image to be tested may be greater than or equal to two. For example, the number of fetuses to be tested may be two, namely, fetus A to be tested and fetus B to be tested. At least one measurement item of the two fetuses to be tested in the ultrasound image to be tested is measured. The at least one measurement item may include, but is not limited to, one or more of the following: head-to-hip length, biparietal diameter length, placental thickness, and maximum depth of amniotic fluid. The first measurement item may be any one or more of the at least one measurement item.

[0061] Exemplarily, the first measurement item may include crown-hip length.

[0062] For example, the first measurement item may be the head-to-hip length. For the two fetuses to be measured, the head-to-hip lengths of the two fetuses to be measured may be measured respectively. According to the measurement result of the first measurement item, the head-to-hip length of the fetus A to be measured is L A , the head-to-hip length of the fetus B to be measured is L B The head-to-hip length L can be calculated A and head-to-hip length L B The difference between them is then used to obtain the first difference ΔL1. The user can also pre-set the first difference threshold L T For example, for the head-to-hip length, the first difference threshold L T If the first difference ΔL1 is greater than or equal to 3 mm, a first prompt message may be output to prompt the user to pay attention to the current measurement result.

[0063] According to the above technical solution, the difference between the measurement results of multiple fetuses to be tested on certain measurement items can be calculated. When the difference is too large, the first prompt information can be output for the user to view. This solution can assist the user in measuring the measurement items and output the first prompt information to the user in time so that the user can take timely countermeasures against this situation.

[0064] Exemplarily, at least one measurement item of at least one fetus to be tested is measured based on the ultrasound image to be tested, including: for a second measurement item in the at least one measurement item, calculating a second difference between a measurement result of any fetus to be tested on the second measurement item and a standard measurement result of a standard fetus on the second measurement item; when the second difference is greater than or equal to a second difference threshold, outputting a second prompt information.

[0065] In one embodiment, the fetus to be tested in the ultrasound image to be tested may include a fetus to be tested A and a fetus to be tested B. At least one measurement item of the two fetuses to be tested in the ultrasound image to be tested is measured. The at least one measurement item may include, but is not limited to, one or more of the following: head-to-hip length, biparietal diameter length, placental thickness, and maximum depth of amniotic fluid. The second measurement item may be any one or more of the at least one measurement item.

[0066] Exemplarily, the second measurement item may include one or more of the following: biparietal diameter length, placental thickness, and maximum depth of amniotic fluid.

[0067] For example, the second measurement item may be the maximum depth of amniotic fluid. For two fetuses to be tested, the maximum depth of amniotic fluid corresponding to each of them may be measured respectively. According to the measurement result of the second measurement item, the maximum depth of amniotic fluid of fetus A to be tested is H. A , the maximum depth of amniotic fluid of fetus B to be tested is H B The maximum depth of amniotic fluid for a standard fetus is H T The maximum depth of amniotic fluid of any fetus to be tested and the maximum depth of amniotic fluid of the standard fetus can be calculated. T For example, the maximum depth H of the amniotic fluid of the fetus A to be tested can be compared. A The maximum depth of amniotic fluid H for a standard fetus T The difference between the two is used to obtain the second difference ΔH1. The user can pre-set the second difference threshold H T1 For example, for the maximum depth of amniotic fluid, the second difference threshold H T1 If the second difference ΔH1 is greater than or equal to 5 mm, a second prompt message may be output to prompt the user to pay attention to the current measurement result.

[0068] For example, the maximum depth of amniotic fluid can be selected as H A and the maximum depth of amniotic fluid is H B The larger value between the two is consistent with the maximum depth of amniotic fluid H of the standard fetus. T A comparison is performed and a second difference between the two is calculated.

[0069] According to the above technical solution, the difference between the measurement results of any fetus to be tested and the standard fetus on certain measurement items can be calculated, and when the difference is too large, the second prompt information can be output for the user to view. This solution can assist the user in measuring the measurement items and output the second prompt information to the user in time so that the user can take timely countermeasures against this situation.

[0070] Exemplarily, after measuring at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested, the method further includes: for a third measurement item among the at least one measurement item, selecting the maximum measurement result output from the measurement results of at least one fetus to be tested on the third measurement item.

[0071] The third measurement item may include one or more of the following: crown-hip length, biparietal diameter, placental thickness, and maximum depth of amniotic fluid. For some measurement items, the largest measurement result can be selected from the measurement results of the fetus to be tested for output, which is convenient for users to view. For example, the larger value can be selected from the measurement results of the crown-hip length of twins as the crown-hip length result of the twins and output, and the output crown-hip length result can be used to calculate the gestational age of the twins.

[0072] The maximum measurement result may be outputted via an output device. The output device may be an output device included in the apparatus for performing the ultrasound image processing method 100 or an output device communicatively connected to the apparatus for performing the ultrasound image processing method 100. The output device may include, but is not limited to, one or more of the following: a display device, a wired and / or wireless communication device, a speaker, a signal light, etc.

[0073] Exemplarily, obtaining an ultrasound image to be tested that includes at least one of the marked multiple fetuses includes: based on the fetal marking result, performing target tracking on an ultrasound image sequence acquired in real time during the user's operation of the ultrasound probe to determine the trajectory of any target fetus to be tested among the at least one fetus to be tested in the ultrasound image sequence; if it is found that the trajectory of the target fetus to be tested is lost, outputting a third prompt message; after the trajectory of the target fetus to be tested is lost for a preset period of time, or when an inspection end instruction is received, determining that the ultrasound image including the target fetus to be tested in the ultrasound image sequence is the ultrasound image to be tested including the target fetus to be tested.

[0074] The target fetus to be tested is any one of the at least one fetus to be tested. Of course, for multiple fetuses to be tested, each fetus can be regarded as a target fetus to be tested to perform the tracking operation described in this embodiment. The target fetus to be tested can be specified by the user or automatically selected by the device for performing the ultrasound image processing method 100.

[0075] In one embodiment, for the fetus "test fetus A" marked in step S130, it can be used as the initial tracking object, and target tracking is performed in the ultrasound image sequence collected in real time during the user's operation of the ultrasound probe. Target tracking can be implemented using any suitable existing or future target tracking algorithm, including but not limited to various tracking algorithms based on correlation filtering and / or various tracking algorithms based on convolutional neural networks (CNN). For example, target tracking can be performed by a visual tracking method (Deep Learning Tracker, DLT). According to the target tracking result, the trajectory of the target fetus to be tested (i.e., the fetus to be tested A) in the ultrasound image sequence can be determined. If it is found that the trajectory of the target fetus to be tested (i.e., the fetus to be tested A) is lost, a third prompt message can be output to prompt the user. The loss of the trajectory of the target fetus to be tested may include that no fetus can be detected in the subsequent ultrasound image, or that the fetus can be detected but is not the target fetus to be tested. Through the third prompt message, the user can promptly know that the current detection position has deviated from the physical area where the target fetus to be tested is located. If the ultrasound examination of the target fetus to be tested has not been completed, the user can promptly return to the physical area where the target fetus to be tested is located to continue detection. If the ultrasound examination of the target fetus to be tested has been completed, the user may not perform any operation or input an examination end instruction to the device for executing the ultrasound image processing method 100 through an input device. The input device may be an input device included in the device for executing the ultrasound image processing method 100 or an input device communicatively connected to the device for executing the ultrasound image processing method 100. The input device may include, but is not limited to, one or more of the following: a mouse, a keyboard, a touch screen, a microphone, etc.

[0076] After the trajectory of the target fetus to be tested (i.e., fetus A to be tested) loses the preset time, or when the inspection end instruction is received, it can be determined that the ultrasound inspection of the target fetus to be tested has ended, and at this time, the ultrasound image containing the target fetus to be tested in the ultrasound image sequence is determined to be the ultrasound image to be tested containing the target fetus to be tested for the measurement operation in step S150 described above. The ultrasound image to be tested containing the target fetus to be tested is at least a partial image of the ultrasound image to be tested containing at least one fetus to be tested.

[0077] The preset duration can be any duration greater than 0 set by the user. For example, the preset duration can be 3 seconds, 5 seconds, 10 seconds, and so on. The end of the ultrasonic examination of the target fetus to be tested is determined by the time when the trajectory of the target fetus to be tested is lost. This solution does not require user intervention, can automatically determine the inspection process, has a high degree of automation, and provides a good user experience. In addition, the user can also input an inspection end instruction through an input device. For example, the input device can be a display device, and the display interface of the display device can include an operable control. After the user clicks on the operable control, the device for executing the ultrasonic image processing method 100 can receive the inspection end instruction through the input device. For example, the operable control can be a rectangular control, and a "End" is marked on one side (such as the right side) of the control.

[0078] According to this solution, whether the ultrasound examination of the target fetus to be tested by needle 5 is completed can be determined by whether the user inputs the inspection end instruction. This solution has a high degree of user autonomy and allows users to

[0079] Users can decide the end time of the inspection according to their needs.

[0080] According to the above technical solution, in the process of performing ultrasound examination on the target fetus to be tested, the target fetus to be tested can be automatically tracked, and when the track of the target fetus to be tested is found to be lost,

[0081] In this case, the third prompt information can be output in time. This method can assist the user to better complete the ultrasound examination of the target fetus to be tested, has a high degree of intelligence, and provides a good user experience.

[0082] The first prompt information, the second prompt information and the third prompt information may be any form of prompt information, including but not limited to one or more of the following: image information, video information, audio information, light information, etc. The first prompt information, the second prompt information and the third prompt information may be outputted by the above output device.

[0083] 5. Exemplarily, multiple fetuses in the initial ultrasound image are marked based on the multiple fetus identification result.

[0084] The method comprises: identifying the position of the cervical opening from an initial ultrasound image; determining a first relative position relationship between a plurality of fetuses and the cervical opening based on the multiple fetus identification results and the cervical opening identification results; sorting the plurality of fetuses based on the first relative position relationship; and marking the plurality of fetuses in the sorted order.

[0085] In one embodiment, the multiple pregnancy recognition model described above can be used to obtain the initial ultrasound image.

[0086] That is, through the multiple pregnancy recognition operation in step S120, in addition to obtaining the multiple pregnancy recognition result, the cervical os recognition result can also be obtained optionally. Of course, optionally, an independent cervical os recognition model different from the above multiple pregnancy recognition model can be used to recognize the cervical os.

[0087] The cervical os recognition model can also be a target detection network model or an image segmentation network model, which can be understood by referring to the above description and will not be repeated here.

[0088] According to the multiple fetus identification results and the cervical ostium identification results, for example, the positions of the two fetuses and the cervical ostium are respectively identified by the target detection network model, and the first relative position relationship between the two fetuses and the cervical ostium can be determined respectively. The relative position relationship can be each fetus and the cervical ostium.

[0089] By way of example and not limitation, the plurality of fetuses may be sorted in order of distance from the cervical opening based on the first relative position relationship. For example, the fetus that is closer to the cervical opening may be marked as fetus A (which may be delivered first), and the fetus that is farther from the cervical opening may be marked as fetus B.

[0090] According to the above technical solution, multiple fetuses are sorted and marked according to the sorting order according to the first relative position relationship between the multiple fetuses and the cervical opening. This method can realize automatic marking of fetuses and effectively reduce the workload of users.

[0091] Exemplarily, marking multiple fetuses in the initial ultrasound image based on the multiple fetus identification result may include: receiving marking information input by a user; and marking the multiple fetuses based on the marking information.

[0092] In one embodiment, the user can manually mark multiple fetuses based on the identified multiple fetuses. For example, the output device can be a display device, and the device for executing the image processing method 100 can display the information of multiple fetuses on a display interface through the display device, such as displaying an ultrasound image containing multiple fetuses on a display interface. The user can mark the fetus based on the position of each fetus through the input device. For example, the user can select the fetus closest to the cervical opening with a mouse, and then input the marking information "fetus A" through the keyboard to indicate that the fetus is marked as fetus A. The order of manual marking by the user can be arbitrary.

[0093] According to the above technical solution, the user is allowed to manually mark multiple fetuses, which can make it easier for the user to remember and distinguish.

[0094] Exemplarily, performing multiple fetus recognition on an initial ultrasound image to obtain a multiple fetus recognition result includes: performing multiple fetus recognition on the initial ultrasound image through a target detection algorithm to obtain a multiple fetus recognition result, the multiple fetus recognition result includes a target detection frame for indicating the position of each fetus. After performing multiple fetus recognition on the initial ultrasound image to obtain the multiple fetus recognition result, method 100 also includes: determining a second relative position relationship of at least some of the multiple fetuses based on the multiple fetus recognition result, the second relative position relationship includes an orientation relationship between two fetuses and / or a distance between target detection frames corresponding to the two fetuses; and outputting position information used to represent the second relative position relationship.

[0095] In one embodiment, multiple fetuses are identified by performing multiple fetus identification on the initial ultrasound image through a target detection algorithm (such as the target detection network model described above), and a multiple fetus identification result including a target detection frame indicating the respective positions of multiple fetuses can be obtained.

[0096] According to the multiple fetus identification results, the second relative position relationship of at least some of the multiple fetuses can be determined. Among them, the second relative position relationship may include the orientation relationship between the two fetuses. For example, taking one of the two fetuses as a reference, in the initial ultrasound image, the other fetus may be located above, below, to the left, to the right, or in any other orientation. The second relative position relationship may also include the distance between the target detection frames corresponding to the two fetuses. The distance between the corresponding features of the two target detection frames can be calculated respectively as the distance between the target detection frames corresponding to the two fetuses. For example, the corresponding features may include the center point of each of the two target detection frames, the upper left corner point of each of the two target detection frames, and the like. The device for executing the image processing method 100 may output the second relative position relationship through the above-mentioned output device, for example, displaying it in a display interface for user reference.

[0097] According to the above technical solution, the second relative position relationship between the two fetuses can be determined based on the target detection frames of the respective fetuses and the relative position relationship can be output. This method is convenient for users to refer to and can better assist users in examining the fetuses.

[0098] Figure 2 FIG. 2 shows a schematic flow chart of an ultrasonic image processing method according to an embodiment of the present invention. Figure 2As shown, in the process of examining the abdomen of a pregnant woman through an ultrasound imaging system, an ultrasound video can be acquired based on the ultrasound probe and imaging module in the ultrasound imaging system. One or more ultrasound images (i.e., initial ultrasound images) can be obtained based on the ultrasound video. One or more ultrasound images are input into the multiple-fetus recognition model to obtain multiple-fetus recognition results corresponding to each ultrasound image. The user can manually mark the multiple-fetus recognition results and input the marked ultrasound image into the standard section recognition model to determine whether the marked ultrasound image belongs to a certain preset type of standard section. In the case where the ultrasound image inputs a preset type of standard section, the ultrasound image is input into the image segmentation model to obtain the target structure, and the measurement items are measured for the target structure. In addition, Figure 2 The operations for training the above three models are also shown. It should be noted that although the multi-birth recognition model is shown as "video annotation and model training", it can also be trained using annotated data based on discrete images, that is, the images in the training data set it uses do not need to come from the same video. The training of the standard section recognition model is similar to this and will not be repeated. Of course, conversely, the image segmentation model can also be trained using video-based annotated data.

[0099] According to yet another aspect of the present invention, an ultrasonic image processing device is provided. Figure 3 FIG. 4 shows a schematic block diagram of an ultrasonic image processing device 300 according to an embodiment of the present invention. Figure 3 As shown, the device 300 may include: a first acquisition module 310 , an identification module 320 , a marking module 330 , a second acquisition module 340 and a measurement module 350 .

[0100] The first acquisition module 310 is used to acquire an initial ultrasound image.

[0101] The recognition module 320 may be used to perform multiple-pregnancy recognition on the initial ultrasound image to obtain a multiple-pregnancy recognition result.

[0102] The marking module 330 may be used to mark multiple fetuses in the initial ultrasound image based on the multiple fetus identification result.

[0103] The second acquisition module 340 may be used to acquire an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses.

[0104] The measurement module 350 may be used to measure at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested.

[0105] Exemplarily, the measurement module 350 may include: a first identification submodule, which can be used to perform standard section recognition on the ultrasound image to be tested to determine whether the ultrasound image to be tested belongs to at least one of the standard sections of a preset type; an image segmentation submodule, which can be used to perform image segmentation on the ultrasound image to be tested when the ultrasound image to be tested belongs to a standard section of a specific preset type, so as to segment at least one target structure from the ultrasound image to be tested, and the at least one target structure belongs to a structure related to the standard section of the specific preset type; a measurement submodule, which can be used to measure at least one measurement item based on the segmentation result of at least one target structure, and the at least one measurement item is a measurement item related to at least one target structure.

[0106] Exemplarily, the number of at least one fetus to be tested is greater than or equal to two, and the device 300 may also include: a first calculation module, which can be used to calculate the first difference between the measurement results of at least one fetus to be tested on the first measurement item of the at least one measurement item after the measurement module 350 measures at least one measurement item of the at least one fetus to be tested based on the ultrasound image to be tested; and a first output module, which can be used to output a first prompt message when the first difference is greater than or equal to a first difference threshold.

[0107] Exemplarily, the first measurement item includes the head-to-hip length.

[0108] Exemplarily, the device 300 may also include: a second calculation module, which can be used to calculate the second difference between the measurement result of any fetus to be tested on the second measurement item and the standard measurement result of the standard fetus on the second measurement item for the second measurement item in the at least one measurement item after the measurement module 350 measures at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested; and a second output module, which can be used to output a second prompt message when the second difference is greater than or equal to a second difference threshold.

[0109] Exemplarily, the second measurement item includes one or more of the following: biparietal diameter length, placental thickness, and maximum depth of amniotic fluid.

[0110] Exemplarily, the device 300 also includes: a selection module for selecting the maximum measurement result output from the measurement results of at least one fetus to be tested on the third measurement item for a third measurement item among the at least one measurement item after the measurement module 350 measures at least one measurement item of at least one fetus to be tested based on the ultrasound image to be tested.

[0111] Exemplarily, the second acquisition module 340 may include: a target tracking submodule, which can be used to perform target tracking on an ultrasound image sequence acquired in real time during the user's operation of the ultrasound probe based on the fetal marking result, so as to determine the trajectory of any target fetus to be tested among at least one fetus to be tested in the ultrasound image sequence; a first output submodule, which can be used to output a third prompt message if it is found that the trajectory of the target fetus to be tested is lost; a first determination submodule, which can be used to determine that the ultrasound image containing the target fetus to be tested in the ultrasound image sequence is the ultrasound image to be tested containing the target fetus to be tested after the trajectory of the target fetus to be tested is lost for a preset period of time, or when an inspection end instruction is received.

[0112] Exemplarily, the marking module 330 may include: a second identification submodule, which can be used to identify the position of the cervical os from the initial ultrasound image; a second determination submodule, which can be used to determine the first relative position relationship between multiple fetuses and the cervical os based on the multiple fetus identification results and the cervical os identification results; a sorting submodule, which can be used to sort the multiple fetuses based on the first relative position relationship; and a first marking submodule, which can be used to mark the multiple fetuses in sorting order.

[0113] Exemplarily, the marking module 330 may include: a receiving submodule, which may be used to receive marking information input by a user; and a second marking submodule, which may be used to mark multiple fetuses based on the marking information.

[0114] Exemplarily, the recognition module 320 may include: a third recognition submodule, which is used to perform multiple-fetus recognition on the initial ultrasound image through a target detection algorithm to obtain a multiple-fetus recognition result, and the multiple-fetus recognition result includes a target detection frame for indicating the position of each fetus. The device 300 may also include: a determination module, which can be used to determine the second relative position relationship of at least some of the multiple fetuses based on the multiple fetus recognition result after the recognition module 320 performs multiple-fetus recognition on the initial ultrasound image to obtain the multiple-fetus recognition result, and the second relative position relationship includes the orientation relationship between the two fetuses and / or the distance between the target detection frames corresponding to the two fetuses; a third output module, which can be used to output position information used to represent the second relative position relationship.

[0115] According to yet another aspect of the present invention, an electronic device is provided. Figure 4 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. As shown in the figure, the electronic device 400 includes a processor 410 and a memory 420, wherein the memory 420 stores computer program instructions, and the computer program instructions are used to execute the above-mentioned ultrasound image processing method 100 when the processor 410 is running.

[0116] Exemplarily, the electronic device 400 may be an ultrasonic diagnostic device or an ultrasonic workstation.

[0117] The ultrasonic diagnostic device may include a probe, a first processor, a first memory, and a first display, etc. In the case where the electronic device 400 is an ultrasonic diagnostic device, the processor 410 may be the first processor, and the memory 420 may be the first memory. The probe may be used to transmit ultrasonic waves to a target object (e.g., a fetus) and receive ultrasonic echoes returned from the target object, thereby obtaining an ultrasonic echo signal. The probe transmits the ultrasonic echo signal to the first processor. The first processor may process the ultrasonic echo signal to obtain an ultrasonic image of the target object, i.e., the initial ultrasonic image, the ultrasonic image to be measured, etc. described herein. The ultrasonic image obtained by the first processor may be stored in the first memory. These ultrasonic images may be optionally displayed on the first display for viewing by the user. In addition, the first processor may be used to execute the ultrasonic image processing method 100 described herein, and may optionally store some processing information generated during the processing in the first memory. The above-mentioned processing information may include intermediate data and / or final results, such as the multiple fetus identification results, fetal marking results, cervical ostium identification results, target structure segmentation results, fetal measurement results on various measurement items, etc. described herein. Optionally, the ultrasonic diagnostic equipment may further include a first input device, such as one or more of a mouse, a keyboard, and a touch screen, for a user to input information, instructions, and the like.

[0118] An ultrasound workstation can also be called an ultrasound imaging workstation. An ultrasound workstation is a device that integrates patient registration, image acquisition, diagnosis editing, report printing, image post-processing, medical record query, statistical analysis and other functional modules. An ultrasound workstation can be communicatively connected to an ultrasound diagnostic device, for example, through any wired or wireless communication method. The ultrasound diagnostic device can transmit information such as the collected ultrasound echo signal and / or the image data of the ultrasound image to the ultrasound workstation.

[0119] The ultrasonic workstation may include a second processor, a second memory, a second display, and the like. In the case where the electronic device 400 is an ultrasonic workstation, the processor 410 may be a second processor, and the memory 420 may be a second memory. The second processor may receive an ultrasonic echo signal and / or image data of an ultrasonic image from an ultrasonic diagnostic device through a communication interface, and may obtain an ultrasonic image based on the ultrasonic echo signal or directly obtain an ultrasonic image based on the image data of the ultrasonic image. The second processor may store the obtained ultrasonic image in a second memory. These ultrasonic images may optionally be displayed on a second display for viewing by a user. In addition, the second processor may be used to execute the ultrasonic image processing method 100 described herein, and may optionally store some processing information generated during the processing in a second memory. Examples of processing information may refer to the description above. Optionally, the ultrasonic diagnostic device may also include a second input device, such as one or more of a mouse, a keyboard, and a touch screen, for a user to input information, instructions, and the like.

[0120] In addition, the ultrasound workstation may also include other functional modules such as a printing device, etc. Through the second processor, the second memory and various functional modules of the ultrasound workstation, functions such as processing, storing, replaying, printing, statistics, and retrieval of ultrasound images can be completed.

[0121] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned ultrasound image processing method 100 when running. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0122] A person skilled in the art may understand the specific implementation scheme of the ultrasonic image processing device, electronic device and storage medium by reading the above description of the ultrasonic image processing method, which will not be described in detail here for the sake of brevity.

[0123] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0125] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0126] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances,

[0127] Well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

[0128] Similarly, it should be understood that in order to streamline the present invention and facilitate understanding of one of the various inventive aspects,

[0129] In the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this should not be considered to be a separate embodiment.

[0130] This inventive method is to reflect an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the respective claims reflect, the inventive subject matter resides in that less than all features of a single disclosed embodiment may be employed.

[0131] The following claims follow the detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0132] Those skilled in the art will understand that, except for mutually exclusive features, all features disclosed in this specification (including accompanying claims, abstracts and drawings) may be used in any combination.

[0133] Unless otherwise expressly stated, the features disclosed in this specification (including the accompanying claims, abstract and drawings) are

[0134] Each feature may be replaced by an alternative feature serving the same, equivalent or similar purpose.

[0135] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features that are included in other embodiments but not other features, the features of the different embodiments may be

[0136] The combination of the embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the embodiments claimed for protection can be used in any combination.

[0137] use.

[0138] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.

[0139] It should be understood that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some modules in the ultrasound image processing device according to the embodiment of the present invention.

[0140] The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals.

[0141] Such a signal may be downloaded from an Internet website, or provided on a carrier signal, 0 or in any other form.

[0142] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0143] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An ultrasonic image processing method, comprising: Acquire an initial ultrasound image; Performing multiple fetus identification on the initial ultrasound image to obtain a multiple fetus identification result, wherein the multiple fetus identification result includes position information indicating positions of multiple fetuses in the initial ultrasound image; marking the multiple fetuses in the initial ultrasound image based on the position information of the multiple fetuses in the initial ultrasound image; Acquire an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses; At least one measurement item of the at least one fetus to be tested is measured based on the ultrasound image to be tested.

2. The method of claim 1, wherein: The measuring at least one measurement item of the at least one fetus to be tested based on the ultrasonic image to be tested comprises: Performing standard section recognition on the ultrasonic image to be tested to determine whether the ultrasonic image to be tested belongs to one of at least one preset type of standard section; In a case where the ultrasound image to be measured belongs to a standard section of a specific preset type, performing image segmentation on the ultrasound image to be measured to segment at least one target structure from the ultrasound image to be measured, wherein the at least one target structure belongs to a structure related to the standard section of the specific preset type; Based on the segmentation result of the at least one target structure, the at least one measurement item is measured, and the at least one measurement item is a measurement item related to the at least one target structure.

3. The method of claim 1, wherein: The number of the at least one fetus to be tested is greater than or equal to two, and after measuring at least one measurement item of the at least one fetus to be tested based on the ultrasound image to be tested, the method further includes: For a first measurement item among the at least one measurement item, calculating a first difference between measurement results of the at least one fetus to be measured on the first measurement item; When the first difference is greater than or equal to a first difference threshold, first prompt information is output.

4. The method of claim 3, wherein: The first measurement item includes the head-to-hip length.

5. The method of claim 1, wherein: After measuring at least one measurement item of the at least one fetus to be tested based on the ultrasonic image to be tested, the method further includes: For a second measurement item among the at least one measurement item, calculating a second difference between a measurement result of any fetus to be tested on the second measurement item and a standard measurement result of a standard fetus on the second measurement item; When the second difference is greater than or equal to a second difference threshold, second prompt information is output.

6. The method of claim 5, wherein: The second measurement item includes one or more of the following: biparietal diameter length, placental thickness, and maximum depth of amniotic fluid.

7. The method of claim 1, wherein: After measuring at least one measurement item of the at least one fetus to be tested based on the ultrasonic image to be tested, the method further includes: For a third measurement item among the at least one measurement item, a maximum measurement result is selected and outputted from the measurement results of the at least one fetus to be measured on the third measurement item.

8. The method according to any one of claims 1 to 7, wherein: The step of acquiring an ultrasound image of at least one fetus to be tested among the marked fetuses comprises: Based on the fetal marking result, target tracking is performed on an ultrasound image sequence acquired in real time during the user's operation of the ultrasound probe to determine a trajectory of any target fetus to be tested among the at least one fetus to be tested in the ultrasound image sequence; If it is found that the trajectory of the target fetus to be detected is lost, outputting a third prompt information; After the trajectory of the target fetus to be tested is lost for a preset time period, or when an examination end instruction is received, the ultrasound image containing the target fetus to be tested in the ultrasound image sequence is determined as the ultrasound image to be tested containing the target fetus to be tested.

9. The method according to any one of claims 1 to 7, wherein: The step of marking the multiple fetuses in the initial ultrasound image based on the position information of the multiple fetuses in the initial ultrasound image comprises: identifying the location of the cervical os from the initial ultrasound image; Determine a first relative position relationship between the plurality of fetuses and the cervical os based on the position information of the plurality of fetuses in the initial ultrasound image and the recognition result of the cervical os; sorting the plurality of fetuses based on the first relative position relationship; The plurality of fetuses are labeled in sorted order.

10. The method according to any one of claims 1 to 7, wherein: The step of marking the multiple fetuses in the initial ultrasound image based on the position information of the multiple fetuses in the initial ultrasound image comprises: Receive tag information input by the user; The multiple fetuses are marked based on the marking information and the position information of the positions of the multiple fetuses in the initial ultrasound image.

11. The method according to any one of claims 1 to 7, wherein: The performing multiple-pregnancy recognition on the initial ultrasound image to obtain a multiple-pregnancy recognition result includes: The initial ultrasound image is subjected to multiple fetus identification by a target detection algorithm to obtain the multiple fetus identification result, wherein the multiple fetus identification result includes a target detection frame for indicating the respective positions of the fetuses, or the multiple fetus identification result includes a target detection frame for indicating the respective positions of the fetuses and category information for indicating the categories to which the multiple fetuses in the initial ultrasound image belong, After performing multiple-pregnancy recognition on the initial ultrasound image to obtain a multiple-pregnancy recognition result, the method further includes: Based on the multiple fetus identification result, determining a second relative position relationship of at least some of the multiple fetuses, where the second relative position relationship includes an orientation relationship between two fetuses and / or a distance between target detection frames corresponding to the two fetuses; Outputting position information indicating the second relative position relationship.

12. An ultrasonic image processing device, comprising: A first acquisition module, used for acquiring an initial ultrasound image; an identification module, configured to perform multiple-fetus identification on the initial ultrasound image to obtain a multiple-fetus identification result, wherein the multiple-fetus identification result includes position information indicating positions of multiple fetuses in the initial ultrasound image; a marking module, configured to mark the multiple fetuses in the initial ultrasound image based on the position information of the multiple fetuses in the initial ultrasound image; A second acquisition module is used to acquire an ultrasound image to be tested that includes at least one fetus to be tested among the marked multiple fetuses; A measuring module is used to measure at least one measurement item of the at least one fetus to be tested based on the ultrasound image to be tested.

13. An electronic device, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the ultrasonic image processing method according to any one of claims 1 to 11.

14. The electronic device according to claim 13, wherein: The electronic equipment is an ultrasonic diagnostic device or an ultrasonic workstation.

15. A storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the ultrasound image processing method according to any one of claims 1 to 11.