An ultrasonic imaging method and ultrasonic imaging apparatus

Abstract

The embodiment of the application discloses an ultrasonic imaging method, which is applied to an ultrasonic imaging device, and the method comprises the following steps: acquiring three-dimensional body data of a first to-be-detected tissue; identifying three-dimensional body data of a fetal spine and three-dimensional body data of a fetal rib from the three-dimensional body data of the first to-be-detected tissue; rendering the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib to obtain a three-dimensional ultrasonic image of a fetal rib structure; marking the fetal rib in the three-dimensional ultrasonic image of the fetal rib structure; and outputting the three-dimensional ultrasonic image of the marked fetal rib structure. The embodiment of the application discloses an ultrasonic imaging device.

Description

[0001] The present disclosure is based on the Chinese patent application No. 201811623609.9, entitled "Ultrasonic imaging method and ultrasonic imaging device", filed on December 28, 2018, and the present disclosure is a divisional of the Chinese patent application No. 201811623609.9, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to medical diagnostic technology, and relate to, but are not limited to, an ultrasonic imaging method and an ultrasonic imaging device. BACKGROUND

[0003] Prenatal ultrasound examination is one of the most important items that a pregnant woman must check during her pregnancy, and its main functions include determining the age of the fetus, analyzing the development of the fetus, detecting fetal deformities or abnormalities, taking photos and dynamic videos of the fetus, and the like. Among them, fetal rib number abnormalities and the position of the conus medullaris are one of the key screening and detection items of prenatal ultrasound.

[0004] Fetal rib (or vertebral) number abnormalities (or lesions) are often associated with fetal chromosomal abnormalities and some clinical syndromes, such as trisomy syndrome, thoracic dysplasia syndrome, and type I bone dysplasia. During the fetal period, the position of the conus medullaris is of great significance for early screening of spinal cord tethering syndrome. Due to the fact that the position of the conus medullaris of the fetus does not change with the change of gestational age, when the spinal cord is pathologically changed due to congenital reasons, the position of the conus medullaris decreases, which causes a neurological impairment syndrome known as spinal cord tethering syndrome. If these congenital abnormalities are not discovered during prenatal examination, it will bring huge spiritual and economic burden to the patient's family and society, and even cause the patient to die of respiratory distress due to thoracic deformity in the neonatal period.

[0005] As a safe, convenient, non-invasive, and highly repeatable imaging technology, two-dimensional and three-dimensional ultrasound can characteristically reflect fetal rib, fetal spine, and vertebral abnormalities, and has become the preferred method for doctors to diagnose fetal rib abnormalities. Among them, two-dimensional ultrasound can check the fetal rib and spinal cord from the sagittal, transverse, and coronal planes, and can fully understand the condition of the fetal rib, fetal spine, and spinal cord; and three-dimensional ultrasound can overcome the problems of lack of spatial sense, poor fidelity, and difficult positioning in the fetal rib and spinal cord examination. However, at the present stage, doctors still need a lot of manual operation when using three-dimensional ultrasound to check the fetal rib and spinal cord, and the current examination still has the following pain points:

[0006] The fetus can be in various postures during the examination, and the median sagittal plane of the fetal spine needs to be taken as the starting plane when the three-dimensional ultrasound volume is collected. After the three-dimensional volume data of the fetal ribs and the fetal spine are acquired, the doctor needs to have a very deep understanding of the three-dimensional space, so as to determine the accurate position of the ribs with defects and diseases of the fetus or the accurate position of the conus medullaris by means of multiple manual rotation and translation geometric operations and virtual display (VR) selection of the region of interest (VOI) clipping operation under three-dimensional ultrasound. SUMMARY

[0007] Therefore, the embodiments of the present application provide an ultrasound imaging method and an ultrasound imaging device.

[0008] The technical scheme of the embodiments of the present application is implemented as follows:

[0009] In one aspect, the embodiments of the present application provide an ultrasound imaging method applied to an ultrasound imaging device, and the method comprises:

[0010] acquiring three-dimensional volume data of a first to-be-measured tissue; identifying three-dimensional volume data of a fetal spine and three-dimensional volume data of a fetal rib from the three-dimensional volume data of the first to-be-measured tissue; rendering the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetal rib to obtain a three-dimensional ultrasound image of a fetal rib structure; marking the fetal rib in the three-dimensional ultrasound image of the fetal rib structure; and outputting the three-dimensional ultrasound image of the fetal rib structure after marking.

[0011] In one aspect, the embodiments of the present application provide an ultrasound imaging method applied to an ultrasound imaging device, and the method comprises: acquiring three-dimensional volume data of a second to-be-measured tissue; identifying three-dimensional volume data of a conus medullaris and three-dimensional volume data of a lumbar vertebra from the three-dimensional volume data of the second to-be-measured tissue; rendering the three-dimensional volume data of the conus medullaris and the three-dimensional volume data of the lumbar vertebra to obtain a three-dimensional ultrasound image of a vertebra structure; marking the conus medullaris in the three-dimensional ultrasound image of the vertebra structure; and outputting the three-dimensional ultrasound image of the vertebra structure after marking.

[0012] In an aspect, an embodiment of the present application provides an ultrasonic imaging method applied to an ultrasonic imaging device, the method comprising: acquiring three-dimensional volume data of a fetus; identifying three-dimensional volume data of a fetal rib from the three-dimensional volume data of the fetus based on a feature of the fetal rib; obtaining a first plane or a first curved surface that passes through at least two ribs in the three-dimensional volume data of the fetal rib and is parallel to or coincides with an arrangement surface of a plurality of fetal ribs in the three-dimensional volume data of the fetal rib and / or obtaining a second plane or a second curved surface that passes through at least one rib in the three-dimensional volume data of the fetal rib and intersects with the arrangement surface of the plurality of fetal ribs in the three-dimensional volume data of the fetal rib according to the identified three-dimensional volume data of the fetal rib; obtaining an image on the first plane or the first curved surface and / or obtaining an image on the second plane or the second curved surface according to the identified three-dimensional volume data of the fetal rib; displaying the image on the first plane or the first curved surface as a two-dimensional image and / or displaying the image on the second plane or the second curved surface as a two-dimensional image.

[0013] In an aspect, an embodiment of the present application provides an ultrasonic imaging method applied to an ultrasonic imaging device, the method comprising: acquiring three-dimensional volume data of a fetus; identifying a conus medullaris region from the three-dimensional volume data of the fetus based on a feature of the conus medullaris of the fetus; determining a position of the conus medullaris region according to the identified conus medullaris region; and displaying the position of the conus medullaris region.

[0014] In an aspect, an embodiment of the present application provides an ultrasonic imaging device, comprising:

[0015] a probe;

[0016] a transmitting circuit configured to stimulate the probe to emit ultrasonic waves to a first to-be-measured tissue;

[0017] a receiving circuit configured to receive, by the probe, ultrasonic echoes returned from the first to-be-measured tissue to obtain an ultrasonic echo signal;

[0018] a processor configured to process the ultrasonic echo signal to obtain a three-dimensional ultrasonic image of a marked fetal rib structure;

[0019] a display configured to display the three-dimensional ultrasonic image of the marked fetal rib structure;

[0020] The processor is further configured to perform the following steps:

[0021] acquire three-dimensional volume data of the first to-be-measured tissue according to the ultrasonic echo information;

[0022] identify three-dimensional volume data of a fetal spine and three-dimensional volume data of a fetal rib from the three-dimensional volume data of the first to-be-measured tissue;

[0023] rendering the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetal rib to obtain a three-dimensional ultrasound image of the fetal rib structure;

[0024] labeling the fetal rib in the three-dimensional ultrasound image of the fetal rib structure;

[0025] outputting the labeled three-dimensional ultrasound image of the fetal rib structure.

[0026] In one aspect, the embodiments of the present application provide an ultrasonic imaging device, comprising:

[0027] a probe;

[0028] a transmitting circuit, which excites the probe to emit ultrasonic waves to a second to-be-measured tissue;

[0029] a receiving circuit, which receives ultrasonic echoes returned from the second to-be-measured tissue through the probe to obtain an ultrasonic echo signal;

[0030] a processor, which processes the ultrasonic echo signal to obtain a three-dimensional ultrasound image of a labeled vertebra structure;

[0031] a display, which displays the three-dimensional ultrasound image of the labeled vertebra structure;

[0032] The processor further performs the following steps:

[0033] acquiring three-dimensional volume data of the second to-be-measured tissue according to the ultrasonic echo information;

[0034] identifying three-dimensional volume data of a spinal cord cone and three-dimensional volume data of a lumbar vertebra from the second three-dimensional volume data;

[0035] rendering the three-dimensional volume data of the spinal cord cone and the three-dimensional volume data of the lumbar vertebra to obtain a three-dimensional ultrasound image of the vertebra structure;

[0036] labeling the spinal cord cone in the three-dimensional ultrasound image of the vertebra structure;

[0037] outputting the labeled three-dimensional ultrasound image of the vertebra structure.

[0038] In the embodiments of the present application, the three-dimensional data of the fetal spine and the three-dimensional data of the fetal rib are identified from the three-dimensional data of the first to-be-detected tissue, or the three-dimensional data of the spinal cord cone and the three-dimensional data of the lumbar vertebra are identified from the three-dimensional data of the second to-be-detected tissue, the identified three-dimensional data is rendered to obtain a three-dimensional ultrasound image of the fetal rib structure or a three-dimensional ultrasound image of the spine, and the fetal rib or the spinal cord cone is marked in the displayed three-dimensional ultrasound image; in this way, the position of the fetal rib or the spinal cord cone is identified and detected from the three-dimensional volume data, the number of the fetal rib is automatically counted and the relative position of the spinal cord cone and the lumbar vertebra is calculated, the workflow of the fetal rib or the spinal cord cone position examination is simplified, the examination efficiency is improved, the doctor is liberated from complex and time-consuming operations, and the dependence of the fetal rib or the spinal cord cone position examination on the skill of the examiner is reduced, and the examination efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 Structure diagram of an ultrasound imaging device provided by the embodiments of the present application Figure 1

[0040] Figure 2 Flowchart of an ultrasound imaging method provided by the embodiments of the present application Figure 1

[0041] Figure 3 Flowchart of an ultrasound imaging method provided by the embodiments of the present application Figure 2

[0042] Figure 4 Flowchart of an ultrasound imaging method provided by the embodiments of the present application Figure 3

[0043] Figure 5 Flowchart of an ultrasound imaging method provided by the embodiments of the present application Figure 4

[0044] Figure 6 Flowchart of an ultrasound imaging method provided by the embodiments of the present application Figure 5

[0045] Figure 7 Diagram of a fetal rib

[0046] Figure 8 Diagram of manually drawing an anatomical trajectory in the related art

[0047] Figure 9 Diagram of a spinal cord cone

[0048] Figure 10 Composition structure diagram of an ultrasound imaging device provided by the embodiments of the present application Figure 2

[0049] ​​​​​​​Figure 11 Fig. 11 is a schematic diagram of the arrangement order of the fetal ribs in the fetal rib structure;

[0050] Figure 12 Fig. 12 is a schematic diagram of the straightening direction of the fetal ribs and the spine in the fetal rib structure in the embodiments of the present application;

[0051] Figure 13 Fig. 13 is a schematic diagram of the cross-sectional display of all the ribs in the embodiments of the present application;

[0052] Figure 14 Fig. 14 is a schematic diagram of the cross-sectional display of the specified ribs in the embodiments of the present application;

[0053] Figure 15 Fig. 15 is a schematic diagram of the three-dimensional skeleton effect of the fetal rib structure in the embodiments of the present application;

[0054] Figure 16 Fig. 16 is a schematic diagram of the annotation effect of the spinal cord cone in the VR image in the embodiments of the present application. DETAILED DESCRIPTION

[0055] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are only used to explain the present application and are not configured to limit the present application. In addition, the embodiments provided below are configured to implement part of the present application, and the technical solutions described in the embodiments of the present application can be implemented in any combination manner without conflict.

[0056] Figure 1 Fig. 1 is a schematic diagram of the structural block diagram of the ultrasound imaging device 10 in the embodiments of the present application. The ultrasound imaging device 10 can include a probe 100, a transmitting circuit 101, a transmitting / receiving selection switch 102, a receiving circuit 103, a beam synthesis circuit 104, a processor 105, and a display 106. The transmitting circuit 101 can excite the probe 100 to transmit ultrasound waves to a target object. The receiving circuit 103 can receive the ultrasound echoes returned from the target object through the probe 100, thereby obtaining an ultrasound echo signal. The ultrasound echo signal is sent to the processor 105 after being subjected to beam synthesis processing by the beam synthesis circuit 104. The processor 105 processes the ultrasound echo signal to obtain an ultrasound image of the target object. The ultrasound image obtained by the processor 105 can be stored in a memory 107. These ultrasound images can be displayed on the display 106.

[0057] The target object includes at least one of a first to-be-measured tissue and a second to-be-measured tissue. The first to-be-measured tissue includes a fetal rib structure, and the second to-be-measured tissue includes a spine.

[0058] The embodiments of the present application provide an ultrasound imaging method, applied to Figure 1The ultrasound imaging device shown, such as Figure 2 The method shown includes:

[0059] S201, obtaining three-dimensional body data of a first to-be-measured tissue;

[0060] The first to-be-measured tissue includes: a fetal rib structure and a tissue other than the fetal rib structure, such as an amniotic fluid region, a placenta, a uterine wall, and the like.

[0061] A doctor can perform a scan on a pregnant woman through a probe to obtain the three-dimensional body data of the first to-be-measured tissue.

[0062] Here, the fetal rib structure belongs to a high echo region and is displayed as high gray scale in an ultrasound image.

[0063] S202, identifying three-dimensional body data of a fetal spine and three-dimensional body data of a fetal rib from the three-dimensional body data of the first to-be-measured tissue;

[0064] Based on the three-dimensional body data of the first to-be-measured tissue obtained in S201, three-dimensional body data of a fetal spine and three-dimensional body data of a fetal rib are identified from the obtained three-dimensional body data. Here, identifying the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib includes at least one of the following two identification methods:

[0065] Identification method one: the three-dimensional body data of the fetal rib structure is first identified from the three-dimensional body data of the first to-be-measured tissue as a whole, and then the three-dimensional body data of the fetal rib in the fetal rib structure is identified from the three-dimensional body data of the fetal rib structure.

[0066] Identification method two: the three-dimensional body data of the fetal rib is directly identified from the three-dimensional body data of the first to-be-measured tissue as an identification object.

[0067] In identification method one, the three-dimensional body data of the fetal rib structure is identified from the three-dimensional body data of the first to-be-measured tissue; the three-dimensional body data of the fetal rib structure is segmented from the three-dimensional body data of the first to-be-measured tissue; and the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib are identified from the three-dimensional body data of the fetal rib structure.

[0068] In identification method two, based on a first rib detection model, different fetal ribs or a fetal spine are taken as different identification objects, and the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib are identified from the three-dimensional body data of the first to-be-measured tissue.

[0069] S203, rendering the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib to obtain a three-dimensional ultrasound image of the fetal rib structure.

[0070] After the three-dimensional data of the fetal rib and the three-dimensional data of the spine included in the first to-be-detected tissue are identified in S202, the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are three-dimensionally rendered to obtain a three-dimensional ultrasound image of the fetal rib structure.

[0071] Here, when the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are three-dimensionally rendered, all data in the three-dimensional data of the first to-be-detected tissue except the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are emptied, the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are stereoscopic ray perspective rendered to form a three-dimensional ultrasound image of the fetal rib structure.

[0072] It should be noted that the rendering manner can be various, and the embodiment of the present application does not limit the rendering manner.

[0073] S204, marking the fetal rib in the three-dimensional ultrasound image of the fetal rib structure;

[0074] According to the position of the three-dimensional data of the fetal rib in the three-dimensional data of the first to-be-detected tissue identified in S202, the position of the three-dimensional data of the fetal rib in the three-dimensional ultrasound image of the fetal rib structure in S203 is determined, and the fetal rib in the fetal rib structure is marked according to the determined position of the fetal rib to obtain a three-dimensional ultrasound image of the fetal rib structure after marking.

[0075] When the fetal rib is marked, the rib identifier of one or more fetal ribs can be marked: for example, T1-T12, wherein T1 to T12 represent the first to twelfth fetal ribs, respectively.

[0076] In actual application, when the fetal rib is marked, the fetal rib structure can be judged according to the shape feature, gray feature and other rib features of the fetal rib to determine whether the fetal rib structure includes an abnormal fetal rib with abnormal shape or lesion. If it is judged that the fetal rib structure includes an abnormal fetal rib, the abnormal fetal rib can be marked by an abnormal identifier, and the abnormal information of the abnormal fetal rib relative to the normal fetal rib is determined.

[0077] S205, outputting the three-dimensional ultrasound image of the fetal rib structure after marking.

[0078] At this time, the content displayed on the display of the ultrasound imaging device is the three-dimensional ultrasound image of the fetal rib structure with the fetal rib marked, so that the user can intuitively view the fetal rib structure of the fetus and distinguish the fetal rib in the fetal rib structure.

[0079] In an embodiment, the method further comprises: straightening the three-dimensional volume data of the fetal rib to obtain straightened rib three-dimensional volume data; straightening the three-dimensional volume data of the fetal spine to obtain straightened spine three-dimensional volume data; performing plane fitting on the straightened rib in the straightened rib three-dimensional volume data and the straightened fetal spine in the straightened spine three-dimensional volume data to obtain a first plane; and obtaining image data on the first plane according to the straightened rib three-dimensional volume data and the straightened spine three-dimensional volume data to obtain a coronal plane image of the fetal rib structure.

[0080] After the three-dimensional volume data of the fetal rib is determined, the fetal rib is straightened according to the three-dimensional volume data of the fetal rib, so that the curved fetal rib is straightened to obtain a straightened rib. After the three-dimensional volume data of the fetal spine is determined, the fetal rib is straightened according to the three-dimensional volume data of the fetal spine, so that the curved fetal spine is straightened to obtain a straightened fetal spine. At this time, the first plane is determined by solving a mathematical equation or a plane fitting method such as least squares method or Hough transform, and the first plane is a coronal plane of longitudinal axes of the straightened rib and the straightened fetal spine. The three-dimensional volume data of each fetal rib and the three-dimensional volume data of the fetal spine on the first plane are obtained to obtain a coronal plane of the fetal rib structure.

[0081] In actual application, after the first plane is determined, image data of each pixel point on the first plane is obtained according to the three-dimensional volume data of each straightened rib and the three-dimensional volume data of the straightened fetal spine, so that a coronal plane image of the fetal rib structure after straightening is obtained.

[0082] In an embodiment, the method further comprises: determining a target fetal rib from the ribs of the fetal rib structure; straightening the three-dimensional volume data of the target fetal rib to obtain straightened target fetal rib three-dimensional volume data; straightening the three-dimensional volume data of the fetal spine to obtain straightened spine three-dimensional volume data; taking a plane in which the straightened target rib is located and perpendicular to the straightened fetal spine as a second plane; and obtaining image data on the second plane according to the straightened target fetal rib three-dimensional volume data and the straightened spine three-dimensional volume data to obtain a cross section of the target fetal rib.

[0083] Here, the target fetal rib can be all fetal ribs in the fetal rib structure or part of the fetal ribs in the fetal rib structure. The target fetal rib can be selected by a user or determined automatically by the system. For example, the three-dimensional volume data of the fetal rib structure is displayed on a display, and the target fetal rib is determined according to a rib identifier manually input by the user or based on rib selection of the user on the fetal rib structure.

[0084] When the target fetal rib is determined, the target fetal rib is straightened according to the three-dimensional volume data of the target fetal rib, so that the curved target fetal rib is straightened, and a straightened target rib is obtained. After the three-dimensional volume data of the fetal spine is determined, the fetal spine is straightened according to the three-dimensional volume data of the fetal spine, so that the curved fetal spine is straightened, and a straightened fetal spine is obtained. At this time, the second plane in which the target fetal rib is located is determined by the way of determining a plane through two straight lines, wherein the second plane is perpendicular to the longitudinal axis of the straightened fetal spine and is coplanar with the longitudinal axis of the straightened target rib. After the second plane is determined, the three-dimensional volume data of the target fetal rib and the three-dimensional volume data of the fetal spine on the second plane are obtained, and a cross section of the fetal rib structure is obtained.

[0085] In this document, the "longitudinal axis" of the fetal rib, the fetal spine, the fetal lumbar vertebrae or other tissues can refer to an axis along the length direction of the tissue, which can be a central axis or other axis along the length direction.

[0086] In actual application, after the second plane is determined, the three-dimensional volume data of the target straightened rib and the three-dimensional volume data of the straightened fetal spine on the second plane are obtained, and a cross section of the fetal rib structure after straightening is obtained.

[0087] It should be noted that the number of cross sections obtained corresponds to the number of fetal ribs in the target fetal rib, and different fetal ribs correspond to different cross sections.

[0088] In an embodiment, straightening the straightening object includes: determining a longitudinal axis of the straightening object; performing equal-interval sampling on the straightening object according to the longitudinal axis of the straightening object to obtain an equal-interval cross section sequence; the straightening object includes different fetal ribs or the fetal spine; and reconstructing the equal-interval cross section sequence along a straight line based on the equal interval.

[0089] The straightening of the straightening object includes extraction of the longitudinal axis and straightening reconstruction. The extraction method of the longitudinal axis can be: a tracking-based longitudinal axis extraction algorithm, a model-based multi-scale longitudinal axis extraction algorithm, a morphology-based longitudinal axis extraction method, a region growing-based center line extraction method, a three-dimensional geometric moment-based method, and a method of positioning a center line by machine learning.

[0090] For example, the tracking-based longitudinal axis extraction algorithm is a semi-automatic algorithm, which generates a cross section perpendicular to the tracking direction in the process of tracking the straightening object based on the initial key point and the terminal point provided by the user, and accurately calculates the center point of the straightening object in the cross section by using the maximum likelihood method and the centroid method. After the tracking process is completed, the center point sequence is interpolated and fitted to obtain the longitudinal axis of the straightening object.

[0091] For example, in the extraction of the longitudinal axis of a straightened object based on a model multi-scale longitudinal axis extraction algorithm, the local straightened object is approximated as a tubular structure, the center of gravity of the tubular structure obtained by calculating the geometric moment is taken as the center of the local straightened object, the eigenvalues of the Hessian matrix corresponding to the body data after multi-scale Gaussian filtering are analyzed, the local straightened object is enhanced, and the direction of the longitudinal axis of the straightened object is estimated according to the eigenvector corresponding to the minimum eigenvalue of the Hessian matrix.

[0092] After the longitudinal axis of the straightened object is determined, the straightened object is straightened and reconstructed. The rib longitudinal axis is first sampled at equal intervals to obtain equal-interval center points. On the basis of the equal-interval center points, a sequence of equal-interval cross sections perpendicular to the longitudinal axis direction of the straightened object is generated. The obtained cross section sequence is stacked together, and the equal-interval cross section sequence is three-dimensionally reconstructed to obtain the straightened straightened object.

[0093] In an embodiment, the method further comprises: segmenting the three-dimensional body data of the fetal rib and the three-dimensional body data of the fetal spine from the three-dimensional body data of the first to-be-detected tissue; performing binaryzation processing on the three-dimensional body data of the fetal rib and the three-dimensional body data of the fetal spine to obtain reconstructed three-dimensional body data; and rendering the reconstructed three-dimensional body data to obtain a three-dimensional skeleton of the fetal rib structure.

[0094] After the three-dimensional body data of the fetal rib and the three-dimensional body data of the fetal spine are identified, binaryzation processing is performed on the three-dimensional body data of the fetal rib and the three-dimensional body data of the spine to obtain reconstructed three-dimensional body data, for example, the gray scale of the fetal rib structure is set to 1, and the body data outside the fetal rib structure is set to 0. The reconstructed three-dimensional body data is rendered by volume rendering or surface rendering to obtain a three-dimensional skeleton including the fetal rib and the spine.

[0095] In actual application, when the spine is identified, only the three-dimensional body data of the vertebrae connected with the ribs in the spine can be identified. Thus, a rib structure including each fetal rib and vertebra is output.

[0096] The ultrasonic imaging method provided in the embodiments of the present application automatically images a stereoscopic virtual display (Virtual Reality, VR) image of a fetal rib structure after three-dimensional body data of a fetus is acquired, and marks the fetal ribs in the fetal rib structure on the stereoscopic VR image, thereby greatly simplifying the workflow of fetal rib examination. Further, the cross sections and coronal sections of the rib skeleton after being straightened are automatically imaged. The doctor is liberated from tedious and complicated manual operation, the dependence on the doctor's technology is reduced, and the examination efficiency is improved.

[0097] The embodiment of the present application provides an ultrasonic imaging method, which is applied to Figure 1 The ultrasonic imaging device is shown in Fig. 1, and the method is shown in Fig. 2. Figure 3 The method comprises the following steps.

[0098] S301, acquiring three-dimensional volume data of a first to-be-detected tissue.

[0099] The first to-be-detected tissue comprises a fetal rib structure and a tissue other than the fetal rib structure, such as an amniotic fluid region, a placenta, a uterine wall and the like.

[0100] A doctor can perform scanning on a pregnant woman through a probe to acquire the three-dimensional volume data of the first to-be-detected tissue.

[0101] Here, the tissue other than the fetal rib structure belongs to a low echo region and is low gray data in the three-dimensional volume data, and the fetal rib structure belongs to a high echo region and is high gray data in the three-dimensional volume data.

[0102] S302, based on a first rib detection model, taking different fetal ribs or a fetal spine as different recognition objects, respectively, recognizing three-dimensional volume data of the fetal spine and three-dimensional volume data of the fetal ribs from the three-dimensional volume data of the first to-be-detected tissue.

[0103] The algorithm used by the first rib detection model can be a machine learning method. The first rib detection model takes the three-dimensional volume data of each fetal rib and the three-dimensional volume data of the spine as training samples, learns the training samples through the machine learning method, and trains the first rib detection model through the training samples. The first rib detection model learns the volume data features of the fetal ribs and the volume data features of the spine after training, wherein the volume data features can comprise principal component analysis (PCA) features, linear discriminant analysis (LDA) features, Harr features, texture features and the like.

[0104] When the first rib detection model receives the three-dimensional volume data of the first to-be-detected tissue, the three-dimensional volume data of the fetal ribs and the three-dimensional volume data of the spine included in the received three-dimensional volume data are recognized according to the learned volume data features of the fetal ribs and the volume data features of the spine.

[0105] In the training samples, the fetal ribs or the spine are taken as targets, the targets are labeled, and the category of each labeled target is indicated. The labeling can be performed through a region of interest (ROI) box containing the target, or the labeling can be performed through a mask (Mask) for accurate segmentation of the target.

[0106] The algorithm used by the first rib detection model can be an image segmentation algorithm. The three-dimensional body data input into the first rib detection model is binarized and segmented, and after morphological, contour extraction, connected domain and other operations, a plurality of candidate regions are obtained. The probability of each candidate region being a fetal rib or a spine is determined according to the characteristics of the body data of each candidate region. The candidate region with the highest probability is selected as the region corresponding to the rib or the spine, and the three-dimensional body data of the selected region is the three-dimensional body data of the fetal rib or the spine.

[0107] In actual applications, the image segmentation algorithm used by the first rib detection model can also be one or more of the following: Level Set, Graph Cut, Snake, Random walker, active contour model algorithm, active shape model algorithm, active appearance model algorithm, and image segmentation algorithms in deep learning such as Fully Convolutional Networks (FCN) and UNet.

[0108] The algorithm used by the first rib detection model can also be a template matching algorithm. A template of the three-dimensional body data of the fetal rib or the three-dimensional body data of the spine is established. The input three-dimensional body data is binarized and segmented, and after morphological, contour extraction, connected domain and other operations, a plurality of candidate regions are obtained. All candidate regions in the body data are traversed according to the established template, and the similarity of all candidate regions and the template is determined. The candidate region with the highest similarity is selected as the target region corresponding to the fetal rib or the spine, and the three-dimensional body data of the target region is the three-dimensional body data of the fetal rib or the three-dimensional body data of the spine.

[0109] It should be noted that the first rib model can directly count the number of fetal ribs included in the first test tissue when identifying different fetal ribs or fetal spines.

[0110] S303, rendering the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib to obtain a three-dimensional ultrasound image of the fetal rib structure;

[0111] After identifying the three-dimensional body data of the fetal rib and the three-dimensional body data of the spine included in the first test tissue in S302, the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib are rendered in three dimensions to obtain a three-dimensional ultrasound image of the fetal rib structure.

[0112] Here, when three-dimensionally rendering the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetal rib, all data in the three-dimensional volume data of the first to-be-measured tissue except the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetal rib are emptied, the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetal rib are stereoscopic ray perspective rendered, and a three-dimensional ultrasound image of the fetal rib structure is formed.

[0113] It should be noted that the rendering manner can be various, and the embodiment of the present application does not limit the rendering manner.

[0114] S304, marking the fetal rib in the three-dimensional ultrasound image of the fetal rib structure;

[0115] According to the position of the three-dimensional data of the fetal rib in the three-dimensional volume data of the first to-be-measured tissue identified in S302, the position of the three-dimensional data of the fetal rib in the three-dimensional ultrasound image of the fetal rib structure in S203 is determined, and the fetal rib in the fetal rib structure is marked according to the determined position of the fetal rib, to obtain a three-dimensional ultrasound image of the fetal rib structure after marking.

[0116] When the fetal rib is marked, the rib identifier of one or more fetal ribs can be marked: for example, T1-T12, wherein T1 to T12 represent the first to twelfth fetal ribs, respectively.

[0117] S305, outputting the three-dimensional ultrasound image of the fetal rib structure after marking.

[0118] At this time, the content displayed on the display of the ultrasound imaging device is the three-dimensional ultrasound image of the fetal rib structure with the fetal rib marked, so that the user can intuitively view the fetal rib structure of the fetus and distinguish the fetal rib in the fetal rib structure.

[0119] The ultrasound imaging method provided by the embodiment of the present application, after obtaining the three-dimensional volume data of the fetus, takes the fetal rib and the spine as different recognition objects, recognizes the fetal rib and the fetal spine, and automatically counts the number of fetal ribs included in the first to-be-measured tissue, and automatically images the stereoscopic VR image of the fetal rib structure, thereby greatly simplifying the workflow of fetal rib examination, liberating the doctor from tedious and complex manual operation, reducing the dependence on the doctor's technology, and reducing the misdiagnosis and missed diagnosis rate.

[0120] The embodiment of the present application provides an ultrasound imaging method, applied to Figure 1 The ultrasound imaging device is shown as in Figure 4 The method includes:

[0121] S401, obtaining three-dimensional volume data of a first to-be-measured tissue;

[0122] The first to-be-detected tissue includes: a fetal rib structure and tissue other than the fetal rib structure, such as an amniotic fluid region, a placenta, a uterine wall, and the like.

[0123] A doctor can perform a scan on a pregnant woman through a probe to obtain three-dimensional volume data of the first to-be-detected tissue.

[0124] Here, the tissue other than the fetal rib structure belongs to a low echo region and is displayed as low gray scale in an ultrasound image, and the fetal rib structure belongs to a high echo region and is displayed as high gray scale in the ultrasound image.

[0125] S402, identify three-dimensional volume data of the fetal rib structure from the three-dimensional volume data of the first to-be-detected tissue;

[0126] When identifying the three-dimensional volume data of the fetal rib structure from the three-dimensional volume data of the first to-be-detected tissue, at least one of the following three structure identification methods can be included:

[0127] The first structure identification method includes: displaying a three-dimensional image corresponding to the three-dimensional volume data of the first to-be-detected tissue; receiving a first input operation based on the three-dimensional image corresponding to the three-dimensional volume data of the first to-be-detected tissue; determining a mark point corresponding to the first input operation; and identifying the three-dimensional volume data of the fetal rib structure from the three-dimensional volume data of the first to-be-detected tissue according to the coordinates of the mark point.

[0128] After obtaining the three-dimensional volume data of the first to-be-detected tissue, a three-dimensional image corresponding to the three-dimensional volume data of the first to-be-detected tissue is displayed on a display. A user performs a first input operation on the fetal rib structure in the three-dimensional image corresponding to the three-dimensional volume data of the first to-be-detected tissue displayed on the display by selecting a mark point, drawing a mark line, or the like through a trackball, a touch screen, or the like, to inform an ultrasound imaging device of the position of the fetal rib structure in space. Here, the mark line is composed of a plurality of continuous mark points. After receiving the first input operation, the ultrasound imaging device determines the position of the fetal rib structure in the three-dimensional volume data of the first to-be-detected tissue through the coordinates of the mark point corresponding to the first input operation or the mark points constituting the mark line.

[0129] For example, the user selects some points on the endpoints of each fetal rib or intermittently on the boundaries of the fetal rib structure, the ultrasound imaging device takes the points selected by the user as mark points, roughly draws a center line of the fetal rib structure or outlines a boundary line of the fetal rib structure according to the coordinates of the mark points, determines the position of the fetal rib structure according to the determined center line or boundary line, and thus obtains the three-dimensional volume data of the fetal rib structure.

[0130] In the second structure recognition mode, a three-dimensional image corresponding to the three-dimensional data of the first to-be-detected tissue is displayed; a second input operation is received based on the three-dimensional image corresponding to the three-dimensional data of the first to-be-detected tissue; a first seed region corresponding to the second input operation is determined; the first seed region is located in a three-dimensional image region corresponding to the fetal rib structure; a first pixel feature of the three-dimensional data of the first seed region is determined; and three-dimensional data of the fetal rib structure is recognized from the three-dimensional data of the first to-be-detected tissue according to the first pixel feature.

[0131] Here, after obtaining the three-dimensional data of the first to-be-detected tissue, a three-dimensional image corresponding to the three-dimensional data of the first to-be-detected tissue is displayed on the display. A user performs a second input operation on the three-dimensional image corresponding to the three-dimensional data of the first to-be-detected tissue displayed on the display based on a tool such as a trackball or a touch screen. After the ultrasonic imaging device receives the second input operation, a region corresponding to the second input operation is determined as a first seed region, the three-dimensional data of the first seed region is taken as prior data, an edge gradient, a grayscale, or other pixel features of the prior data are obtained as first pixel features, and three-dimensional data of the fetal rib structure is recognized from the three-dimensional data of the first to-be-detected tissue according to the first pixel features of the prior data.

[0132] The method for recognizing three-dimensional data of the fetal rib structure from the three-dimensional data of the first to-be-detected tissue according to the pixel features of the prior data can include one or more image processing methods such as template matching, image feature extraction, edge extraction, and morphological operation, one or more image segmentation methods such as a graphcut algorithm, a grabcut algorithm, a level set method, an active contour model algorithm, an active shape model algorithm, a seed region growing method, and a region segmentation merging method, and one or more machine learning methods such as a deep learning method, a support vector machine, an adaboost, and a random forest algorithm.

[0133] For example, when the three-dimensional data of the fetal rib structure is recognized from the three-dimensional data of the first to-be-detected tissue by template matching, the prior data is taken as a template, the edge gradient, the grayscale, or other pixel features of the template are calculated, the three-dimensional data of the first to-be-detected tissue is traversed through the template to find an optimal solution with the smallest difference from the pixel features of the template, and the recognition of the fetal rib structure is realized.

[0134] For example, when the three-dimensional data of the fetal rib structure is identified from the three-dimensional data of the first to-be-measured tissue by the seed region growing method, a seed region is determined in the fetal rib structure region, the pixels of the seed region are taken as seed pixels, then according to the first pixel feature of the seed pixels, the pixels meeting the first pixel feature are merged into the seed region from the three-dimensional data of the first to-be-measured tissue, the newly added pixels are taken as new seed pixels for continuous merging, until no new pixel meeting the condition is found, and finally the three-dimensional data of the fetal rib structure is identified.

[0135] Here, the workflow through certain user interaction operation acquires certain prior data in the fetal rib structure body data as known information, and the difficulty of identification of the fetal rib structure is reduced through the first pixel feature of the known information.

[0136] In the third structure identification mode, at least two first candidate regions are determined from the three-dimensional data of the first to-be-measured tissue, and the body data feature of the three-dimensional data of each first candidate region is acquired; according to the body data feature of each first candidate region, the first matching degree of each first candidate region with the fetal rib structure is determined; the first candidate region with the highest first matching degree is determined as the target region corresponding to the fetal rib structure; and the three-dimensional data of the target region corresponding to the fetal rib structure is taken as the three-dimensional data of the fetal rib structure.

[0137] Here, the rib structure detection model can receive the three-dimensional data of the first to-be-measured tissue, determine at least two first candidate regions from the three-dimensional data of the first to-be-measured tissue, acquire the body data feature of the three-dimensional data of each first candidate region, determine the first matching degree of each first candidate region with the fetal rib structure according to the body data feature of each first candidate region, determine the first candidate region with the highest first matching degree as the target region corresponding to the fetal rib structure, and take the three-dimensional data of the target region corresponding to the fetal rib structure as the three-dimensional data of the fetal rib structure.

[0138] The algorithm used by the rib structure detection model can be a machine learning method, the rib structure detection model takes the three-dimensional data of each fetal rib structure as a training sample, learns the training sample by the machine learning method, and trains the rib structure detection model through the training sample. The trained rib structure detection model learns the body data feature of the fetal rib structure, wherein the body data feature can include PCA feature, LDA feature, Harr feature, texture feature, etc.

[0139] When the rib structure detection model receives the three-dimensional volume data of the first tissue to be tested, it identifies the three-dimensional volume data of the fetal rib structure included in the received three-dimensional volume data based on the learned volume data features of the fetal rib structure.

[0140] In the training samples, the fetal rib structure is used as the target, and the target is labeled, with the category of each labeled target indicated. Labeling can be done using a region of interest (ROI) bounding box containing the target, or using a mask that precisely segments the target.

[0141] The algorithm used in the rib structure detection model can be an image segmentation algorithm. The three-dimensional volume data of the input rib structure detection model is binarized and segmented. After performing morphological, contour extraction, and connected domain operations, multiple first candidate regions are obtained. The probability of each first candidate region being a fetal rib structure is determined based on the volume data features of each first candidate region. The first candidate region with the highest probability is selected as the target region corresponding to the fetal rib structure, and the three-dimensional volume data of the selected target region is the three-dimensional volume data of the fetal rib structure.

[0142] In practical applications, the image segmentation algorithm used in the rib structure detection model can also be one or more of the following: LevelSet, Graph Cut, Snake, Random Walker, Active Contour Model, Active Shape Model, Active Appearance Model, as well as image segmentation algorithms in deep learning such as FCN and UNet.

[0143] The rib structure detection model can also employ a template matching algorithm to establish a template for the three-dimensional volumetric data of the fetal rib structure. The rib structure detection model performs binarization segmentation on the input three-dimensional volumetric data and then performs morphological, contour extraction, and connected domain operations to obtain multiple first candidate regions. Based on the established template, it iterates through all first candidate regions in the volumetric data and determines the similarity between each first candidate region and the template. The first candidate region with the highest similarity is selected as the target region corresponding to the fetal rib structure, and the three-dimensional volumetric data of the target region is the three-dimensional volumetric data of the fetal rib structure.

[0144] It should be noted that the specific identification method for recognizing the three-dimensional volume data of the fetal rib structure from the three-dimensional volume data of the first tissue to be tested in this embodiment of the application is not limited in any way.

[0145] S403. The three-dimensional volume data of the fetal rib structure is segmented from the three-dimensional volume data of the first tissue to be tested;

[0146] After the three-dimensional data of the fetal rib structure is recognized, the three-dimensional data of the fetal rib structure is segmented from the three-dimensional data of the first tissue to be tested according to the position of the three-dimensional data of the fetal rib structure in the three-dimensional data of the first tissue to be tested.

[0147] In actual application, S403 and S404 can be implemented at the same time, that is, the three-dimensional data of the fetal rib structure is segmented from the three-dimensional data of the first tissue to be tested while the three-dimensional data of the fetal rib structure is recognized.

[0148] S404, three-dimensional data of a fetal spine and three-dimensional data of a fetal rib are recognized from the three-dimensional data of the fetal rib structure;

[0149] In S403, after the three-dimensional data of the fetal rib structure is segmented from the three-dimensional data of the first tissue to be tested, the three-dimensional data of the fetal rib and the three-dimensional data of the spine are recognized from the three-dimensional data of the fetal rib structure, which can include at least one of the following two rib recognition methods:

[0150] Rib recognition method one: based on the shape characteristics of the fetal spine and the shape characteristics of the fetal rib, the fetal spine and the fetal rib included in the three-dimensional data of the fetal rib structure are located; a three-dimensional image corresponding to the three-dimensional data of the fetal rib structure is displayed; a third input operation is received based on the three-dimensional image corresponding to the three-dimensional data of the fetal rib structure; a reference fetal rib and a rib identifier corresponding to the reference fetal rib are determined according to the third input operation; the reference fetal rib is at least one of the fetal ribs in the fetal rib structure; the three-dimensional data of the fetal rib is recognized based on the reference fetal rib and the rib identifier corresponding to the reference fetal rib.

[0151] After the three-dimensional data of the fetal rib structure is determined, a three-dimensional image corresponding to the three-dimensional data of the fetal rib structure is displayed on the display. The user performs a third input operation on the three-dimensional image corresponding to the three-dimensional data of the rib structure displayed on the display based on a trackball, a touch screen or the like, to identify part of the fetal rib structure and the rib identifier of the identified fetal rib structure.

[0152] After the ultrasonic imaging device receives the third input operation, the fetal rib identified by the user is determined through the third input operation, the fetal rib identified by the user is taken as a reference fetal rib, the rib identifier input by the user is taken as the rib identifier of the reference fetal rib, and each fetal rib in the fetal rib structure and the rib identifier corresponding to each fetal rib are determined according to the reference fetal rib. Here, the reference fetal rib can include one or more fetal ribs.

[0153] When the ultrasound imaging device determines each of the fetal ribs in the fetal rib structure and the rib label corresponding to each of the fetal ribs according to the reference fetal rib, the fetal rib structure can be determined according to one or more of a gray histogram projection method, a contour extraction algorithm, an edge extraction algorithm, a connected domain method, a blob detection algorithm, a template matching algorithm, an image feature extraction algorithm, and a morphological operation algorithm.

[0154] For example, the user selects T3 and T9 ribs on the three-dimensional image corresponding to the fetal rib structure by using a trackball, a touch screen, or the like, and marks a point on the T3 and T9 ribs or draws a center line of the T3 and T9 ribs, respectively, that is, marks the two ribs as the reference fetal ribs. The number and positions of the fetal ribs are obtained by projecting the coronal plane of the three-dimensional volume data of the fetal rib structure along the Y axis according to the gray histogram projection method, obtaining a pixel statistical graph, and calculating the number of pixel peaks and the positions of the peaks in the pixel statistical graph.

[0155] The second rib detection model can be a machine learning method. The second rib detection model takes the three-dimensional volume data of each of the fetal ribs and the three-dimensional volume data of the spine as training samples, learns the training samples by using the machine learning method, and trains the second rib detection model by using the training samples. The trained second rib detection model learns the volume data features of the fetal ribs and the volume data features of the spine, where the volume data features can include PCA features, LDA features, Harr features, texture features, and the like.

[0156] When the second rib detection model receives the three-dimensional volume data of the fetal rib structure, the three-dimensional volume data of the fetal ribs and the three-dimensional volume data of the spine included in the received three-dimensional volume data are identified according to the learned volume data features of the fetal ribs and the volume data features of the spine.

[0157]

[0158] ​In the training sample, the fetus rib or spine is taken as the target, the target is labeled, and the category of each labeled target is indicated. The labeling can be performed by means of a ROI box containing the target, or by means of a mask precisely segmenting the target.

[0159] The algorithm adopted by the second rib detection model can be an image segmentation algorithm. The three-dimensional body data input into the second rib detection model is subjected to binary segmentation, and morphological, contour extraction, connected domain, and other operations to obtain a plurality of candidate regions. The probability of each candidate region being a fetus rib or spine is determined according to the characteristics of the body data of each candidate region. The candidate region with the highest probability is selected as the region corresponding to the rib or spine, and the three-dimensional body data of the selected region is the three-dimensional body data of the fetus rib or spine.

[0160] In actual application, the image segmentation algorithm adopted by the second rib detection model can also be one or more of the following: Level Set, Graph Cut, Snake, Random walker, active contour model algorithm, active shape model algorithm, active appearance model algorithm, and image segmentation algorithms in deep learning such as Fully Convolutional Networks (FCN) and UNet.

[0161] The algorithm adopted by the second rib detection model can also be a template matching algorithm. A template of the three-dimensional body data of the fetus rib or the three-dimensional body data of the spine is established. The three-dimensional body data input into the second rib detection model is subjected to binary segmentation, and morphological, contour extraction, connected domain, and other operations to obtain a plurality of candidate regions. All candidate regions in the body data are traversed according to the established template, and the similarity of all candidate regions and the template is determined. The candidate region with the highest similarity is selected as the target region corresponding to the fetus rib or spine, and the three-dimensional body data of the target region is the three-dimensional body data of the fetus rib or the three-dimensional body data of the spine.

[0162] The algorithm adopted by the second rib detection model can also be one or more of the following: image edge extraction, histogram image gray projection statistics, image contour extraction, morphological processing, threshold segmentation, blob detection, and other methods to directly calculate the number of fetus ribs in the three-dimensional body data of the fetus rib structure.

[0163] It should be noted that the second rib detection model can directly count the number of fetus ribs included in the fetus rib structure when identifying different fetus ribs or fetus spines.

[0164] S405, rendering the three-dimensional body data of the fetus spine and the three-dimensional body data of the fetus rib to obtain a three-dimensional ultrasound image of the fetus rib structure;

[0165] After the three-dimensional data of the fetal rib and the three-dimensional data of the spine included in the first to-be-measured tissue are identified in S404, the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are three-dimensionally rendered to obtain a three-dimensional ultrasound image of the fetal rib structure.

[0166] Here, when the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are three-dimensionally rendered, all data in the three-dimensional data of the first to-be-measured tissue except the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine are emptied, the three-dimensional ray perspective rendering is performed on the three-dimensional data of the fetal rib and the three-dimensional data of the fetal spine to form the three-dimensional ultrasound image of the fetal rib structure.

[0167] It should be noted that the rendering manner can be various, and the embodiment of the present application does not limit the rendering manner.

[0168] S406, marking the fetal rib in the three-dimensional ultrasound image of the fetal rib structure;

[0169] According to the position of the three-dimensional data of the fetal rib in the three-dimensional data of the first to-be-measured tissue identified in S404, the position of the three-dimensional data of the fetal rib in the three-dimensional ultrasound image of the fetal rib structure in S203 is determined, and the fetal rib in the fetal rib structure is marked according to the determined position of the fetal rib to obtain the three-dimensional ultrasound image of the fetal rib structure after marking.

[0170] When the fetal rib is marked, the rib identifier of one or more fetal ribs can be marked: for example, T1-T12, wherein T1 to T12 respectively represent the first to twelfth fetal ribs.

[0171] S407, outputting the three-dimensional ultrasound image of the fetal rib structure after marking.

[0172] At this time, the content displayed on the display of the ultrasound imaging device is the three-dimensional ultrasound image of the fetal rib structure with the fetal rib marked, so that the user can intuitively view the fetal rib structure of the fetus and distinguish the fetal rib in the fetal rib structure.

[0173] The ultrasound imaging method provided in the embodiments of the present application can automatically or semi-automatically identify, locate and segment the three-dimensional data of the rib structure of the fetus after obtaining the three-dimensional data of the fetus, automatically image the three-dimensional VR image of the rib structure of the fetus, automatically or semi-automatically count the number of the ribs of the fetus, mark the ribs of the fetus in the three-dimensional VR image. Further, the three-dimensional rib skeleton is extracted, and the cross section and the coronal section of all ribs or specified ribs after being straightened are automatically imaged. Thus, the workflow of the rib examination of the fetus is greatly simplified, the doctor is liberated from tedious and complicated manual operation, the dependence on the skill of the doctor is reduced, and the examination efficiency is improved. In addition, the stability of the rib count result and the imaging quality are both in a more optimal state compared with manual operation, and the misdiagnosis and missed diagnosis rates are reduced.

[0174] The embodiments of the present application provide an ultrasound imaging method, applied to Figure 1 The ultrasound imaging device is shown in FIG. 1, and the method is shown in FIG. 2. Figure 5 The method comprises the following steps.

[0175] S501, obtaining three-dimensional data of a second to-be-measured tissue;

[0176] The second to-be-measured tissue comprises the spine of the fetus and the tissue other than the spine of the fetus, such as the amniotic fluid region, the placenta, the uterine wall and the like.

[0177] The doctor can scan the pregnant woman through the probe to obtain the three-dimensional data of the second to-be-measured tissue.

[0178] Here, the rib structure of the fetus belongs to a high echo region and is displayed as high gray scale in the ultrasound image.

[0179] S502, identifying three-dimensional data of the conus medullaris and three-dimensional data of the lumbar vertebrae from the three-dimensional data of the second to-be-measured tissue;

[0180] Based on the three-dimensional data of the second to-be-measured tissue obtained in S201, the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebrae are identified from the obtained three-dimensional data.

[0181] The conus medullaris can be identified from the three-dimensional data of the second to-be-measured tissue through the conus medullaris detection model.

[0182] In an embodiment, the method comprises: determining at least two second candidate regions from the three-dimensional volume data of the second to-be-detected tissue, obtaining a volume data feature of the three-dimensional volume data of each second candidate region; determining a second matching degree of each second candidate region with the fetal spine according to the volume data feature of each second candidate region; determining a second candidate region with the highest second matching degree as a target region corresponding to the conus medullaris; and taking the three-dimensional volume data of the target region corresponding to the conus medullaris as the three-dimensional volume data of the conus medullaris.

[0183] Here, the conus medullaris detection model determines at least two second candidate regions from the three-dimensional volume data of the second to-be-detected tissue, obtains a volume data feature of the three-dimensional volume data of each second candidate region, determines a second matching degree of each second candidate region with the fetal spine according to the volume data feature of each second candidate region, determines a second candidate region with the highest second matching degree as a target region corresponding to the conus medullaris, and takes the three-dimensional volume data of the target region corresponding to the conus medullaris as the three-dimensional volume data of the conus medullaris.

[0184] The algorithm used by the conus medullaris detection model can be a machine learning method. The conus medullaris detection model takes the three-dimensional volume data of the conus medullaris as a training sample, learns the training sample by the machine learning method, and trains the conus medullaris detection model by the training sample. The trained conus medullaris detection model learns the volume data feature of the conus medullaris, wherein the volume data feature can include PCA feature, LDA feature, Harr feature, texture feature, etc.

[0185] When the conus medullaris detection model receives the three-dimensional volume data of the second to-be-detected tissue, it identifies the three-dimensional volume data of the conus medullaris included in the received three-dimensional volume data according to the learned volume data feature of the conus medullaris.

[0186] In the training sample, the target, i.e., the conus medullaris, is labeled. The labeling can be performed by means of a ROI box containing the target, or by means of a mask precisely segmenting the target.

[0187] The algorithm used by the conus medullaris detection model can be an image segmentation algorithm. The three-dimensional volume data input into the conus medullaris detection model is binarized and segmented, and morphological, contour extraction, and connected domain operations are performed to obtain a plurality of second candidate regions. The probability of each second candidate region being the conus medullaris is determined according to the volume data feature of each second candidate region. The first candidate region with the highest probability is selected as the target region corresponding to the conus medullaris, and the three-dimensional volume data of the selected target region is the three-dimensional volume data of the conus medullaris.

[0188] In practical applications, the image segmentation algorithm used by the spinal cord cone detection model can also be one or more of a level set, a graph cut, a snake, a random walker, an active contour model algorithm, an active shape model algorithm, an active appearance model algorithm, and an image segmentation algorithm in deep learning such as FCN and UNet.

[0189] The algorithm used by the spinal cord cone detection model can also be a template matching algorithm, and a template of three-dimensional body data of the spinal cord cone is established. The spinal cord cone detection model performs binary segmentation on the input three-dimensional body data, and obtains a plurality of second candidate regions after performing morphological, contour extraction, and connected domain operations. The template is traversed in all second candidate regions in the body data, the similarity of all second candidate regions and the template is determined, the second candidate region with the highest similarity is selected as the target region corresponding to the spinal cord cone, and the three-dimensional body data of the target region is the three-dimensional body data of the fetal spinal cord cone.

[0190] It should be noted that the specific identification manner of identifying the three-dimensional body data of the spinal cord cone from the three-dimensional body data of the second to-be-detected tissue in the embodiments of the present application is not limited in any way.

[0191] The three-dimensional body data of the lumbar vertebrae is identified in at least one of the following two identification manners:

[0192] Identification manner one: the fetal spine as a whole is first identified as the identification object from the three-dimensional body data of the second to-be-detected tissue to identify the three-dimensional body data of the fetal spine, and then the three-dimensional body data of the lumbar vertebrae is identified from the three-dimensional body data of the fetal spine.

[0193] Identification manner two: the lumbar vertebrae is directly identified as the identification object from the three-dimensional body data of the second to-be-detected tissue.

[0194] In identification manner one, the three-dimensional body data of the fetal spine is identified from the three-dimensional body data of the second to-be-detected tissue; the three-dimensional body data of the fetal spine is segmented from the three-dimensional body data of the second to-be-detected tissue; and the three-dimensional body data of the lumbar vertebrae is identified from the three-dimensional body data of the fetal spine.

[0195] In identification manner two, the three-dimensional body data of the lumbar vertebrae is identified as the identification object from the three-dimensional body data of the second to-be-detected tissue based on a first lumbar vertebra detection model.

[0196] In an embodiment, the identifying the three-dimensional volume data of the lumbar vertebrae from the three-dimensional volume data of the second to-be-detected tissue comprises: determining at least two third candidate regions from the three-dimensional volume data of the second to-be-detected tissue, obtaining a volume data feature of the three-dimensional volume data of each third candidate region; determining a third matching degree of each third candidate region with the lumbar vertebrae in the fetal spine according to the volume data feature of each third candidate region; determining a third candidate region with the highest third matching degree as a target region corresponding to the lumbar vertebrae; and taking the three-dimensional volume data of the target region corresponding to the lumbar vertebrae as the three-dimensional volume data of the lumbar vertebrae.

[0197] Here, the first lumbar vertebra detection model determines at least two third candidate regions from the three-dimensional volume data of the second to-be-detected tissue, obtains a volume data feature of the three-dimensional volume data of each third candidate region, determines a third matching degree of each third candidate region with the lumbar vertebrae in the fetal spine according to the volume data feature of each third candidate region, determines a third candidate region with the highest third matching degree as a target region corresponding to the lumbar vertebrae, and takes the three-dimensional volume data of the target region corresponding to the lumbar vertebrae as the three-dimensional volume data of the lumbar vertebrae.

[0198] The algorithm used by the first lumbar vertebra detection model can be a machine learning method. The first lumbar vertebra detection model takes the three-dimensional volume data of the lumbar vertebrae as a training sample, learns the training sample through the machine learning method, and trains the first lumbar vertebra detection model through the training sample. The trained first lumbar vertebra detection model learns the volume data features of the lumbar vertebrae, wherein the volume data features can include PCA features, LDA features, Harr features, texture features, and the like.

[0199] When the first lumbar vertebra detection model receives the three-dimensional volume data of the second to-be-detected tissue, it identifies the three-dimensional volume data of the lumbar vertebrae included in the received three-dimensional volume data according to the learned volume data features of the lumbar vertebrae.

[0200] S503, rendering the three-dimensional volume data of the conus medullaris and the three-dimensional volume data of the lumbar vertebrae to obtain a three-dimensional ultrasound image of a vertebral structure;

[0201] After identifying the three-dimensional volume data of the conus medullaris and the three-dimensional volume data of the lumbar vertebrae in the second to-be-detected tissue in S502, the three-dimensional volume data of the conus medullaris and the three-dimensional volume data of the lumbar vertebrae are rendered in three dimensions to obtain a three-dimensional ultrasound image of a vertebral structure.

[0202] Here, when the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebrae are three-dimensionally rendered, all data in the three-dimensional data of the second to-be-detected tissue except the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebrae are emptied, the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebrae are stereoscopic ray perspective rendered, and a three-dimensional ultrasound image of a vertebral structure is formed.

[0203] It should be noted that the rendering manner can be various, and the embodiment of the present application does not limit the rendering manner.

[0204] S504, marking the conus medullaris in the three-dimensional ultrasound image of the vertebral structure;

[0205] According to the position of the three-dimensional data of the conus medullaris in the three-dimensional data of the second to-be-detected tissue identified in S502, the position of the three-dimensional data of the conus medullaris in the three-dimensional ultrasound image of the fetal rib structure in S203 is determined, and the conus medullaris is marked according to the determined position of the conus medullaris, to obtain a marked three-dimensional ultrasound image of the vertebral structure.

[0206] Here, the displayed three-dimensional ultrasound image of the vertebral structure can only include the conus medullaris and the lumbar vertebrae, or can include the lumbar vertebrae and other vertebrae such as the vertebral bone.

[0207] In actual application, the distance between the conus medullaris and the lumbar vertebrae is detected, and the detected distance is displayed on the display screen. The detected distance is compared with the set distance range, and if the detected distance and the set distance range do not match, a prompt information is sent to prompt that the current position of the conus medullaris is abnormal.

[0208] S505, outputting the marked three-dimensional ultrasound image of the vertebral structure.

[0209] At this time, the content displayed on the display of the ultrasonic imaging device is the three-dimensional ultrasound image of the vertebral structure marked with the conus medullaris, so that the user can intuitively view the position of the conus medullaris of the fetus and determine the distance between the conus medullaris and the lumbar vertebrae.

[0210] In an embodiment, marking the conus medullaris in the three-dimensional ultrasound image of the vertebral structure comprises: determining the three-dimensional coordinates of the end of the conus medullaris; determining the longitudinal axis of the lumbar vertebrae; determining a reference line or a third plane passing through the end of the conus medullaris and perpendicular to the longitudinal axis of the lumbar vertebrae according to the three-dimensional coordinates of the end of the conus medullaris and the longitudinal axis of the fetal spine; and mapping the reference line or the third plane in the three-dimensional ultrasound image of the vertebral structure.

[0211] According to the two-dimensional coordinates of the end of the spinal cord cone on the sagittal plane and the specific position of the sagittal plane in the three-dimensional body data, the three-dimensional coordinates of the end of the spinal cord cone in the three-dimensional body data are calculated, and according to the three-dimensional coordinates of the end of the spinal cord cone and the longitudinal axis of the lumbar vertebrae in the three-dimensional body data, a plane or a straight line passing through the end point of the end of the spinal cord cone and perpendicular to the lumbar vertebrae is calculated, and the plane or the straight line is referred to as a third plane. Mapping the third plane to the three-dimensional VR image of the fetal spine can intuitively represent the relative position of the end of the spinal cord cone relative to the fetal lumbar vertebrae, and the result is calculated and marked on the VR image. The longitudinal axis of the lumbar vertebrae in the three-dimensional body data represents the posture of the fetal spine.

[0212] The ultrasonic imaging method provided in the embodiments of the present application can automatically image the three-dimensional VR image of the fetal spine after obtaining the three-dimensional body data of the fetus, and mark the spinal cord cone on the three-dimensional VR image, thereby greatly simplifying the workflow of the spinal cord cone examination. Further, the specific position of the spinal cord cone on the lumbar vertebrae is marked on the three-dimensional VR image. The doctor is liberated from tedious and complex manual operation, the dependence on the doctor's technology is reduced, and the examination efficiency is improved.

[0213] The ultrasonic imaging method provided in the embodiments of the present application can automatically image the three-dimensional VR image of the fetal spine after obtaining the three-dimensional body data of the fetus, and mark the spinal cord cone on the three-dimensional VR image, thereby greatly simplifying the workflow of the spinal cord cone examination. Further, the specific position of the spinal cord cone on the lumbar vertebrae is marked on the three-dimensional VR image. The doctor is liberated from tedious and complex manual operation, the dependence on the doctor's technology is reduced, and the examination efficiency is improved. Figure 1 The ultrasonic imaging device shown in FIG. 1, such as Figure 6 As shown in FIG. 1, the method comprises the following steps.

[0214] S601, obtaining three-dimensional body data of a second to-be-measured tissue;

[0215] The second to-be-measured tissue includes the fetal spine and tissues other than the fetal spine, such as amniotic fluid regions, placenta, uterine wall, etc.

[0216] The doctor can perform scanning on the pregnant woman through the probe to obtain the three-dimensional body data of the second to-be-measured tissue.

[0217] Here, the fetal rib structure belongs to a high echo region and is displayed as high gray scale in the ultrasonic image.

[0218] S602, identifying three-dimensional body data of a spinal cord cone from the three-dimensional body data of the second to-be-measured tissue;

[0219] After obtaining the three-dimensional body data of the second to-be-measured tissue based on S601, the three-dimensional body data of the spinal cord cone is identified from the obtained three-dimensional body data.

[0220] The spinal cord cone can be recognized from the three-dimensional volume data of the second to-be-detected tissue by a spinal cord cone detection model. The spinal cord cone detection model determines at least two second candidate regions from the three-dimensional volume data of the second to-be-detected tissue, obtains volume data features of the three-dimensional volume data of each second candidate region, determines a second matching degree of each second candidate region with the fetal spine according to the volume data features of each second candidate region, determines a second candidate region with the highest second matching degree as a target region corresponding to the spinal cord cone, and takes the three-dimensional volume data of the target region corresponding to the spinal cord cone as the three-dimensional volume data of the spinal cord cone.

[0221] S603, recognize the three-dimensional volume data of the fetal spine from the three-dimensional volume data of the second to-be-detected tissue;

[0222] After obtaining the three-dimensional volume data of the second to-be-detected tissue based on S601, recognize the three-dimensional volume data of the fetal spine from the obtained three-dimensional volume data.

[0223] When recognizing the three-dimensional volume data of the fetal spine from the three-dimensional volume data of the second to-be-detected tissue, at least one of the following three spine recognition modes can be included:

[0224] The first spine recognition mode is to display a three-dimensional image corresponding to the three-dimensional volume data of the second to-be-detected tissue, receive a fourth input operation based on the three-dimensional image corresponding to the three-dimensional volume data of the second to-be-detected tissue, determine a landmark point corresponding to the fourth input operation, and recognize the three-dimensional volume data of the spine from the three-dimensional volume data of the second to-be-detected tissue according to the coordinates of the landmark point corresponding to the fourth input operation.

[0225] After obtaining the three-dimensional volume data of the second to-be-detected tissue, display a three-dimensional image corresponding to the three-dimensional volume data of the first to-be-detected tissue on a display. A user can perform a fourth input operation on the three-dimensional image of the fetal spine displayed on the display based on the three-dimensional volume data of the second to-be-detected tissue by selecting a landmark point, drawing a landmark line, or the like through a trackball, a touch screen, or the like, to inform an ultrasonic imaging device of the position of the fetal spine in space. Here, the landmark line is composed of a plurality of continuous landmark points. After receiving the fourth input operation, the ultrasonic imaging device determines the position of the fetal spine in the three-dimensional volume data of the second to-be-detected tissue through the coordinates of the landmark point corresponding to the fourth input operation or the landmark points constituting the landmark line.

[0226] For example, the user selects some points on the endpoints of each vertebra of the fetal spine or intermittently on the boundary of the fetal spine, the ultrasonic imaging device takes the points selected by the user as landmark points, roughly draws a center line of the fetal spine or outlines a boundary line of the fetal spine according to the coordinates of the landmark points, determines the position of the fetal spine according to the determined center line or boundary line, and thus obtains the three-dimensional volume data of the fetal spine.

[0227] The second seed region is located in the three-dimensional image region corresponding to the spine; the second pixel feature of the three-dimensional data of the second seed region is determined; and the three-dimensional data of the spine is identified in the three-dimensional data of the second tissue to be measured according to the second pixel feature.

[0228] Here, after obtaining the three-dimensional data of the second tissue to be measured, the three-dimensional image corresponding to the three-dimensional data of the second tissue to be measured is displayed on the display. The user performs a fifth input operation on the three-dimensional image corresponding to the three-dimensional data of the second tissue to be measured displayed on the display based on a tool such as a trackball, a touch screen, etc. After the ultrasonic imaging device receives the fifth input operation, the region corresponding to the fifth input operation is determined as the second seed region, the three-dimensional data of the second seed region is taken as prior data, the pixel features such as edge gradient and grayscale of the prior data are obtained as the second pixel feature, and the three-dimensional data of the fetal spine is identified from the three-dimensional data of the second tissue to be measured according to the second pixel feature of the prior data.

[0229] The method of identifying the three-dimensional data of the fetal spine from the three-dimensional data of the second tissue to be measured according to the pixel feature of the prior data can include one or more image processing methods in template matching, image feature extraction, edge extraction, and morphological operation, can also include one or more image segmentation methods in graphcut algorithm, grabcut algorithm, level set method, active contour model algorithm, active shape model algorithm, seed region growing method, and region segmentation merging method, and can further include one or more machine learning methods in deep learning method, support vector machine, adaboost, and random forest algorithm.

[0230] For example, when the three-dimensional data of the fetal spine is identified from the three-dimensional data of the second tissue to be measured by template matching, the prior data is taken as a template, the pixel features such as edge gradient and grayscale of the template are calculated, the three-dimensional data of the second tissue to be measured is traversed through the template to find an optimal solution with the smallest difference from the pixel features of the template, and the identification of the fetal rib structure is realized.

[0231] For example, when the three-dimensional data of the fetal spine is identified from the three-dimensional data of the second to-be-tested tissue by using the seed region growing method, a seed region is determined in the fetal spine region, the pixels of the seed region are taken as seed pixels, then according to the second pixel feature of the seed pixels, the pixels meeting the second pixel feature are merged into the seed region from the three-dimensional data of the second to-be-tested tissue, the newly added pixels are taken as new seed pixels for continuous merging, until no new pixel meeting the condition is found, and finally the three-dimensional data of the fetal spine is identified.

[0232] Here, the workflow through certain user interaction operation acquires certain prior data in the three-dimensional data of the fetal spine as known information in advance, and the difficulty of identification of the fetal spine is reduced through the pixel feature of the known information.

[0233] The third spinal cord identification mode is to determine at least two fourth candidate regions from the three-dimensional data of the second to-be-tested tissue, acquire the volume data feature of the three-dimensional data of each fourth candidate region, determine the fourth matching degree of each fourth candidate region with the fetal spine according to the volume data feature of each fourth candidate region, determine the fourth candidate region with the highest fourth matching degree as the target region corresponding to the fetal spine, and take the three-dimensional data of the target region corresponding to the fetal spine as the three-dimensional data of the fetal spine.

[0234] Here, the spinal cord detection model can receive the three-dimensional data of the second to-be-tested tissue, determine at least two fourth candidate regions from the three-dimensional data of the second to-be-tested tissue, acquire the volume data feature of the three-dimensional data of each fourth candidate region, determine the fourth matching degree of each fourth candidate region with the fetal spine according to the volume data feature of each fourth candidate region, determine the fourth candidate region with the highest fourth matching degree as the target region corresponding to the fetal spine, and take the three-dimensional data of the target region corresponding to the fetal spine as the three-dimensional data of the fetal spine.

[0235] The algorithm used by the spinal cord detection model can be a machine learning method, the spinal cord detection model takes the three-dimensional data of the fetal spine as a training sample, and learns the training sample by using the machine learning method, so as to train the spinal cord detection model through the training sample. The trained spinal cord detection model learns the volume data feature of the fetal spine, wherein the volume data feature can include PCA feature, LDA feature, Harr feature, texture feature and the like.

[0236] When the spinal cord detection model receives the three-dimensional data of the second to-be-tested tissue, the three-dimensional data of the fetal spine included in the received three-dimensional data is identified according to the learned volume data feature of the fetal spine.

[0237] In the training sample, the fetus spine is taken as a target, and the target is labeled. The labeling can be performed by means of an ROI box containing the target or by means of a mask precisely segmenting the target.

[0238] The algorithm used by the spine detection model can be an image segmentation algorithm. The three-dimensional body data input into the spine detection model is binarized and segmented, and morphological, contour extraction, connected domain, and other operations are performed to obtain a plurality of fourth candidate regions. The probability of each fourth candidate region being the fetus spine is determined according to the features of the body data of each fourth candidate region. The fourth candidate region with the highest probability is selected as the target region corresponding to the fetus spine, and the three-dimensional body data of the selected target region is the three-dimensional body data of the fetus spine.

[0239] In actual applications, the image segmentation algorithm used by the spine detection model can also be one or more of the following: Level Set, Graph Cut, Snake, Random walker, active contour model algorithm, active shape model algorithm, active appearance model algorithm, and image segmentation algorithms in deep learning such as FCN and UNet.

[0240] The algorithm used by the spine detection model can also be a template matching algorithm, and a template of the three-dimensional body data of the fetus spine is established. The three-dimensional body data input into the spine detection model is binarized and segmented, and morphological, contour extraction, connected domain, and other operations are performed to obtain a plurality of fourth candidate regions. The similarity between all fourth candidate regions in the body data and the template, i.e., the fourth matching degree, is determined by traversing all fourth candidate regions according to the established template. The fourth candidate region with the highest similarity is selected as the target region corresponding to the fetus spine, and the three-dimensional body data of the target region is the three-dimensional body data of the fetus spine.

[0241] It should be noted that the specific identification method of the three-dimensional body data of the fetus spine from the three-dimensional body data of the second to-be-detected tissue in the embodiments of the present application is not limited in any way.

[0242] S604, the three-dimensional body data of the fetus spine is segmented from the three-dimensional body data of the second to-be-detected tissue;

[0243] After the three-dimensional body data of the fetus spine is identified, the three-dimensional body data of the fetus spine is segmented from the three-dimensional body data of the second to-be-detected tissue according to the position of the three-dimensional body data of the fetus spine in the three-dimensional body data of the second to-be-detected tissue.

[0244] In actual applications, S603 and S604 can be implemented simultaneously, i.e., the three-dimensional body data of the fetus spine is segmented from the three-dimensional body data of the second to-be-detected tissue while the three-dimensional body data of the fetus spine is identified.

[0245] S605, identify the three-dimensional body data of the lumbar vertebrae and the three-dimensional body data of the lumbar vertebrae from the three-dimensional body data of the fetal spine;

[0246] After the three-dimensional body data of the fetal spine is segmented from the three-dimensional body data of the second to-be-detected tissue in S604, when the three-dimensional body data of the lumbar vertebrae is identified from the three-dimensional body data of the fetal spine, at least one of the following two lumbar vertebrae identification methods can be included:

[0247] The first lumbar vertebrae identification method is based on the shape characteristics of the vertebrae, locates the vertebrae of the fetal spine from the three-dimensional body data of the fetal spine, displays the three-dimensional image corresponding to the three-dimensional body data of the fetal spine, receives a sixth input operation based on the three-dimensional image corresponding to the three-dimensional body data of the fetal spine, determines a reference vertebra and a vertebra identifier corresponding to the reference vertebra based on the sixth input operation, the reference vertebra is at least one vertebra in the fetal spine, and the three-dimensional body data of the lumbar vertebrae is identified based on the reference vertebra and the vertebra identifier corresponding to the reference vertebra.

[0248] After the three-dimensional body data of the fetal spine is determined, the three-dimensional image corresponding to the three-dimensional body data of the fetal spine is displayed on the display. The user performs a sixth input operation on the three-dimensional image corresponding to the three-dimensional body data of the fetal spine displayed on the display based on the trackball, touch screen, etc. Tool, to identify part of the vertebrae in the fetal spine and the vertebrae identifier of the identified fetal spine.

[0249] After the ultrasound imaging device receives the sixth input operation, the vertebrae identified by the user is determined by the sixth input operation, the vertebrae identified by the user is taken as the reference vertebra, and the vertebrae identifier input by the user is taken as the vertebrae identifier of the reference vertebra. The lumbar vertebrae in the fetal spine and the vertebrae identifier corresponding to the lumbar vertebrae are determined according to the reference vertebra.

[0250] When the ultrasound imaging device determines the lumbar vertebrae in the fetal spine and the vertebrae identifier corresponding to the lumbar vertebrae according to the reference vertebra, the vertebrae included in the fetal spine can be determined according to one or more of the following methods: gray level histogram projection method, contour extraction algorithm, edge extraction algorithm, connected domain method, blob detection algorithm, template matching algorithm, image feature extraction algorithm, and morphological operation algorithm. The lumbar vertebrae is determined from the determined vertebrae included in the fetal spine.

[0251] The second lumbar vertebrae identification method is based on a lumbar vertebra detection model for the lumbar vertebrae, and the three-dimensional body data of the lumbar vertebrae is identified from the three-dimensional body data of the fetal spine. The lumbar vertebra detection model is trained by sample body data of the lumbar vertebrae.

[0252] The algorithm used by the lumbar vertebra detection model can be a machine learning method. The lumbar vertebra detection model takes three-dimensional body data of a lumbar vertebra as a training sample, learns the training sample by a machine learning method, and trains the lumbar vertebra detection model by the training sample. The trained lumbar vertebra detection model learns the body data features of the lumbar vertebra, wherein the body data features can include PCA features, LDA features, Harr features, texture features, and the like.

[0253] When the lumbar vertebra detection model receives three-dimensional body data of a fetal spine, the three-dimensional body data of the lumbar vertebra included in the received three-dimensional body data is identified according to the learned body data features of the lumbar vertebra.

[0254] In the training sample, the lumbar vertebra is taken as a target and labeled. The labeling can be performed by a ROI box containing the target, or by a mask (Mask) performing accurate segmentation of the target.

[0255] The algorithm used by the lumbar vertebra detection model can be an image segmentation algorithm. The three-dimensional body data input into the lumbar vertebra detection model is binarized and segmented, and morphological, contour extraction, connected domain, and the like operations are performed to obtain a plurality of candidate regions. The probability of each candidate region being a lumbar vertebra is determined according to the body data features of each candidate region. The candidate region with the highest probability is selected as the region corresponding to the lumbar vertebra, and the three-dimensional body data of the selected region is the three-dimensional body data of the lumbar vertebra.

[0256] In actual application, the image segmentation algorithm used by the lumbar vertebra detection model can also be one or more of the following: Level Set, Graph Cut, Snake, Random walker, active contour model algorithm, active shape model algorithm, active appearance model algorithm, and image segmentation algorithms in deep learning such as Fully Convolutional Networks (FCN) and UNet.

[0257] The algorithm used by the lumbar vertebra detection model can also be a template matching algorithm. A template of the three-dimensional body data of the lumbar vertebra is established. The three-dimensional body data input into the lumbar vertebra detection model is binarized and segmented, and morphological, contour extraction, connected domain, and the like operations are performed to obtain a plurality of candidate regions. All candidate regions in the body data are traversed according to the established template, and the similarity of all candidate regions and the template is determined. The candidate region with the highest similarity is selected as the target region corresponding to the fetal rib or spine, and the three-dimensional body data of the target region is the three-dimensional body data of the lumbar vertebra.

[0258] The algorithm used by the lumbar vertebra detection model can also be one or more of the following: image edge extraction, histogram image gray projection statistics, image contour extraction, morphological processing, threshold segmentation, and blob detection, to directly calculate the number of fetal rib structures in the three-dimensional data of the fetal rib.

[0259] In actual applications, the lumbar vertebra detection model and the first lumbar vertebra detection model can be a detection model.

[0260] S605, rendering the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebra to obtain a three-dimensional ultrasound image of the vertebral structure;

[0261] After identifying the three-dimensional data of the conus medullaris in the second to-be-detected tissue in S602 and identifying the three-dimensional data of the lumbar vertebra in the second to-be-detected tissue in S605, the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebra are three-dimensionally rendered to obtain a three-dimensional ultrasound image of the vertebral structure.

[0262] Here, when the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebra are three-dimensionally rendered, all data in the three-dimensional data of the second to-be-detected tissue except the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebra are emptied, and the three-dimensional data of the conus medullaris and the three-dimensional data of the lumbar vertebra are stereoscopic light perspective rendered to form a three-dimensional ultrasound image of the vertebral structure.

[0263] It should be noted that the rendering method can be various, and the embodiments of the present application do not limit the rendering method.

[0264] S606, marking the conus medullaris in the three-dimensional ultrasound image of the vertebral structure;

[0265] According to the position of the three-dimensional data of the conus medullaris in the three-dimensional data of the second to-be-detected tissue identified in S502, the position of the three-dimensional data of the conus medullaris in the three-dimensional ultrasound image of the fetal rib structure in S203 is determined, and the conus medullaris is marked according to the determined position of the conus medullaris to obtain a marked three-dimensional ultrasound image of the vertebral structure.

[0266] In actual applications, the distance between the conus medullaris and the lumbar vertebra is detected, and the detected distance is displayed on the display screen. The detected distance is compared with the set distance range, and if the detected distance does not match the set distance range, an alarm is issued.

[0267] S607, outputting the marked three-dimensional ultrasound image of the vertebral structure.

[0268] At this time, the display on the ultrasound imaging device shows a three-dimensional ultrasound image of the vertebral structure marked with the conus medullaris, allowing the user to visually view the location of the fetal conus medullaris and determine the distance between the conus medullaris and the lumbar vertebrae.

[0269] The ultrasound imaging method provided in this application automatically or semi-automatically identifies, locates, and segments the three-dimensional volume data of the fetus's spine after acquiring the fetal three-dimensional volume data. It then automatically images a stereoscopic VR map of the vertebral structure and marks the conus medullaris on the VR map. This greatly simplifies the workflow of conus medullaris examination, freeing doctors from tedious manual operations, reducing reliance on the doctor's skills, and improving examination efficiency. Furthermore, the stability and imaging quality of the conus medullaris localization results are superior to manual methods, reducing the rate of misdiagnosis and missed diagnosis.

[0270] In related technologies, ultrasound examinations of fetal ribs and the position of the conus medullaris are still performed entirely manually. For example, after acquiring ultrasound data of the fetal ribs, doctors often need to adjust the rotation and translation of the X, Y, and Z axes to adjust the orientation of the data, and manually adjust the size and position of the Volume of Interest (VOI) to better observe the overall structure of the fetal ribs or spine. Figure 7 The fetal rib diagram shown requires manual counting of the number of fetal ribs or any abnormalities in area 701 corresponding to the rib structure, in order to manually check whether the ribs are real or abnormal. If rib A in area 701 is abnormal, the position of rib A needs to be manually determined.

[0271] If you want to obtain a cross-section or coronal section of a single fetal rib, all fetal ribs, or vertebrae, such as Figure 8 As shown, the Curved Multi Planar Reformation (CMPR) function needs to be used manually in [the following context]: Figure 8 (A) An anatomical trajectory 801 is drawn on the fetal ribs or vertebrae. After setting parameters such as thickness, a transverse or coronal section 8(B) of the stretched rib or vertebra is obtained, allowing observation of any abnormalities in the affected rib or vertebra. During examination of the conus medullaris, the doctor needs to manually and continuously adjust the sagittal plane of the spine to the optimal position, which is subjectively considered the best section for imaging the conus medullaris. For example... Figure 9 The spinal conus medullaris shown is positioned relative to the lumbar vertebrae by the human eye. This method is subjective and easily affected by image quality and observation angle, leading to inaccurate results.

[0272] It is evident that each manual procedure in the examination of fetal ribs or conus medullaris in related technologies is quite complex and time-consuming, and highly dependent on the doctor's skills and experience. For example, in the CMPR procedure, manual tracing requires high skill and patience. More importantly, the quality of the doctor's tracing directly affects the quality of the transverse or coronal imaging of the ribs or vertebrae, resulting in a lack of consistency and stability in imaging effects and quality, thus affecting the diagnostic results.

[0273] The ultrasound imaging method provided in this application can effectively assist doctors in disease diagnosis, significantly improve work efficiency, and enhance the quality of acquired key diagnostic data. As an example, the ultrasound imaging device used in the ultrasound imaging method of this application embodiment can be as follows: Figure 10 As shown, it includes: a volume data acquisition module 1001, a recognition module 1002, a statistics module 1003, an imaging module 1004, and a display module 1005. Among them, Figure 10 The module shown can be located in Figure 1 The processor 105 in the ultrasound imaging device shown.

[0274] The volume data acquisition module 1001 is used to acquire three-dimensional volume data of the fetal ribs, fetal spine (also known as the spinal column) and spinal cord via ultrasound.

[0275] when Figure 1 The transmitting circuit 101 sends a set of delayed-focused pulses to the probe 100. The probe 100 emits ultrasound waves towards the tissue to be tested and, after a certain delay, receives the ultrasound echoes reflecting back from the tissue containing tissue information, converting these ultrasound echoes back into electrical signals. The receiving circuit 103 receives these electrical signals and sends these ultrasound echo signals to the beamforming module 104. The ultrasound echo signals undergo focusing delay, weighting, and channel summation in the beamforming module 104, and are then processed by the processor 105 to obtain three-dimensional volume data including the fetal rib structure and spine.

[0276] The identification module 1002 is used for the identification, localization and segmentation of ultrasound rib structures or the spine, as well as the identification, localization and segmentation of the conus medullaris.

[0277] The identification and location methods for fetal rib structures (also known as rib bones) can be divided into three types: manual, semi-automatic, and automatic. Identification methods include identifying the overall structure of the rib skeleton and identifying each rib (T1-T12) and the centerline separately. The methods for identifying fetal ribs are as follows:

[0278] 1. Manually obtain the location of the fetal rib structure

[0279] The user selects a mark point or draws a mark line on the rib structure in the volume data through a certain workflow by using a trackball, a touch screen or the like to inform the system of the position of the rib structure in space, for example, selecting an end point of each fetal rib or taking some points on the boundary of the rib structure, roughly drawing a center line of the rib structure or drawing a boundary line of the rib structure.

[0280] 2. A method for automatically identifying a fetal rib structure

[0281] The machine learning method is used to learn the features or rules that can distinguish the fetal ribs from other non-rib structures in the database, and then the learned features or rules are used to locate and identify the fetal rib structure in other volume data. First, a database is constructed by using volume data of multiple fetal ribs and corresponding calibration results. The calibration results can be set according to actual task needs, which can be a ROI box containing the target or a Mask (mask) for accurate segmentation of the target.

[0282] It should be noted that if each rib or spine of the fetus is recognized and located as a different category instead of the structure composed of all ribs and spines as a whole target, the category of the rib or spine of each ROI box or Mask needs to be specified, that is, all fetal ribs and spines that are calibrated into different categories need to be converted into a multi-target recognition and positioning problem. After the database is constructed, a machine learning algorithm is designed to learn the features or rules that can distinguish the fetal rib or spine region from the non-fetal rib or non-spine region in the fetal rib and spine database to realize the positioning and identification of the fetal rib and spine in the volume data.

[0283] The method for identifying the fetal rib structure includes but is not limited to the following methods:

[0284] Method one, based on sliding window: first, the features in the sliding window are extracted, the extracted features can be PCA features, LDA features, Harr features, texture features, etc., or a deep neural network can be used for feature extraction, then the extracted features are matched with the database, and KNN, SVM, random forest, neural network, etc. are used as discriminators for classification to determine whether the current sliding window is the region of interest and to obtain the corresponding category.

[0285] Method two, bounding box method based on deep learning, common forms are: through the stacking of base convolution layer and fully connected layer to learn the features and regress the parameters of the constructed database, for the input three-dimensional body data, the corresponding bounding box of the region of interest can be directly regressed by the network, and the class of the tissue structure in the region of interest is obtained, the algorithm used by the network model includes R-CNN, Fast R-CNN, Faster-RCNN, SSD, YOLO, etc.

[0286] Method three, end-to-end semantic segmentation network method based on deep learning, which is similar to method two based on deep learning bounding box structure, the difference is that the fully connected layer is removed, and the up-sampling or de-convolution layer is added to make the size of the input and output the same, so as to directly obtain the region of interest of the input image and the corresponding class, common networks include FCN, U-Net, Mask R-CNN, etc.

[0287] Method four, using method one, method two or method three to locate the target, and then classifying the located target by the classifier. The class judgment method can be: first, extracting the features of the target ROI or Mask, the feature extraction method can be PCA feature, LDA feature, Harr feature, texture feature, etc., or a deep neural network can be used for feature extraction, then the extracted features are matched with the database, and KNN, SVM, random forest, neural network, etc. Discriminator is used for classification.

[0288] The method of identifying the rib structure of the fetus can be an image segmentation algorithm, which accurately segments the rib structure of the fetus in the body data. Here, the rib structure of the fetus usually appears as a high-echo arc-shaped target with high gray value, which can be segmented out by the image segmentation algorithm.

[0289] For example, first, the body data is binarized and segmented, and after some necessary morphological, contour extraction, connected domain and other operations, a plurality of candidate regions are obtained, then the probability of each candidate region being a rib structure is judged according to the shape, length-width ratio and other characteristics, and the region with the highest probability is selected as the rib structure region. Other image segmentation algorithms can also be used, such as one or more of level set, graph cut, snake, random walker, active contour model algorithm, active shape model algorithm, active appearance model algorithm, and some image segmentation algorithms in deep learning, such as FCN, UNet, etc.

[0290] For example, the rib structure of the fetus can also be detected in the volume data by using a template matching method. The rib structure of the fetus has a special shape, and a template can be established by collecting some data of the rib structure of the fetus in advance. During detection, all possible regions in the volume data are traversed, and similarity matching is performed with the template. The region with the highest similarity is selected as the target region.

[0291] 3. A method for semi-automatically obtaining the position of the rib of the fetus:

[0292] The user needs to use a trackball, a touch screen or the like to obtain some prior data in the volume data of the rib structure of the fetus as known information or to set some parameters or conditions in advance by a certain user interaction workflow to help reduce the difficulty of recognition, and then combine one or more image processing methods such as a template matching algorithm, an image feature extraction algorithm, an edge extraction algorithm, a morphological operation algorithm, one or more image segmentation algorithms such as a graphcut algorithm, a grabcut algorithm, a level set method, an active contour model algorithm, an active shape model algorithm, a seed region growing method and a region segmentation and merging method, and one or more machine learning methods such as a deep learning method, a support vector machine, an adaboost and a random forest algorithm to realize recognition and determination of the position of the rib structure of the fetus in the volume data.

[0293] For example, when the template matching method is used, a sample volume data needs to be obtained in advance. A trackball, a touch screen or the like can be used to take out data containing part or all of the rib structure information of the fetus to make a template. For example, a rectangular frame is used to intercept an image containing the rib structure of the fetus on one or more cross sections as a template. Then, the edge gradient and grayscale information of the template are calculated. Then, the template is used to traverse the entire volume data to find the optimal solution with the smallest difference from the template information to realize recognition of the rib of the fetus.

[0294] When the graphcut algorithm is used to segment the rib region of the fetus, seed points representing the target region and the background region need to be specified in the rib structure of the fetus and the non-rib structure of the fetus, respectively. For example, the user specifies different seed points by clicking and drawing lines inside and outside the rib structure by using a trackball. The grabcut algorithm needs to manually draw a frame to enclose the rib structure target region to be segmented or draw some lines of different colors in the rib structure region and the non-rib structure region to specify the pixels of the target region and the background region, respectively, so as to obtain a perfect segmentation result.

[0295] Segmentation methods such as level set and active contour model need user interaction to give the initial contour curve, and then the curve evolution is carried out according to the functional energy minimization to approximate the boundary of the fetal rib, so as to realize the segmentation of the fetal rib. The seed region growing method also needs the user to provide a set of seed pixels representing different growing regions, and then the pixels meeting the conditions in the seed pixel region are merged into the growing region represented by the seed pixel. The newly added pixels are continuously merged as new seed pixels until no new pixels meeting the conditions are found. Finally, the fetal rib structure is segmented. Alternatively, the detection range can be reduced to reduce the identification difficulty by setting a rectangular frame or specifying some landmark points of the rib structure through the interactive workflow by the user in advance.

[0296] Through the above method, the position of the fetal rib structure in the volume data can be identified.

[0297] In practical applications, the above-mentioned method of identifying the fetal rib structure is also applicable to the identification of the spine.

[0298] It should be noted that when constructing the database of machine learning and when constructing the template of template matching, the sample can be the whole structure containing the spine and all 12 ribs as a labeled sample or template, or the 12 ribs and the spine can be labeled and identified or template matched as different categories of multiple targets.

[0299] The identification and positioning of the spinal cord cone need to automatically identify and segment the spinal cord cone in the spinal cord volume data and determine its accurate position. The method is as follows:

[0300] The machine learning method is used to learn the characteristics or rules that can distinguish the spinal cord cone and other non-spinal cord cone tissues in the database, and then the learned characteristics or rules are used to locate and identify the spinal cord cone in other volume data. First, a plurality of spinal cord data are sampled to construct a database. The labeling result can be set according to the actual task needs, which can be a ROI frame containing the target, or a mask (mask) for accurate segmentation of the target. After the database is constructed, a machine learning algorithm is designed to learn the characteristics or rules that can distinguish the spinal cord cone region and the non-spinal cord cone region in the spinal cord cone database to realize the positioning and identification of the spinal cord cone in the volume data.

[0301] Common methods include but are not limited to the following methods:

[0302] Method one, based on sliding window method: first, the region in the sliding window is extracted, the extracted feature method can be PCA feature, LDA feature, Harr feature, texture feature, etc., and deep neural network can also be used for feature extraction, then the extracted features are matched with the database, and KNN, SVM, random forest, neural network, etc. Discriminator is used for classification to determine whether the current sliding window is the region of interest and to obtain the corresponding category.

[0303] Method two, Bounding-Box method based on deep learning for detection and identification, common forms are: through the stacking of base convolution layer and full connection layer to construct the database for feature learning and parameter regression, for the input three-dimensional body data, the corresponding Bounding-Box of the region of interest can be directly regressed through the network, and the category of the tissue structure in the region of interest is obtained. The network model can use R-CNN, Fast R-CNN, Faster-RCNN, SSD, YOLO, etc.

[0304] Method three, end-to-end semantic segmentation network method based on deep learning, this kind of method is similar to the structure of the second method based on deep learning Bounding-Box, the difference is that the full connection layer is removed, and the up-sampling or deconvolution layer is added to make the size of the input and output the same, so as to directly obtain the region of interest of the input image and the corresponding category, common network has FCN, U-Net, Mask R-CNN, etc.

[0305] Method four, only method one, method two or method three is used to locate the target, and then the classification judgment is carried out on the positioning result of the target. The classification judgment method can be: first, the features of the target ROI or Mask are extracted, the extracted features can be PCA feature, LDA feature, Harr feature, texture feature, etc., and deep neural network can also be used for feature extraction, then the extracted features are matched with the database, and KNN, SVM, random forest, neural network, etc. Discriminator is used for classification.

[0306] The method of identifying the spinal cord cone can also be an image segmentation algorithm, which can accurately segment the spinal cord cone in the body data. Here, the spinal cord cone usually appears as a high gray value high echo arc-shaped target, which can be segmented out by image segmentation algorithm.

[0307] For example, first, the volume data is binarized and segmented, and after some necessary morphological, contour extraction, connected domain, etc. operations, a plurality of candidate regions are obtained, then the probability of each candidate region according to the shape, aspect ratio, etc. characteristics is judged that the region is the conus medullaris, and a region with the highest probability is selected as the conus medullaris region. Other image segmentation algorithms can also be used, for example, one or more of the following: Level Set, Graph Cut, Snake, Random walker, Active Contour Model algorithm, Active Shape Model algorithm, Active Appearance Model algorithm, and some image segmentation algorithms in deep learning, such as FCN, UNet, etc.

[0308] For example, the conus medullaris of the fetus can be detected in the volume data by using a template matching method, for example, the shape of the conus medullaris of the fetus is relatively special, and some data of the conus medullaris of the fetus can be collected in advance to establish a template, and when detecting, all possible regions in the volume data are traversed, and similarity matching is performed with the template, and the region with the highest similarity is selected as the target region.

[0309] Through the above method, the accurate position of the conus medullaris of the fetus in the volume data can be automatically located.

[0310] The statistical module 1003 is used for semi-automatic or automatic statistics of the number of ribs and vertebrae of the fetus.

[0311] After the position of the rib structure or the spine of the fetus in the volume data is determined by the recognition module 1002, the statistical module 1003 identifies and marks the 12 ribs, the spine and the lumbar vertebrae of the fetus by semi-automatic or automatic method, calculates the number of ribs of the fetus, and displays the marking results of the ribs, the spine and the lumbar vertebrae on the display module 1005.

[0312] It should be noted that the identification and position of the lumbar vertebrae and the marking of the name are used as a reference for the position of the end of the conus medullaris. Clinically, the specific vertebra level corresponding to the end of the conus medullaris is often used to judge the position of the end of the conus medullaris of the fetus and its rising rule, for example, the end of the conus medullaris of a normal adult is located at the lumbar vertebra L1-2 level. The calculation and marking of the number and position of the lumbar vertebrae are not used as a judgment index for congenital disease examination, but only as a reference and measurement index for indicating the specific position of the end of the conus medullaris of the fetus.

[0313] 1. Semi-automatic statistics of the number of fetal rib bodies:

[0314] Based on the information provided by the user through a certain manual operation workflow, and the fetal rib structure data segmented by the recognition module 1002, one or more of the following methods are used to count the number of fetal ribs or vertebrae and to label the name of each rib or vertebra.

[0315] For example, firstly, one or more ribs or vertebrae are selected from the segmented fetal rib or vertebral data using tools such as a trackball or touchscreen. For instance, a point is marked on the T3 and T9 ribs, or the center lines of the T3 and T9 ribs are drawn respectively, thus marking these two ribs as known. Then, based on the segmented rib data and the fixed arrangement order and spatial position relationship between the T1 to T12 ribs, such as the ribs above the T2 ribs being always T1 and below the T3 ribs being always T3 if no rib structure is missing, the number of ribs is counted and the names are marked using a certain algorithm.

[0316] For example, a coronal plane of the volume data segmented by histogram grayscale projection is projected along the Y-axis. The number and location of the fetal ribs are obtained by calculating the number of pixel peaks and the position of the peaks in the pixel statistics graph. The distance between the approximate area information of the drawn points and lines and the obtained location information is calculated. The closest ones are marked as T3 and T9 ribs. All other ribs are then named or only the ribs between T3 and T9 are marked.

[0317] Similarly, the position of each segmented rib can be determined by finding the contours. The number of ribs can be calculated by counting the outermost enclosing contour (the largest contour). Of course, some image preprocessing may be needed to remove irrelevant structures and reduce interference. The above function can also be achieved by calculating the number and position of specific contours. Clump detection, template matching, etc. are similar and will not be elaborated further.

[0318] 2. Method for automatically counting fetal ribs or vertebrae

[0319] After segmenting the fetal rib structure or spinal body data, the known spatial relationships between the fetal ribs can be utilized, such as... Figure 11 The arrangement of the ribs (T1-T12) shown, or other known prior knowledge combined with image processing methods including but not limited to image edge extraction, histogram image grayscale projection statistics, image contour extraction, morphological processing, threshold segmentation, and clumping detection, directly calculates the number of fetal ribs in the body data. Here, in Figure 11In the shown arrangement order of the ribs, only T3, T6, T9 and T12 are labeled, i.e. only the 3rd rib, the 6th rib, the 9th rib and the 12th rib are labeled.

[0320] For example, the fetal rib structure or the coronal section image of the spine extracted by the machine learning or segmentation algorithm in the recognition module 1002 is first subjected to threshold segmentation, and then subjected to histogram statistical projection of the gray value along the direction perpendicular to the spine or parallel to the rib, with the longitudinal coordinate being the image height and the transverse coordinate being the pixel number. Then, the threshold value is set to calculate the peak number of the statistical graph, so as to obtain the number and position of the fetal rib, and then the fetal rib is labeled. Similarly, the position of the spine can also be obtained and labeled.

[0321] For another example, the number of ribs can be directly obtained by combining contour extraction and blob detection.

[0322] The machine learning method can also be used. The 12 different ribs and the spine are all regarded as an identification target to construct a learning database. The learning database can distinguish the characteristics or rules of each specific rib from other ribs or the spine, and learn the characteristics or rules of each different rib and the spine to locate and identify the ribs and the spine in the fetal rib structure in the other body data. The calibration result can be set according to the actual task requirement, which can be a ROI box containing the target or a Mask (mask) for accurately segmenting the target. Here, each rib or spine of the fetus is regarded as a different class to identify and locate the target, and the class of the rib or vertebral body of each ROI box or Mask needs to be specified, which is a multi-target identification and positioning problem. After the database is constructed, a machine learning algorithm is designed to learn the characteristics or rules that can distinguish the fetal rib region from the non-fetal rib region in the fetal rib database to realize the positioning and identification of the fetal rib in the body data.

[0323] The commonly used methods are similar to the recognition module 1002, such as: method one, a traditional method based on a sliding window to extract features and a discriminator to classify. Method two, a Bounding-Box method based on deep learning for detection and identification. Method three, an end-to-end semantic segmentation network method based on deep learning. Method four, a method of using only method one, method two or method three to locate the target, and then classifying the located target by a classifier.

[0324] The template matching method can also be used to detect different ribs and spines in the fetal rib structure body data. For example, different templates are established according to each different rib and spine of the fetus. When detecting, all possible regions of the segmented and extracted fetal rib structure are traversed, and similarity matching is performed with the templates. The region with the highest similarity is selected as the target region, and then the matching result is labeled and counted.

[0325] By combining, but not limited to, one or more of the above methods, the rib structures in the identified fetal rib structures can be statistically analyzed, and the number of ribs can be marked and counted.

[0326] It should be noted that the method used by the statistics module 1003 to count the number of vertebrae in the spine is the same as the method used to count the number of fetal ribs in the fetal rib structure. This allows the system to count the number of vertebrae in the spine and identify each vertebra, thereby identifying the lumbar vertebrae.

[0327] Imaging module 100 is used for automatic imaging of the fetal rib structure.

[0328] During automated imaging of the fetal rib structure, a straightening algorithm is used to straighten and reconstruct the rib structure data, and then the required volumetric data sections are identified and located for imaging. For example... Figure 11 The main contents of the automatic imaging shown include automatic coronal imaging of ribs or straightened ribs, automatic imaging of cross-sections of each rib or straightened rib, automatic imaging of cross-sections of manually selected ribs, automatic imaging of three-dimensional rib skeleton extraction, and automatic imaging of VR images of vertebrae with the position of the end of the conus medullaris marked.

[0329] The volume data of the ribs and spine need to be straightened and reconstructed to obtain complete coronal and transverse planes. The straightening of the objects is divided into two parts: extraction of the longitudinal axis and straightening reconstruction. The objects to be straightened include the ribs and spine.

[0330] Methods for extracting the longitudinal axis of a straightened object include: tracking-based longitudinal axis extraction algorithms, model-based multi-scale longitudinal axis extraction algorithms, morphology-based longitudinal axis extraction methods, region-growing-based centerline extraction methods, three-dimensional geometric moment-based methods, and machine learning-based centerline localization methods.

[0331] The longitudinal axis extraction algorithm based on tracking is a semi-automatic algorithm. It uses initial and termination points provided by user interaction to generate cross-sections perpendicular to the tracking direction during tracking. The algorithm then uses maximum likelihood and centroid methods to accurately calculate the center points of the ribs or spine within the cross-sections. After tracking ends, the longitudinal axis is obtained by interpolation fitting of the center point sequence. At this point, the longitudinal axis is sampled at equal intervals to generate a sequence of equally spaced rib or spine cross-sections perpendicular to the rib or spine direction. Finally, the equally spaced cross-sections are reconstructed to straighten the ribs or spine.

[0332] The model-based multi-scale longitudinal axis extraction algorithm approximates the local rib or spine as a tubular structure, takes the center of gravity of the tubular structure obtained by the computational geometry moment as the center of the local rib or spine, enhances the local rib or spine structure by analyzing the eigenvalues of the Hessian matrix of a voxel corresponding to the multi-scale Gaussian filter, and estimates the longitudinal axis direction according to the eigenvector corresponding to the minimum eigenvalue of the Hessian matrix. After the longitudinal axis is obtained, the ribs and vertebrae can be straightened and reconstructed. The longitudinal axis of the ribs is first sampled at equal intervals to obtain equal-interval center points, and on this basis, a sequence of equal-interval cross sections perpendicular to the direction of the ribs or vertebrae is regenerated, stacked together, and then the cross section of the ribs or the three-dimensional reconstruction diagram of the ribs at different angles is obtained to achieve the purpose of straightening.

[0333] The imaging of the coronal surface of the ribs, the imaging of the cross section of the ribs, the imaging of the rib skeleton, and the labeling of the spinal cord cone will be described below. Among them, the imaging object in the imaging of the coronal surface of the ribs and the imaging of the cross section of the ribs can be ribs or straightened ribs.

[0334] First, the imaging of the coronal surface of the ribs

[0335] The display of the coronal surface of the ribs needs to straighten the spine and ribs at the same time, as shown in Figure 12 , the ribs and the spine need to be straightened along the long axis direction, as shown by the arrow 1201 and the arrow 1202, after the rib volume data is straightened in the two directions indicated by the arrows by using the straightening method, the longitudinal axes of all the ribs and the spine are almost located on the same plane, and the plane is determined by solving mathematical equations or similar least squares method or Hough transform plane fitting method, and the coronal surface of the fetal rib structure required is obtained by imaging the volume data of the plane.

[0336] Second, the imaging of the cross section of the ribs

[0337] The display of the cross section of the ribs needs to straighten the ribs along the long axis 1202 in Figure 12 , in order to accurately find and determine the cross section of all the ribs, the spine is still straightened along the arrow 1202 direction in Figure 12 . A plane is determined by using two straight lines to determine the cross section of each rib. Among them, the cross section must be perpendicular to the longitudinal axis of the spine and coplanar with the longitudinal axis of the rib, so that the cross section of each rib can be uniquely determined.

[0338] The display of the cross section of the ribs is divided into Figure 13 , the display of all the cross sections of the ribs, and Figure 14 , the automatic display of the specified cross section of the ribs selected manually.

[0339] Third, imaging of rib skeleton

[0340] The structure of each rib T1-T12 and the center line of the spine are identified by the recognition module 1002 or the statistics module 1003, and the identified ribs are labeled.

[0341] First, the body data of the identified and segmented rib and spine structure is reconstructed into new body data, for example, the place of the fetal rib structure is 1, and the place of the non-fetal rib structure is 0. Then, the reconstructed body data is rendered using volume rendering or surface rendering to obtain a three-dimensional skeleton rendering diagram of the fetal rib and vertebral structure as shown in FIG. 8. Figure 15

[0342] In actual application, the cross section of each straightened rib and the three-dimensional skeleton can be displayed on the same display interface.

[0343] Fourth, labeling of conus medullaris

[0344] The specific position (two-dimensional coordinates) of the end of the conus medullaris located by the recognition module 02 on the sagittal plane is combined with the specific position of the sagittal plane in the body data to calculate the three-dimensional geometric coordinate point of the end of the conus medullaris in the body data. Finally, according to the three-dimensional coordinate point and the posture of the fetal spine (such as the center line of the spine) recognized and located in the body data, a plane or a straight line passing through the end point of the conus medullaris and perpendicular to the spine is calculated. The plane or straight line is mapped to the three-dimensional VR diagram of the vertebral structure, which can directly represent the relative position of the end of the conus medullaris relative to the fetal lumbar vertebrae, and the result (such as L1-L2) is calculated and labeled on the VR diagram.

[0345] In actual application, the fetal rib, lumbar vertebra and conus medullaris can be detected at the same time, and the rib and conus medullaris are labeled on the VR image including the fetal rib structure and vertebral structure. As shown in FIG. 10, a straight line 1701 is used to represent the specific position of the fetal conus medullaris corresponding to the position of the vertebra. Figure 16

[0346] The present application provides an ultrasonic detection method for fetal rib number abnormality and conus medullaris position abnormality. After obtaining the three-dimensional body data of the fetus, the fetal rib structure is automatically or semi-automatically (manually specifying 1 or 2 ribs) recognized, located and segmented, and the accurate position of the fetal conus medullaris is automatically located. The number of fetal ribs is automatically / semi-automatically counted, and the cross section, coronal section and three-dimensional VR diagram of the straightened cross section of all ribs or specified ribs are automatically imaged. The specific position of the conus medullaris in the lumbar vertebra is labeled on the three-dimensional VR diagram.

[0347] ​​The method greatly simplifies the workflow of fetal rib examination and conus medullaris position examination, liberates the doctor from tedious and complex manual operation, puts more effort into disease diagnosis, reduces the dependence on the doctor's technology, improves the examination efficiency; and the stability and imaging quality of the rib quantity statistical result and the conus medullaris positioning result are better than manual operation, reducing the misdiagnosis and missed diagnosis rate.

[0348] In an embodiment, an ultrasound imaging method is also provided. The ultrasound imaging method can be applied to the ultrasound imaging device described above. In the method, three-dimensional body data of fetal ribs can be automatically or semi-automatically identified from three-dimensional body data of a fetus based on characteristics of the fetal ribs, and then clinically valuable cross-sectional images such as coronal images, transverse images, longitudinal cross-sectional images of the ribs, etc. can be automatically or semi-automatically obtained based on the identified three-dimensional body data of the fetal ribs.

[0349] For example, in the embodiment, three-dimensional body data of a fetus can be obtained first. The three-dimensional body data of the fetus can be obtained in real time by an ultrasound imaging device, for example, the ultrasound imaging device transmits ultrasonic waves to the fetus through an ultrasonic probe and receives ultrasonic echoes, obtains ultrasonic echo signals, and a processor of the ultrasound device processes the ultrasonic echo signals to obtain the three-dimensional body data of the fetus. The three-dimensional body data of the fetus can also be obtained in advance by the ultrasound imaging device and stored, and read in for processing when it is needed to obtain cross-sectional images or three-dimensional images of fetal ribs.

[0350] Then, the processor can identify three-dimensional body data of fetal ribs from three-dimensional body data of a fetus based on characteristics of the fetal ribs. Here, the fetal ribs will exhibit specific characteristics on the ultrasound images due to their own characteristics. Therefore, the processor can identify the three-dimensional body data of the fetal ribs based on image characteristics of the fetal ribs (i.e. characteristics exhibited by the image data thereof), such as mean, variance, distribution characteristics, texture, morphological characteristics of gray scale or pixel or voxel values, etc.

[0351] Then, the processor can obtain a first plane (e.g. coronal plane) or first curved surface that passes through at least two ribs in the three-dimensional body data of the fetal ribs and is parallel to or coincides with the arrangement plane of multiple fetal ribs in the three-dimensional body data of the fetal ribs and / or obtain a second plane (e.g. transverse plane) or second curved surface that passes through at least one rib in the three-dimensional body data of the fetal ribs and intersects with the arrangement plane of multiple fetal ribs in the three-dimensional body data of the fetal ribs based on the identified three-dimensional body data of the fetal ribs, and then obtain an image on the first plane or first curved surface and / or obtain an image on the second plane or second curved surface based on the identified three-dimensional body data of the fetal ribs.

[0352] After the image on the first plane or the first curved surface and / or the image on the second plane or the second curved surface is obtained, the processor can display the image on the first plane or the first curved surface as a two-dimensional image and / or display the image on the second plane or the second curved surface as a two-dimensional image through the display.

[0353] In this embodiment, the method can further include obtaining a three-dimensional ultrasound image of the fetal rib according to the three-dimensional body data of the identified fetal rib, and displaying the three-dimensional ultrasound image of the fetal rib.

[0354] In this embodiment, the three-dimensional body data of the fetal spine and the three-dimensional body data of the fetal rib can be automatically identified from the three-dimensional body data of the first tissue to be detected based on a first rib detection model.

[0355] In this embodiment, the three-dimensional body data of the fetal rib can also be identified from the three-dimensional body data of the fetus based on the features of the fetal rib using a template matching method. For example, at least two first candidate regions can be determined from the three-dimensional body data of the fetus, the body data features of the three-dimensional body data of each first candidate region are obtained, and the first matching degree of each first candidate region with the fetal rib is determined according to the body data features of each first candidate region, and the first candidate region with the highest first matching degree is determined as the target region corresponding to the fetal rib, and the three-dimensional body data of the target region corresponding to the fetal rib is taken as the three-dimensional body data of the fetal rib.

[0356] In this embodiment, when identifying the three-dimensional body data of the fetal rib, the operation input by the user or other device connected to the ultrasound imaging device through a wired or wireless network can also be based on, that is, it can be semi-automatically performed. For example, a three-dimensional ultrasound image corresponding to the three-dimensional body data of the fetus can be displayed, a first input operation is received based on the three-dimensional ultrasound image corresponding to the three-dimensional body data of the fetus, a landmark point corresponding to the first input operation is determined, and the three-dimensional body data of the fetal rib is identified from the three-dimensional body data of the fetus according to the coordinates of the landmark point.

[0357] In this embodiment, the overall structure of the fetal rib and the fetal spine can also be identified first, and then the fetal rib is identified therefrom. For example, the three-dimensional body data of the fetal rib structure including the fetal rib and the fetal spine can be identified from the three-dimensional body data of the fetus, and then the three-dimensional body data of the fetal rib is identified from the three-dimensional body data of the fetal rib structure.

[0358] In this embodiment, the ribs can also be straightened, and then the image on the first plane or the first curved surface and / or the image on the second plane can be obtained based on the three-dimensional volume data of the straightened ribs. For example, the three-dimensional volume data of the recognized ribs of the fetus can be straightened to obtain the three-dimensional volume data of the straightened ribs, and then based on the three-dimensional volume data of the straightened ribs, the first plane or the first curved surface passing through at least two ribs of the three-dimensional volume data of the straightened ribs and parallel to or coinciding with the arrangement plane of the plurality of ribs of the fetus in the three-dimensional volume data of the straightened ribs can be obtained, and / or the second plane passing through at least one rib of the three-dimensional volume data of the straightened ribs and intersecting with the arrangement plane of the plurality of ribs of the fetus in the three-dimensional volume data of the straightened ribs can be obtained, and then based on the three-dimensional volume data of the straightened ribs, the image on the first plane or the first curved surface and / or the image on the second plane can be obtained.

[0359] In this embodiment, the fetus spine can also be recognized, and after the fetus ribs and the fetus spine are straightened, the image on the first plane and / or the second plane can be obtained based on the three-dimensional volume data of the straightened ribs. For example, based on the characteristics of the fetus spine, the three-dimensional volume data of the fetus spine can be recognized from the three-dimensional volume data of the fetus, and the recognized fetus ribs and the recognized fetus spine are usually connected to each other, so here the fetus ribs and the fetus spine are collectively referred to as the fetus rib structure. In this embodiment, the three-dimensional volume data of the fetus rib structure can be straightened, i.e. the three-dimensional volume data of the fetus ribs and the three-dimensional volume data of the fetus spine are straightened to obtain the three-dimensional volume data of the straightened rib structure, and accordingly the three-dimensional volume data of the straightened rib structure will include the three-dimensional volume data of the straightened ribs and the three-dimensional volume data of the straightened spine. Then, based on the three-dimensional volume data of the straightened rib structure, the first plane passing through the straightened ribs and the straightened spine and / or the second plane passing through at least one straightened rib and intersecting with the straightened spine can be obtained, and based on the three-dimensional volume data of the straightened rib structure, the image on the first plane and / or the image on the second plane can be obtained.

[0360] In this embodiment, when the image on the first plane is obtained, the image on the first plane can be obtained based on the data within a certain thickness range, so that the image on the first plane can display more information. For example, the three-dimensional volume data within a predetermined thickness range in the direction perpendicular to the first plane can be obtained from the three-dimensional volume data of the straightened ribs and / or the three-dimensional volume data of the straightened spine, and the image on the first plane can be obtained based on the three-dimensional volume data within the predetermined thickness range.

[0361] The three-dimensional volume data within the predetermined thickness range can be determined manually by the user. For example, the user can directly draw the predetermined thickness range on the displayed three-dimensional image or two-dimensional cross-sectional image of the straightened three-dimensional volume data by inputting device, or can determine a predetermined plane (e.g., a plane in which upper and lower edges of the pedicle or vertebral body are located, or other plane according to actual needs, etc.) on the displayed three-dimensional image or two-dimensional cross-sectional image of the straightened three-dimensional volume data by inputting device, and then take the region between the upper and lower edge planes as the predetermined thickness range, etc.

[0362] In this embodiment, the three-dimensional volume data within the predetermined thickness range can contain the pedicle and / or vertebral body of the fetal spine. At this time, the predetermined thickness range can also be manually set to contain the pedicle and / or vertebral body of the fetal spine. Alternatively, the predetermined thickness range can also be determined automatically or semi-automatically. For example, based on the characteristics of the pedicle and / or vertebral body of the spine, the pedicle and / or vertebral body can be identified from the straightened spine three-dimensional volume data, and the three-dimensional volume data within the predetermined thickness range can be determined to contain the identified pedicle and / or vertebral body, for example, three planes can be fitted according to the upper and lower edges of the identified pedicle and / or vertebral body, and the range between any two of the three planes can be taken as the aforementioned predetermined thickness range.

[0363] In this embodiment, the method of identifying the pedicle and / or vertebral body from the straightened spine three-dimensional volume data based on the characteristics of the pedicle and / or vertebral body of the spine can refer to the method of identifying a specific tissue structure from three-dimensional volume data in the aforementioned various embodiments, and similar methods can be used and will not be described one by one here.

[0364] In this embodiment, the image on the first plane can be obtained from the three-dimensional volume data within the predetermined thickness range in various suitable ways. For example, the three-dimensional volume data within the predetermined thickness range can be weighted in the direction perpendicular to the first plane to obtain the image on the first plane, etc. For example, all voxels within the thickness range on a path perpendicular to the first plane can be weighted and averaged to obtain the value of the pixel point on the first plane corresponding to the path in that direction, and similarly the values of all pixel points on the first plane can be obtained, i.e., the image on the first plane can be obtained.

[0365] In this embodiment, the straightened rib three-dimensional volume data can be obtained from the identified fetal rib three-dimensional volume data in various suitable ways. For example, the longitudinal axis of the identified fetal rib three-dimensional volume data can be determined, and the identified fetal rib three-dimensional volume data can be sampled according to the longitudinal axis to obtain a sequence of cross sections, and then the sequence of cross sections can be reconstructed along a straight line to obtain the straightened rib three-dimensional volume data.

[0366] In this embodiment, similarly, various suitable methods can be used to straighten the three-dimensional volume data of the identified fetal spine to obtain straightened three-dimensional volume data of the spine. For example, the longitudinal axis of the identified three-dimensional volume data of the fetal spine can be determined, and the three-dimensional volume data of the identified fetal spine can be sampled according to the longitudinal axis to obtain a section sequence. The section sequence can then be reconstructed along a straight line to obtain straightened three-dimensional volume data of the spine.

[0367] In this embodiment, the number of fetal ribs can also be automatically determined and displayed based on the identified three-dimensional volumetric data of the fetal ribs. The number of fetal ribs can be displayed as numbers or other suitable symbols.

[0368] In this embodiment, the fetal ribs can also be marked based on the identified three-dimensional volumetric data of the fetal ribs, and the markings of the fetal ribs can be displayed. The markings can be various suitable markings, such as the aforementioned T1, T2, etc., or other suitable text, numbers, symbols, colors, etc.

[0369] In this embodiment, the specific solutions for each step can refer to the methods in the foregoing embodiments or be the same as or similar to the steps in the foregoing embodiments, and will not be described in detail here.

[0370] In one embodiment, an ultrasound imaging method is provided, which can be applied to the aforementioned ultrasound imaging device. The method may include: acquiring three-dimensional volumetric data of the fetus; identifying the conus medullaris region from the three-dimensional volumetric data based on the characteristics of the fetal conus medullaris; determining the location of the conus medullaris region based on the identified region; and displaying the location of the conus medullaris region. Various suitable methods can be used to display the location of the conus medullaris region, such as using suitable symbols, colored areas, text, arrows, geometric shapes, etc.

[0371] In this embodiment, the location of the determined conus medullaris region can be the location of the end of the conus medullaris. For example, the location of the end of the conus medullaris can be determined based on the identified conus medullaris region and displayed. The location of the end of the conus medullaris can be displayed in various suitable ways, such as suitable composites, colors, dots, lines, arrows, numbers, distance from a suitable reference position, etc.

[0372] In this embodiment, the conus medullaris region can be identified using a target matching method. For example, at least two second candidate regions can be determined from the three-dimensional volume data of the fetus, the volume data features of the three-dimensional volume data of each second candidate region can be obtained, and based on the volume data features of each second candidate region, a second matching degree between each second candidate region and the conus medullaris can be determined, and the second candidate region with the highest second matching degree can be determined as the conus medullaris region.

[0373] In this embodiment, the fetal sagittal plane can be identified from the three-dimensional volume data of the fetus first, and then the conus medullaris region is identified from the sagittal plane image. For example, a sagittal plane image passing through the spine of the fetus can be determined from the three-dimensional volume data of the fetus according to the characteristics of the sagittal plane passing through the spine of the fetus, and then the conus medullaris region is determined in the sagittal plane image passing through the spine of the fetus based on the characteristics of the conus medullaris. Here, the sagittal plane can be the median sagittal plane and / or the sagittal plane adjacent to the median sagittal plane.

[0374] In this embodiment, the lumbar region of the fetus can also be identified from the three-dimensional volume data of the fetus, and the position of the conus medullaris region is displayed relative to the lumbar region, so that the user can conveniently see the relative position relationship between the conus medullaris and the lumbar region. For example, the lumbar region of the fetus can be identified from the three-dimensional volume data of the fetus based on the characteristics of the lumbar region of the fetus, an ultrasound image of the lumbar region is displayed, and the position of the conus medullaris region (e.g., the end of the conus medullaris, etc.) is displayed relative to the ultrasound image of the lumbar region. Here, displaying the position of the conus medullaris region relative to the ultrasound image of the lumbar region can include various suitable ways, for example, the ultrasound image of the lumbar region and the position of the conus medullaris region can be displayed simultaneously, so that the user can directly see the relative position relationship between the two, or the distance between the conus medullaris region and the lumbar region is displayed through text or symbols, or the relative position relationship between the two is displayed through symbols representing the lumbar region and the conus medullaris region, etc.

[0375] In this embodiment, the specific solutions of each step (e.g., identifying the conus medullaris region, identifying the lumbar region, identifying the sagittal plane of the fetus, etc.) can refer to the methods in the foregoing embodiments or be the same as or similar to the similar steps in the foregoing embodiments, which will not be repeated here.

[0376] The embodiments of the present application also provide an ultrasonic imaging device 10, as shown in the figure, comprising: Figure 1

[0377] a probe 100;

[0378] a transmitting circuit 101 for exciting the probe 100 to emit ultrasonic waves to a first to-be-measured tissue;

[0379] a receiving circuit 103 for receiving ultrasonic echoes returned from the first to-be-measured tissue through the probe 100 to obtain an ultrasonic echo signal;

[0380] a processor 105 for processing the ultrasonic echo signal to obtain a three-dimensional ultrasonic image of a marked fetal rib structure;

[0381] a display 106 for displaying the three-dimensional ultrasonic image of the marked fetal rib structure;

[0382] ​The processor 105 further performs the following steps:

[0383] According to the ultrasound echo information, three-dimensional volume data of the second to-be-measured tissue is acquired; three-dimensional volume data of the spinal cord cone and three-dimensional volume data of the lumbar vertebrae are identified from the second three-dimensional volume data; the three-dimensional volume data of the spinal cord cone and the three-dimensional volume data of the lumbar vertebrae are rendered to obtain a three-dimensional ultrasound image of the vertebrae structure; the spinal cord cone is marked in the three-dimensional ultrasound image of the vertebrae structure; and the three-dimensional ultrasound image of the marked vertebrae structure is output.

[0384] The embodiment of the present application further provides an ultrasonic imaging device 10, as shown in the accompanying drawings, comprising: Figure 1

[0385] a probe 100;

[0386] a transmitting circuit 101 configured to stimulate the probe 100 to emit ultrasonic waves to the second to-be-measured tissue;

[0387] a receiving circuit 103 configured to receive, by the probe 100, ultrasonic echoes returned from the second to-be-measured tissue to obtain an ultrasonic echo signal;

[0388] a processor 105 configured to process the ultrasonic echo signal to obtain a three-dimensional ultrasound image of the marked vertebrae structure;

[0389] a display 106 configured to display the three-dimensional ultrasound image of the marked vertebrae structure;

[0390] The processor 105 further performs the following steps:

[0391] According to the ultrasound echo information, three-dimensional volume data of the second to-be-measured tissue is acquired; three-dimensional volume data of the spinal cord cone and three-dimensional volume data of the lumbar vertebrae are identified from the second three-dimensional volume data; the three-dimensional volume data of the spinal cord cone and the three-dimensional volume data of the lumbar vertebrae are rendered to obtain a three-dimensional ultrasound image of the vertebrae structure; the spinal cord cone is marked in the three-dimensional ultrasound image of the vertebrae structure; and the three-dimensional ultrasound image of the marked vertebrae structure is output.

[0392] Correspondingly, the embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the ultrasonic imaging method.

[0393] The above ultrasonic imaging system and computer readable storage medium embodiments are similar to the description of the method embodiments, and have similar beneficial effects. For technical details not disclosed in the ultrasonic imaging system and computer readable storage medium embodiments of the present application, please refer to the description of the method embodiments for understanding.​

[0394] If the ultrasound imaging method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk, and various program code storage media. Thus, the embodiments of the present application are not limited to any specific hardware and software combination.

[0395] It should be understood that throughout the specification the terms "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Therefore, appearances of the phrases "in one embodiment" or "in an embodiment" at various places throughout the specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that the sequence of the processes described above does not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above sequence number of the embodiments of the present application is only for description, and does not represent the advantages or disadvantages of the embodiments.

[0396] It should be noted that in this document, the terms "comprise", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0397] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0398] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place, or distributed on multiple network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0399] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional units.

[0400] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instructing related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes mobile storage devices, read-only memories (ROM), magnetic discs or optical discs, and various storage media that can store program codes.

[0401] Alternatively, the integrated units of the present application, if implemented in the form of software functional modules and sold or used as independent products, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes mobile storage devices, ROM, magnetic discs or optical discs, and various storage media that can store program codes.

[0402] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An ultrasound imaging method, applied to an ultrasound imaging device, the method comprising: Obtain three-dimensional volumetric data of the fetus; Based on the characteristics of the fetal ribs, the three-dimensional volume data of the fetal ribs are identified from the three-dimensional volume data of the fetus. The characteristics of the fetal ribs are the specific features they exhibit on ultrasound images; Based on the identified three-dimensional volume data of the fetal ribs, a first plane or a first curved surface is automatically obtained that passes through at least two ribs in the three-dimensional volume data of the fetal ribs and is parallel or coincident with the arrangement surface of multiple fetal ribs in the three-dimensional volume data of the fetal ribs. A second plane or a second curved surface is also obtained that passes through at least one rib in the three-dimensional volume data of the fetal ribs and intersects with the arrangement surface of multiple fetal ribs in the three-dimensional volume data of the fetal ribs. Based on the identified three-dimensional volume data of the fetal ribs, an image is obtained on the first plane or the first curved surface, and an image is obtained on the second plane or the second curved surface; Display the image on the first plane or the first curved surface as a two-dimensional image and display the image on the second plane or the second curved surface as a two-dimensional image.

2. The method according to claim 1, characterized in that, Also includes: A three-dimensional ultrasound image of the fetal ribs is obtained based on the three-dimensional volume data of the identified fetal ribs. A three-dimensional ultrasound image of the fetal ribs is displayed.

3. The method according to claim 1 or 2, characterized in that, The three-dimensional volume data of the fetal ribs, identified from the three-dimensional volume data of the fetus based on the characteristics of the fetal ribs, includes: Based on the first rib detection model, the three-dimensional volume data of the fetal spine and the three-dimensional volume data of the fetus are identified from the three-dimensional volume data of the fetus.

4. The method according to claim 1 or 2, characterized in that, The three-dimensional volume data of the fetal ribs, identified from the three-dimensional volume data of the fetus based on the characteristics of the fetal ribs, includes: From the three-dimensional volume data of the fetus, at least two first candidate regions are determined, and the volume data features of the three-dimensional volume data of each first candidate region are obtained; Based on the volume data features of each first candidate region, a first matching degree between each first candidate region and the fetal rib is determined; The first candidate region with the highest first matching degree is determined as the target region corresponding to the fetal rib; The three-dimensional volume data of the target region corresponding to the fetal rib is used as the three-dimensional volume data of the fetal rib.

5. The method according to claim 1 or 2, characterized in that, The three-dimensional volume data of the fetal ribs, identified from the three-dimensional volume data of the fetus based on the characteristics of the fetal ribs, includes: Displays a three-dimensional ultrasound image corresponding to the three-dimensional volume data of the fetus; The first input operation is received based on the three-dimensional ultrasound image corresponding to the three-dimensional volume data of the fetus. Determine the marker point corresponding to the first input operation; The three-dimensional volume data of the fetal ribs are identified from the three-dimensional volume data of the fetus based on the coordinates of the marker points.

6. The method according to claim 1 or 2, characterized in that, The three-dimensional volume data of the fetal ribs, identified from the three-dimensional volume data of the fetus based on the characteristics of the fetal ribs, includes: Three-dimensional volume data of the fetal rib structure is identified from the three-dimensional volume data of the fetus, wherein the fetal rib structure includes the fetal ribs and the fetal spine; The three-dimensional volume data of the fetal ribs were identified from the three-dimensional volume data of the fetal rib structure.

7. The method according to claim 1, characterized in that, Based on the identified three-dimensional volume data of the fetal ribs, a first plane or first curved surface is obtained that passes through at least two ribs in the three-dimensional volume data of the fetal ribs and is parallel or coincident with the arrangement plane of multiple fetal ribs in the three-dimensional volume data of the fetal ribs; and a second plane or second curved surface is obtained that passes through at least one rib in the three-dimensional volume data of the fetal ribs and intersects with the arrangement plane of multiple fetal ribs in the three-dimensional volume data of the fetal ribs, including: The three-dimensional volume data of the identified fetal ribs is straightened to obtain the three-dimensional volume data of the straightened ribs; Based on the three-dimensional volume data of the straightened ribs, a first plane or a first curved surface is obtained that passes through at least two ribs in the three-dimensional volume data of the straightened ribs and is parallel to or coincides with the arrangement surface of multiple fetal ribs in the three-dimensional volume data of the straightened ribs; and a second plane is obtained that passes through at least one rib in the three-dimensional volume data of the straightened ribs and intersects with the arrangement surface of multiple fetal ribs in the three-dimensional volume data of the straightened ribs. Based on the identified three-dimensional volume data of the fetal ribs, images are obtained on the first plane or first curved surface and images are obtained on the second plane or second curved surface, including... Based on the three-dimensional data of the straightened ribs, an image is obtained on the first plane or the first curved surface, and an image is obtained on the second plane.

8. The method according to claim 1, characterized in that, Based on the identified three-dimensional volume data of the fetal ribs, a first plane or first curved surface is obtained that passes through at least two ribs in the three-dimensional volume data of the fetal ribs and is parallel or coincident with the arrangement plane of multiple fetal ribs in the three-dimensional volume data of the fetal ribs; and a second plane or second curved surface is obtained that passes through at least one rib in the three-dimensional volume data of the fetal ribs and intersects with the arrangement plane of multiple fetal ribs in the three-dimensional volume data of the fetal ribs, including: Based on the characteristics of the fetal spine, the three-dimensional volume data of the fetal spine is identified from the three-dimensional volume data of the fetus. Straighten the three-dimensional volume data of the fetal rib structure to obtain straightened three-dimensional volume data of the rib structure, wherein the three-dimensional volume data of the fetal rib structure includes the three-dimensional volume data of the fetal ribs and the three-dimensional volume data of the fetal spine, and the straightened three-dimensional volume data of the rib structure includes the straightened rib three-dimensional volume data and the straightened spine three-dimensional volume data. Based on the three-dimensional volume data of the straightened rib structure, a first plane passing through the straightened rib and the straightened spine is obtained, and a second plane passing through at least one straightened rib and intersecting with the straightened spine is obtained. Based on the identified three-dimensional volume data of the fetal ribs, images are obtained on the first plane or first curved surface and images are obtained on the second plane or second curved surface, including... Based on the three-dimensional volume data of the straightened rib structure, an image on the first plane is obtained, and an image on the second plane is obtained based on the three-dimensional volume data of the straightened rib structure.

9. The method according to claim 8, characterized in that, Based on the straightened rib three-dimensional volume data and the straightened spine three-dimensional volume data, an image on the first plane is obtained, including: Obtain three-dimensional volume data within a predetermined thickness range in a direction perpendicular to the first plane from the straightened rib three-dimensional volume data and the straightened spine three-dimensional volume data; The image on the first plane is obtained based on the three-dimensional volume data within the predetermined thickness range.

10. The method according to claim 8, characterized in that, Obtaining an image on the first plane based on three-dimensional volume data within a predetermined thickness range includes: weighting the three-dimensional volume data within the predetermined thickness range in a direction perpendicular to the first plane to obtain an image on the first plane.

11. The method according to claim 9 or 10, characterized in that, Also includes: Based on the characteristics of the vertebral arch and / or vertebral body of the spine, the vertebral arch and / or vertebral body are identified from the three-dimensional volume data of the straightened spine; The three-dimensional volume data within the predetermined thickness range is determined such that the three-dimensional volume data within the predetermined thickness range includes the identified vertebral arch and / or vertebral body.

12. The method according to any one of claims 7 to 10, characterized in that, The three-dimensional volume data of the identified fetal ribs is straightened to obtain the straightened rib three-dimensional volume data, including: Determine the longitudinal axis of the three-dimensional volume data of the identified fetal ribs; The three-dimensional volume data of the identified fetal ribs are sampled according to the longitudinal axis to obtain a section sequence; The cross-sectional sequence is reconstructed along a straight line to obtain the three-dimensional volume data of the straightened rib.

13. The method according to any one of claims 8 to 10, characterized in that, Straightening the identified three-dimensional volume data of the fetal spine to obtain straightened three-dimensional volume data of the spine includes: Determine the longitudinal axis of the three-dimensional volumetric data of the identified fetal spine; The three-dimensional volume data of the identified fetal spine are sampled according to the longitudinal axis to obtain a section sequence; The cross-sectional sequence is reconstructed along a straight line to obtain the three-dimensional volume data of the straightened spine.

14. The method according to any one of claims 1, 2, 7-10, characterized in that, The method further includes: The number of fetal ribs is determined based on the three-dimensional volumetric data of the identified fetal ribs; This shows the stated number of fetal ribs; and / or Fetal ribs are labeled based on the three-dimensional volumetric data of the identified fetal ribs; The markings show the fetal ribs.

Images