Method, device and storage medium for object analysis of medical images

Abstract

The present disclosure relates to a method, device and storage medium for object analysis of medical images. The method may include the following steps. A 3D medical image containing an object may be obtained. The 3D medical image may be segmented to obtain a segmentation result of the object. A set of image slices may be obtained in the 3D medical image in an extended direction. Internal representative points of the segmented object in each image slice in the set of image slices may be obtained. The internal representative points may be sparsely sampled, and a set of image blocks in the 3D medical image may be obtained based on the sampled internal representative points. Object analysis may be performed based on the set of image blocks. The method and device obtain image slices and internal representative points on a 3D medical image; and cut the 3D medical image into blocks based on the internal representative points. This method can improve the prediction efficiency while keeping the center point of the image block located in an organ or tissue (such as a blood vessel).

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number 2021115586203, application date December 20, 2021, and invention name “Method, device and storage medium for object analysis of medical images”. Technical Field

[0002] The present disclosure relates to the field of medical images, and more specifically, to a method, device and storage medium for performing object analysis on medical images. Background Art

[0003] Vascular disease has always been a major problem threatening human health. A considerable proportion of vascular diseases are caused by the accumulation of plaque lesions on the blood vessel walls, which causes vascular stenosis, and abnormal bulging on the blood vessel walls, which causes aneurysms. However, existing technologies have certain deficiencies in the detection and identification of vascular lesions.

[0004] Taking head and neck artery plaques as an example, head and neck artery disease usually refers to arterial stenosis or blockage caused by the accumulation of atherosclerotic plaques in the arterial wall. Patients with intracranial artery stenosis and blockage have limited blood supply to the brain, which can easily lead to ischemic stroke in patients. If the plaque ruptures, it is very easy to block and damage blood vessels, causing acute stroke in patients. According to the composition of atherosclerotic plaques, plaques can be further divided into calcified plaques, non-calcified plaques, and mixed plaques, among which mixed plaques have both calcified and non-calcified plaque components. Non-calcified and mixed plaques are prone to rupture.

[0005] Computed Tomography Angiography (CTA) or Magnetic Resonance Angiography (MRA) is a commonly used vascular imaging technique that can image blood vessels and their lesions in various parts of the body. Non-calcified plaques, mixed plaques, and aneurysms have low contrast with surrounding tissues on the images and are easily confused with surrounding tissues, resulting in missed detection.

[0006] At present, the detection methods of vascular lesions in head and neck CTA generally include manual analysis and automatic analysis software. Manual plaque analysis relies heavily on the experience of radiologists and cardiovascular experts. Atherosclerotic plaques, aneurysms and other lesions are discretely distributed on the complex structure of the head and neck artery walls. Analyzing vascular lesions in massive CTA data is undoubtedly an extremely time-consuming task for doctors. The uncertainty of non-calcified and mixed plaques increases the difficulty of doctors' diagnosis. In order to improve the efficiency and accuracy of vascular lesion diagnosis and liberate doctors from the tedious work of reading films, it is urgent to develop an automatic analysis solution for vascular lesions.

[0007] Existing vascular lesion analysis software can reduce doctors' daily diagnostic workload to a certain extent, but it also has certain shortcomings. For example, the semi-automatic analysis software provided by CT equipment manufacturers such as Siemens requires a lot of manual interaction to complete vascular segmentation, diameter estimation, and wall morphology analysis, and this solution generally only targets local blood vessels.

[0008] Recently, deep learning technology has been gradually applied to vascular lesion detection and has achieved remarkable results. However, the existing schemes generally use a single detection model. When a single model is used to predict lesions, the sensitivity (also referred to as sensitivity here) is high, but the precision (also referred to as precision here) is low. Generally, a classification model is used to further eliminate the false positive lesions found. However, the distribution of the detected lesion samples is extremely uneven, so the classification effect is extremely unstable. The existing scheme may also add false positive sample training to improve the detection accuracy. However, adding negative samples will result in a lower dice coefficient for the lesion detection results, which will affect the accuracy of lesion quantitative analysis. In addition, the existing scheme divides the CTA image into multiple image blocks at a fixed step, and then predicts lesions for each image block. This scheme has the problem of too many image blocks and low prediction efficiency; the image blocks selected by this scheme will predict lesions in non-vascular areas, and the prediction results are prone to more false positive lesions. Existing solutions for obtaining the center point of blood vessels generally rely on blood vessel skeletonization or centerline extraction. However, the calculation of blood vessel skeletonization is time-consuming, and the obtained center points are relatively redundant, which reduces the efficiency of model prediction. In addition, skeletonization is easily affected by the results of blood vessel segmentation, especially at the bifurcation of blood vessels, the distribution of skeletonization results is extremely uneven, and the center point of the selected image block is generally not in the blood vessel, which has an adverse effect on the prediction result; the blood vessel centerline extraction scheme not only needs to determine the starting point of the blood vessel, but is also easily affected by the results of blood vessel segmentation, resulting in centerline growth errors and affecting the extraction of blood vessel center points. For example, when the blood vessel segmentation results are discontinuous, the centerline extraction is very likely to fail. Summary of the invention

[0009] The present disclosure is provided to solve the above-mentioned problems existing in the prior art.

[0010] A method for object analysis of medical images is needed, which can obtain a set of image slices in the direction of extension in the 3D medical image, and then obtain internal representative points in each image slice; based on the internal representative points, the 3D medical image is cut into blocks. This method can improve the prediction efficiency while keeping the center point of the image block located in the blood vessel, so as to meet the actual situation that the center point of the lesion connected domain is located in the center of the blood vessel during the training of the detection model, so as to approach the optimal prediction effect of the detection model. At the same time, the present disclosure can improve the sensitivity and accuracy of vascular lesion detection, and make the predicted lesion results and lesion annotations have a higher dice coefficient, so as to improve the accuracy of lesion quantitative analysis.

[0011] According to a first scheme of the present disclosure, a method for performing object analysis on a medical image is provided, and the method for performing object analysis on a medical image may include obtaining a 3D medical image containing an object. The method may also include segmenting the 3D medical image to obtain a segmentation result of the object. The method may also include acquiring a set of image slices in the 3D medical image in an extended direction. The method may also include acquiring internal representative points of the segmented object in each image slice in the set of image slices. The method may also include acquiring a set of image blocks in the 3D medical image based on a set of internal representative points of the object in the set of image slices. The method may also include performing object analysis based on the set of image blocks.

[0012] According to a second embodiment of the present disclosure, a device for performing object analysis on a medical image is provided, and the device for performing object analysis on a medical image may include an interface and a processor. The interface may be configured to acquire a 3D medical image containing an object. The processor may be configured to (e.g., via an interface) include acquiring a 3D medical image containing an object. The processor may also be configured to segment the 3D medical image to obtain a segmentation result of the object. The processor may also be configured to acquire a set of image slices in the 3D medical image in an extending direction. The processor may also be configured to acquire internal representative points of the segmented object in each image slice in the set of image slices. The processor may also be configured to acquire a set of image blocks in the 3D medical image based on a set of internal representative points of the object in the set of image slices. In addition, the processor may be configured to perform object analysis based on the set of image blocks.

[0013] According to a third scheme of the present disclosure, a computer storage medium is provided, on which executable instructions are stored, and when the executable instructions are executed by a processor, the steps of a method for object analysis on a medical image are implemented. The method may include acquiring a 3D medical image containing an object. The method may also include segmenting the 3D medical image to obtain a segmentation result of the object. The method may also include acquiring a group of image slices in the 3D medical image in an extended direction. The method may also include acquiring internal representative points of the segmented object in each image slice in the group of image slices. The method may also include acquiring a group of image blocks in the 3D medical image based on a group of internal representative points of the object in the group of image slices. The method may also include performing object analysis based on the group of image blocks.

[0014] According to the method, device and storage medium for object analysis of medical images according to various embodiments of the present disclosure, for example, for lesion detection of blood vessels (as an example of an object), the present disclosure has the following advantages over existing solutions:

[0015] The present disclosure calculates the center point of the blood vessel (as an example of an internal representative point) with reference to the blood vessel segmentation result in a more reasonable way, by sampling a set of image slices in the extension direction, obtaining the center point in the slice, and then sparsely sampling the obtained center point to obtain the blood vessel center point. A set of image blocks can be obtained by cropping in this way for subsequent vascular lesion analysis. In this way, the problems of redundant center points obtained by the traditional skeletonization scheme and inaccurate center point positioning at the bifurcation of blood vessels can be avoided, and the speed is significantly improved compared with the skeletonization scheme. In addition, it can also avoid the problem that the centerline extraction scheme is overly dependent on the blood vessel segmentation result, such as the failure of centerline extraction when the segmentation result is discontinuous.

[0016] The present disclosure does not rely on complex manual interactions and can accomplish accurate and efficient detection of vascular lesions in a series of images containing extended organs or tissues (e.g., blood vessels). This series of images, such as a head and neck CTA image sequence, contains the head, neck arteries, and aortic arch, each of which has a large number of branches. The object analysis method of the present disclosure can achieve accurate and efficient detection of vascular lesions in a complex vascular tree.

[0017] Compared with the automatic analysis software of vascular lesions based on deep learning, the present invention has a more reasonable reference to the vascular segmentation results. With the vascular center point calculation scheme proposed in the present invention, the image slices are sampled along the x, y, z directions (or the directions of the adaptive calculation of the main components) at a certain step, and the vascular center points in the slices are obtained, and then the obtained center points are sparsely sampled to obtain the proposed vascular center reference points. The image blocks cropped in this way can avoid the problems of redundant center points obtained by the traditional skeletonization scheme and inaccurate center point positioning at the vascular bifurcation. This scheme is also faster than the skeletonization scheme; in addition, it can also avoid the problem that the centerline scheme is overly dependent on the vascular segmentation results, such as the failure of centerline extraction when the segmentation results are discontinuous. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar components in different views. The drawings generally illustrate various embodiments by way of example and not limitation, and together with the description and claims, serve to illustrate the disclosed embodiments. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0019] Figure 1 A method for performing object analysis on a medical image according to an embodiment of the present disclosure is shown.

[0020] Figure 2 The window adjustment and block cutting process of a 3D medical image according to an embodiment of the present disclosure is shown.

[0021] Figure 3 The object analysis process based on a group of image blocks according to an embodiment of the present disclosure is shown.

[0022] Figure 4 The lesion detection process of a 3D medical image according to an embodiment of the present disclosure is shown.

[0023] Figure 5 The mask dice coefficient optimization process for lesion prediction of 3D medical images according to an embodiment of the present disclosure is shown.

[0024] FIG. 6( a ) shows a blood vessel segmentation result according to an embodiment of the present disclosure.

[0025] FIG. 6( b ) shows a blood vessel center reference point according to an embodiment of the present disclosure.

[0026] FIG. 7( a ) shows the rib segmentation result according to another embodiment of the present disclosure.

[0027] FIG. 7( b ) shows the center point of a rib according to another embodiment of the present disclosure.

[0028] Figure 8 The figure illustrates an illustrative block diagram of an exemplary apparatus for performing object analysis on a medical image according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the present disclosure is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present disclosure. For the various steps described herein, if there is no necessity for a causal relationship between each other, the order in which they are described as examples herein should not be regarded as a limitation, and those skilled in the art should know that they can be adjusted in order as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0030] Figure 1 A method for performing object analysis on a medical image according to an embodiment of the present disclosure is shown. Figure 2 FIG. 2 shows the window adjustment and block cutting process of a 3D medical image according to an embodiment of the present disclosure. Figure 1 and 2As shown, a method for object analysis of medical images begins with step S1, and a 3D medical image containing an object is obtained. The object may be any organ or tissue extending a certain length, such as but not limited to at least one of a blood vessel, a digestive tract, a mammary duct, a respiratory tract, or a lesion therein. The lesion is a lesion or abnormality such as an atherosclerotic plaque, an aneurysm, a stent, etc. in a blood vessel. The 3D medical image is a CTA image containing blood vessels, a CTA image containing ribs, or a CTA image containing lungs. In this embodiment, blood vessels are used as an example of an object for illustration. The 3D medical image is a CTA image containing blood vessels, and this embodiment is used to illustrate the detection of vascular aneurysm lesions. As another possible embodiment, the 3D medical image is a CTA image containing ribs. As yet another possible embodiment, the 3D medical image is a CTA image containing lungs.

[0031] 3D medical images need to conform to the medical imaging format of digital imaging and meet the Digital Imaging and Communications in Medicine (DICOM) protocol. 3D medical images also need to meet the basic requirements of CTA images, such as no contrast agent filling and no obvious motion artifacts.

[0032] In step S2, the 3D medical image can be windowed (step S201) and normalized (step S202) by preset window width and window position. Note that step S2 is optional. Among them, the window width and window position can be set as follows. The grayscale value of a known 3D medical image with a lesion annotation mask can be obtained. A histogram of the grayscale values ​​of the known 3D medical image can be obtained, and then the total cumulative value of the vertical axis in the histogram can be obtained; wherein the horizontal axis of the histogram is the grayscale value, and the vertical axis is the grayscale cumulative value. A threshold value can be set, such as 0.05×total cumulative value, and the vertical axis of the histogram is accumulated from the left and right sides of the histogram. If the accumulated value exceeds the preset threshold, the corresponding left and right horizontal axis coordinates are respectively taken as the statistical minimum and maximum grayscale values ​​min_v, max_v. The window width and window position can be calculated by the minimum and maximum grayscale values ​​min_v, max_v. The formula for calculating the window width and window position by the minimum and maximum grayscale values ​​min_v, max_v is as follows:

[0033] Window width ww(window width)=max_v-min_v,

[0034] Window level wl(window level)=(max_v-min_v) / 2.

[0035] Through step S2, the 3D medical image is windowed and normalized, so that the 3D medical image becomes a more standard and more regulated image to be processed, which is beneficial to the processing of subsequent steps, such as segmenting the 3D medical image.

[0036] In step S3, the 3D medical image may be segmented (step S203) to obtain a segmentation result of the object; wherein FIG6(a) shows a blood vessel segmentation result according to an embodiment of the present disclosure. FIG7(a) shows a rib segmentation result according to another embodiment of the present disclosure.

[0037] In this embodiment, the 3D medical image is segmented by a vascular segmentation model to obtain a CTA image vascular segmentation result, wherein the vascular segmentation model is obtained by training data containing vascular labels. In some embodiments, the vascular segmentation model can adopt a 3D U-Net network structure. The training method of the vascular segmentation model may include: based on experienced radiologists, the blood vessels in the training sample images are marked as a gold standard during training. Then the marked training sample images are input into the vascular segmentation model to obtain the vascular segmentation result, and the loss between the vascular segmentation result and the gold standard is calculated. The network parameters of the vascular segmentation model are adjusted according to the loss. When the loss is less than or equal to a preset threshold or convergence is reached, it indicates that the vascular segmentation model training converges. Optionally, the Dice loss function, the cross entropy loss function or other types of loss functions can be used when calculating the loss; when adjusting the network parameters, the stochastic gradient descent SGD optimizer or other types of optimizers can be used, which are not specifically limited here.

[0038] In step S4, a set of image slices may be acquired in the 3D medical image in the extending direction (see, for example, Figure 2 Step S204 in the figure); wherein the extension direction includes any one of the direction along the coordinate axis, the principal component direction obtained by performing principal component analysis on the segmentation result of the object, and a preset direction. For 3D medical images of different parts or organs, a specific extension direction is selected in a targeted manner to obtain image slices, so that more reasonable image slices can be obtained. For example, for a 3D medical image with uniform blood vessel distribution, a group of image slices are obtained along the direction of the coordinate axis of the image, which can be faster and more efficient; for an object with regular distribution of components, a group of image slices are obtained along the principal component direction of the image, which will provide a more accurate and reasonable reference to the blood vessel segmentation result.

[0039] In this embodiment, the extending direction is along the coordinate axes (X, Y and Z axes).

[0040] In step S5, the internal representative points of the segmented objects in each image slice in the set of image slices may be obtained (for example, see Figure 2 Step S205 in the figure). Step S5 may adopt various methods to obtain the internal representative points of the segmented object.

[0041] In some embodiments, a connected domain can be marked for each image slice and internal representative points of the connected domain can be obtained; the internal representative points of the connected domain of each image slice are sparsely sampled to obtain a sparse set of internal representative reference points. Sparse sampling of the internal representative points of the connected domain of each image slice to obtain a sparse set of internal representative reference points specifically includes: sparse sampling of a set of internal representative points, and expansion of each internal representative point after sparse sampling to obtain an expanded mask of each internal representative point; marking a connected domain for the expanded mask of each internal representative point, and sampling another set of internal representative points in the marked connected domain as a sparse set of internal representative reference points, so that the internal representative reference points can be ensured to be inside the object to be detected. The image blocks cropped according to the internal representative reference points can avoid the problems of redundant center points obtained by the traditional skeletonization scheme, inaccurate center point positioning of complex parts or tissues (such as vascular bifurcations), etc. This scheme is also faster than the skeletonization scheme; in addition, it can also avoid the problem that the centerline scheme is overly dependent on the vascular segmentation result, such as failure of centerline extraction when the segmentation result is discontinuous.

[0042] The following is a specific description using the example of the center point as the internal representative point. The following steps can be used to obtain the center point.

[0043] 1) Based on the blood vessel segmentation mask, image slices are acquired at certain intervals (e.g., 10 mm) along the extension direction, such as the x, y, and z axes;

[0044] 2) Mark the connected domain for each image slice, and obtain the midpoint of the connected domain as the candidate center point;

[0045] 3) Repeat step 2) along the extension direction, such as the x, y, and z axes, to obtain the midpoints of the plurality of image blocks. In another embodiment, the extension direction is a principal component direction obtained by performing principal component analysis on the segmentation result of the object, specifically, performing principal component analysis on a plurality of image slices, calculating the blood vessel distribution in the image slices, and obtaining the first principal component direction of the plurality of image slices.

[0046] 4) The center point of the sparse image slice is set to a certain side length, and the center point is expanded into a cube according to the set side length to obtain an extended mask of the center point of the image;

[0047] 5) Mark the connected domain of the extended mask of the image center point, and obtain the center point of the side length again. This center point is the center point of the sparse image block.

[0048] Among them, Figure 6(b) shows the center reference point of the blood vessel according to an embodiment of the present disclosure. Figure 7(b) shows the center point of the rib according to another embodiment of the present disclosure. By comparison, it can be seen that when dealing with the lung nodule detection scenario, the lung segmentation result can be obtained by segmentation, and then the threshold is used to initially screen the nodule mask. At this time, the initially screened nodule mask is a discontinuous tubular connected domain, and the center point of the connected domain cannot be obtained by extracting the center line. Obviously, the center reference point calculation scheme proposed in this scheme can cope with this scenario.

[0049] In step S6, a set of image blocks in the 3D medical image may be obtained based on a set of internal representative points of the object in the set of image slices. Step S6 may specifically include: based on a set of internal representative reference points, cropping the 3D medical image according to a preset size to obtain a set of image blocks (for example, see Figure 2 Step S206 in , or see Figure 4 In step S401 of FIG. 4 , the present embodiment uses the internal representative reference point as the center point and expands outward to crop the 3D medical image. This ensures that the cropped image block contains the object to be detected, avoids the problems of redundant center points obtained by the traditional skeletonization scheme, inaccurate center point positioning of complex parts or tissues (such as vascular bifurcations), and is faster than the skeletonization scheme. In addition, it can also avoid the problem that the centerline scheme is overly dependent on the vascular segmentation result, such as failure of centerline extraction when the segmentation result is discontinuous.

[0050] In step S7, object analysis may be performed based on the set of image blocks.

[0051] The disclosed embodiment obtains a set of image slices in the extended direction in the 3D medical image, calculates the internal representative points of the object on the image slices in a more reasonable manner with reference to the blood vessel segmentation results, and then sparsely samples the obtained internal representative points. Finally, a set of image blocks can be obtained by cropping according to the sampled internal representative points for subsequent lesion analysis. In this way, it can be ensured that the image blocks for lesion analysis contain objects, and it can also avoid the problems of redundant center points obtained by traditional skeletonization schemes and inaccurate center point positioning at blood vessel bifurcations, which significantly improves the speed compared to the skeletonization scheme. In addition, it can also avoid the problem that the centerline extraction scheme is overly dependent on the blood vessel segmentation results, such as the failure of centerline extraction when the segmentation results are discontinuous.

[0052] Figure 3 The object analysis process based on a group of image blocks according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, performing object analysis based on the set of image blocks specifically includes:

[0053] In step S301, a first lesion analysis result of the object may be determined based on the set of image blocks using a first model.

[0054] In some embodiments, Figure 4 As shown, step S301 may specifically include: based on the group of image blocks, using the first model to perform analysis image block by image block, and then obtaining the first lesion analysis result through false positive suppression processing, that is, predicting image block by image block through the first lesion detection model (first model) (step S402), and then obtaining the first CTA image lesion prediction result (i.e., lesion prediction mask0 or the first analysis result) through the false positive suppression module (step S403). Among them, the first lesion detection model uses image blocks containing lesion annotation information as training samples for training, and the training process includes data set preparation and model tuning. First, the lesion annotation data is divided into a training set, a tuning set, and a test set according to a certain ratio, and the bounding box (bbox) of the connected domain containing the lesion annotation mask in the training set and the tuning set is determined, and the block is cut based on the bounding box. The cutting method is to randomly obtain a certain x, y, z coordinate of the bbox as the center point, and cut along the center point according to a certain side length to obtain a number of lesion CTA image blocks and image blocks with lesion annotation masks as positive samples required for training and tuning.

[0055] In some embodiments, the first lesion detection model may use 3D UNet, and the lesion detection model training method may include: inputting the CTA image block in the above training set into the model to obtain the lesion segmentation result, and calculating the loss between the lesion detection result and the image block with the lesion annotation mask. The network parameters of the model are adjusted according to the loss, and convergence is reached when the loss is less than a preset threshold. The loss function generally uses Dice, cross entropy or focal loss function, and the network optimization method generally uses stochastic gradient descent method SGD, which is not specifically limited here.

[0056] The false positive suppression module uses prior knowledge to remove some false positive connected domains in the lesion prediction mask, such as removing connected domains whose intersection area with the vascular segmentation mask is less than 20% of the connected domain area, and removing connected domains whose connected domain area is less than a certain threshold.

[0057] In step S302, each connected domain of the lesion can be determined based on the first lesion analysis result of the object, and a group of internal representative points of each connected domain can be obtained; based on the group of internal representative points of each connected domain, another group of image blocks in the 3D medical image can be obtained. That is, the connected domain of the lesion prediction mask0 can be calculated, the center point of the connected domain can be obtained (step S404), and the CTA windowed image can be cropped along the center point according to the preset side length to obtain another group of image blocks (step S405).

[0058] In step S303, a second analysis result of the lesion of the object can be determined based on the other group of image blocks using the second model. In some embodiments, step S303 may specifically include: based on the other group of image blocks, using the second model to perform analysis image block by image block, and then obtaining the second analysis result through false positive suppression processing. That is, the second lesion detection model (second model) predicts image block by image block (step S406), and then obtains the second lesion prediction result (i.e., lesion prediction mask1 or the second analysis result) through the false positive suppression module (step S407), wherein the second lesion detection model uses the image blocks containing the annotation information of the lesions together with the false positive image blocks obtained by the analysis of the first model as training samples for training. The training process of the second lesion detection model also includes data set preparation and model tuning. The difference in data preparation is that the training set also needs to add the false positive image blocks predicted by the first lesion detection model. Specifically, the connected domains of the lesion prediction mask0 are compared with the corresponding annotated masks one by one to determine whether the overlapping area between the connected domain and the lesion in the annotated mask is greater than 20% of the area of ​​the connected domain. If not, it is determined to be a false positive connected domain. The false positive connected domain bbox is cut to obtain the false positive samples required for training, and the false positive samples are added to the positive sample training set of the second lesion detection model to form new training samples.

[0059] The same false positive suppression module is then used to optimize the lesion prediction mask1. Similarly, the third lesion detection model, the fourth lesion detection model, etc. can be iteratively obtained to further improve the false positive suppression effect.

[0060] It should be noted that the proposed second lesion detection model relies on the negative image block samples predicted by the first lesion detection model to improve the accuracy of lesion detection, which is different from the existing scheme of randomly sampling negative samples on CTA images; in addition, the iterative process proposed in the present disclosure has the characteristics of sensitivity of the prediction results of the first lesion detection model and high lesion DICE coefficient, and the second lesion detection model has high accuracy. The proposed DICE coefficient optimization is also particularly suitable for this scenario.

[0061] In step S304, each connected domain corresponding to the first lesion analysis result may be compared with the second lesion analysis result, and the first lesion analysis result of the area where the overlap between each connected domain and the second lesion analysis result is greater than a predetermined threshold is retained as the lesion analysis result of the object.

[0062] Figure 5 FIG. 2 shows a dice coefficient optimization process for a lesion prediction mask of a 3D medical image according to an embodiment of the present disclosure. Figure 5As shown, step S304 specifically includes, for each connected domain of the first lesion analysis result: obtaining the connected domain and its bounding box (step S501); based on the bounding box of the connected domain, cropping the second lesion analysis result to obtain the corresponding image block; calculating the overlap area between the connected domain and the corresponding image block obtained by cropping (step S502), if the overlap area is less than or equal to a predetermined threshold, then clearing the connected domain of the first lesion analysis result (step S503); if the overlap area is greater than the predetermined threshold, then retaining the connected domain of the first lesion analysis result. For example, the lesion prediction mask0 can be compared with the mask1 predicted by the second lesion detection model one by one, and the area where the overlap area between the connected domain in the lesion prediction mask0 and the lesion in mask1 is greater than 20% of the volume of the connected domain is retained, and the processed lesion prediction mask0 is the final lesion prediction mask.

[0063] The specific implementation steps are:

[0064] 1) Obtain several lesion connected domains in the lesion prediction mask0, and perform operations according to steps 2)-6) for each connected domain (step S504);

[0065] 2) Get the bounding box bbox_i of the i-th connected domain and the connected domain mask0_i corresponding to the bounding box;

[0066] 3) Get the volume of the i-th connected domain, volume_i = sum(mask0_i);

[0067] 4) Based on the bounding box bbox_i, cut the block on the lesion prediction mask1 to obtain the corresponding mask1_i;

[0068] 5) Get the overlap area of ​​the i-th connected domain on mask0 and mask1, where overlap_volume_i = sum(mask1_i[mask0_i>0]);

[0069] 6) If overlap_volume_i<0.2·volume_i, clear the connected domain in mask0. Note that the threshold of 0.2 for area overlap is only an example, and other thresholds may also be set.

[0070] As another possible embodiment, this solution can also be used for rib fracture detection, to obtain a number of rib center reference points, and then set a certain side length to crop the CTA windowed image along the center point to obtain a number of image blocks.

[0071] The first lesion detection model and the false positive suppression module are used to obtain the first lesion prediction result for the acquired image block, and the center point of the connected domain in the first lesion prediction result is obtained. Then, the CTA window adjustment image is cut to obtain several image blocks. The first lesion detection model has a high sensitivity because it is trained with all positive image block samples. The second lesion detection model and the false positive suppression module are then used to obtain the second lesion prediction result. The second lesion detection model uses positive and negative samples predicted by the first lesion detection model as training sets, which can improve the accuracy. Since false positive lesions are added during the training of the second lesion detection model, the dice coefficient of the predicted lesion result is lower than that of the first lesion detection model. Finally, it is proposed to optimize the predicted lesion result, remove the connected domain with a small overlap area between the lesion connected domain in the first lesion prediction result and the lesion area of ​​the second lesion prediction result, and the optimized first lesion prediction result is the final lesion prediction result. This optimization can ensure that the algorithm has a high sensitivity and accuracy while the detected lesion connected domain still has a high dice coefficient.

[0072] The first lesion detection model of the disclosed embodiment is trained with all-positive image block samples to improve the detection sensitivity and the dice coefficient of the lesion. The negative image block samples of the training set predicted by the first lesion detection model are then iteratively added to the training set to train the second lesion detection model to improve the accuracy of the lesion. In order to further improve the detection accuracy, the third lesion detection model can be iteratively obtained. In order to avoid the decrease of the dice of the lesion after adding negative samples, a dice coefficient scheme is proposed to optimize the lesion prediction result obtained by the first lesion detection model, that is, the connected domain with a small overlap area between the lesion connected domain in the first lesion prediction result and the lesion area of ​​the second lesion prediction result is removed, and the optimized first lesion prediction result is the final lesion prediction result. False positive suppression processing is performed on both the first lesion detection model and the second lesion detection model. The false positive suppression processing will eliminate false positive lesions in the non-vascular area and the volume less than a certain threshold in the result.

[0073] In summary, this embodiment first adjusts the window and normalizes the CTA image by using the acquired window width and window position. Then, the vascular segmentation model is used to obtain the vascular segmentation result, and then the image slices are sampled along the x, y, and z directions at a certain step, and the center points of the blood vessels in the slices are obtained. Then, the acquired center points are sparsely sampled to obtain the proposed vascular center reference points, and the CTA windowed image is cut accordingly to obtain several image blocks. The center point calculation scheme of this scheme improves the prediction efficiency while keeping the center point of the image block in the blood vessel, so as to conform to the actual situation that the center point of the lesion connected domain is located in the center of the blood vessel during model training, so as to approach the optimal prediction effect of the model.

[0074] Figure 8 FIG. 1 is a block diagram illustrating an exemplary apparatus for performing object analysis on a medical image according to an embodiment of the present disclosure, Figure 8 As shown, an object analysis device 800 includes an interface 807 and a processor 801. The interface 807 may be configured to receive a 3D medical image containing an object. The processor 801 may be configured to: execute the method for performing object analysis on a medical image according to various embodiments of the present disclosure.

[0075] Through the interface 807, the device for object analysis of medical images can be connected to a network (not shown), such as but not limited to a local area network or the Internet in a hospital. However, the communication mode implemented by the interface 807 is not limited to the network, and may include NFC, Bluetooth, WIFI, etc.; it may be a wired connection or a wireless connection. Taking the network as an example, the interface 807 can connect the device for object analysis of medical images to external devices such as an image acquisition device (not shown), a medical image database 808, and an image data storage device 809. The image acquisition device can be any type of imaging modality, such as but not limited to computed tomography (CT), digital subtraction angiography (DSA), magnetic resonance imaging (MRI), functional MRI, dynamic contrast enhancement-MRI, diffusion MRI, spiral CT, cone beam computed tomography (CBCT), positron emission tomography (PET), single photon emission computed tomography (SPECT), X-ray imaging, optical tomography, fluorescence imaging, ultrasound imaging, and radiotherapy field imaging.

[0076] In some embodiments, the object analysis device 800 may be a dedicated intelligent device or a general intelligent device. For example, the object analysis device 800 may be a computer customized for image data acquisition and image data processing tasks, or a server placed in the cloud. For example, the device 800 is integrated into an image acquisition device.

[0077] The object analysis apparatus 800 may include a processor 801 and a memory 804 , and may further include at least one of an input / output 802 and an image display 803 .

[0078] The processor 801 may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor 801 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor 801 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc. As will be appreciated by those skilled in the art, in some embodiments, the processor 801 may be a special-purpose processor rather than a general-purpose processor. The processor 801 may include one or more known processing devices, such as those from Intel. TM Pentium manufactured TM 、Core TM , Xeon TM or Itanium series of microprocessors, Turion manufactured by AMDTM TM , Athlon TM 、Sempron TM 、Opteron TM FX TM 、Phenom TM series, or various processors manufactured by Sun Microsystems. Processor 801 may also include a graphics processing unit, such as from GPUs by Nvidia TM Manufactured series, by Intel TM Made by GMA, Iris TM series, or by AMD TM Made by Radeon TM Processor 801 may also include an accelerated processing unit, such as an AMD TM Desktop A-4(6,6) series manufactured by Intel TM Xeon Phi TMSeries. The disclosed embodiments are not limited to any type of processor or processor circuit that is otherwise configured to acquire a 3D medical image containing an object; segment the 3D medical image to obtain a segmentation result of the object; acquire a set of image slices in the 3D medical image in an extended direction; acquire internal representative points of the segmented object in each image slice in the set of image slices; acquire a set of image blocks in the 3D medical image based on a set of internal representative points of the object in the set of image slices; perform object analysis based on the set of image blocks; or manipulate any other type of data consistent with the disclosed embodiments. In addition, the term "processor" or "image processor" may include more than one processor, for example, a multi-core design or multiple processors, each having a multi-core design. The processor 801 can execute a sequence of computer program instructions stored in the memory 804 to perform various operations, processes, and methods disclosed herein.

[0079] The processor 801 may be communicatively coupled to the memory 804 and configured to execute computer executable instructions stored therein. The memory 804 may include a read-only memory (ROM), a flash memory, a random access memory (RAM), a dynamic random access memory (DRAM) such as a synchronous DRAM (SDRAM) or a Rambus DRAM, a static memory (e.g., a flash memory, a static random access memory), etc., on which the computer executable instructions are stored in any format. In some embodiments, the memory 804 may store computer executable instructions of one or more image processing programs 805. The computer program instructions may be accessed by the processor 801, read from the ROM or any other suitable memory location, and loaded into the RAM for execution by the processor 801. For example, the memory 804 may store one or more software applications. The software applications stored in the memory 804 may include, for example, an operating system (not shown) for a general computer system and an operating system for a soft control device.

[0080] In addition, the memory 804 may store the entire software application or only a portion of the software application (e.g., image processing program 805) that can be executed by the processor 801. In addition, the memory 804 may store a plurality of software modules for implementing the various steps of the method for object analysis on a medical image consistent with the present disclosure or the process for training the first model and the second model.

[0081] In addition, the memory 804 may store data generated / buffered when executing the computer program, for example, medical image data 806, including medical images sent from an image acquisition device, a medical image database 808, an image data storage device 809, etc. In some embodiments, the medical image data 806 may include a 3D medical image containing an object to be analyzed, and the image processing program 805 will segment it, obtain image slices, obtain internal representative points, crop image blocks, and perform object analysis.

[0082] In some embodiments, an image data storage device 809 may be provided to exchange image data with a medical image database 808, and the memory 804 may communicate with the medical image database 808 to obtain a medical image including a plurality of parts for which blood vessel segmentation is to be performed. For example, the image data storage device 809 may reside in other medical image acquisition devices (e.g., a CT scan performed on the patient). The medical image of the patient may be transmitted and stored in the medical image database 808, and the object analysis device 800 may obtain the medical image of a specific patient from the medical image database 808 and perform object analysis on the medical image of the specific patient.

[0083] In some embodiments, the memory 804 may communicate with the medical image database 808 to transmit and store the object segmentation result together with the obtained object analysis result into the medical image database 808 .

[0084] In addition, the parameters of the trained first model and / or second model can be stored in the medical image database 808 so as to be accessed, acquired and used by other object analysis devices when necessary. In this way, when facing a patient, the processor 801 can obtain the trained first model and / or second model of the corresponding population, so as to perform blood vessel segmentation based on the obtained trained model.

[0085] In some embodiments, the first model and / or the second model (particularly the learning network) may be stored in the memory 804. Alternatively, the learning network may be stored in a remote device, a separate database (such as a medical image database 808), a distributed device, and may be used by the image processing program 805.

[0086] In addition to displaying medical images, the image display 803 can also display other information, such as object segmentation results, center point calculation results, and object analysis results. For example, the image display 803 can be an LCD, CRT, or LED display.

[0087] Input / output 802 may be configured to allow object analysis device 800 to receive and / or send data. Input / output 802 may include one or more digital and / or analog communication devices that allow the device to communicate with a user or other machines and devices. For example, input / output 802 may include a keyboard and mouse that allow a user to provide input.

[0088] In some embodiments, the image display 803 may present a user interface so that the user may conveniently and intuitively modify (such as edit, move, modify, etc.) the generated anatomical labels using the input / output 802 together with the user interface.

[0089] The interface 807 may include a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transmission adapter such as an optical fiber, USB 6.0, Lightning, a wireless network adapter such as a Wi-Fi adapter, a telecommunications (6G, 4G / LTE, etc.) adapter. The device may be connected to a network through the interface 807. The network may provide a local area network (LAN), a wireless network, a cloud computing environment (e.g., software as a service, a platform as a service, an infrastructure as a service, etc.), a client-server, a wide area network (WAN), etc.

[0090] The embodiments of the present disclosure also provide a computer storage medium on which computer executable instructions are stored, and when the computer executable instructions can be executed by a processor, the method of performing object analysis on a medical image according to various embodiments of the present disclosure is implemented. The storage medium may include a read-only memory (ROM), a flash memory, a random access memory (RAM), a dynamic random access memory (DRAM) such as a synchronous DRAM (SDRAM) or a Rambus DRAM, a static memory (e.g., a flash memory, a static random access memory), etc., on which computer executable instructions can be stored in any format.

[0091] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present disclosure with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the present disclosure, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0092] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, a person of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present disclosure. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present disclosure may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0093] The above embodiments are only exemplary embodiments of the present disclosure and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present disclosure, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A method for object analysis of a medical image, characterized in that: include: Acquire a 3D medical image containing an object; Segmenting the 3D medical image to obtain a segmentation result of the object; Acquiring a set of image slices in the 3D medical image in an extending direction; Acquire internal representative points of the segmented objects in each image slice in the set of image slices; Sparsely sampling the internal representative points, and acquiring a group of image blocks in the 3D medical image based on the sampled internal representative points; Based on the set of image blocks, determining a first lesion analysis result of the object using a first model, wherein the first model is a 3D UNet; Determine each connected domain of the lesion based on the first lesion analysis result of the object, and obtain a group of internal representative points of each connected domain; Based on a group of internal representative points of each connected domain, acquiring another group of image blocks in the 3D medical image; Determine a second lesion analysis result of the object using a second model based on the other set of image blocks; Compare each connected domain corresponding to the first lesion analysis result with the second lesion analysis result, and retain the first lesion analysis result of the area where the overlap between each connected domain and the second lesion analysis result is greater than a predetermined threshold as the lesion analysis result of the object.

2. The method according to claim 1, characterized in that The extending direction includes any one of a direction along a coordinate axis, a principal component direction obtained by performing principal component analysis on a segmentation result of the object, and a preset direction.

3. The method according to claim 1, characterized in that The object is at least one of blood vessels, digestive tract, mammary duct, respiratory tract or lesions therein.

4. The method according to claim 1, characterized in that Compare each connected domain corresponding to the first lesion analysis result with the second lesion analysis result, and retain the first lesion analysis result of the area where the overlap between each connected domain and the second lesion analysis result is greater than a predetermined threshold. The lesion analysis results as the object specifically include, for each connected domain of the first lesion analysis result: Get the connected domain and its bounding box; Based on the bounding box of the connected domain, cropping the second lesion analysis result to obtain a corresponding image block; The overlapping area between the connected domain and the corresponding image block obtained by cropping is calculated. If the overlapping area is less than or equal to a predetermined threshold, the connected domain of the first lesion analysis result is cleared; if the overlapping area is greater than the predetermined threshold, the connected domain of the first lesion analysis result is retained.

5. The method according to any one of claims 1 to 4, characterized in that: The internal representative point is the center point.

6. The method according to any one of claims 1 to 4, characterized in that: Obtaining the internal representative points of the objects segmented in each image slice in the group of image slices specifically includes: marking a connected domain for each image slice and obtaining the internal representative points of the connected domain; and performing sparse sampling on the internal representative points of the connected domain of each image slice to obtain a sparse set of internal representative reference points.

7. The method according to claim 6, characterized in that The internal representative points of the connected domain of each image slice are sparsely sampled to obtain a set of sparse internal representative reference points, which specifically include: Sparsely sample a group of internal representative points, and expand each internal representative point after the sparse sampling to obtain an expanded mask of each internal representative point; The extended mask of each internal representative point is used to mark the connected domain, and another set of internal representative points is sampled again in the marked connected domain as a set of internal representative reference points after sparseness.

8. The method according to claim 7, characterized in that Based on a set of internal representative points of the object in the set of image slices, acquiring a set of image blocks in the 3D medical image specifically comprises: Based on a set of internal representative reference points, the 3D medical image is cropped according to a preset size to obtain a set of image blocks.

9. The method according to claim 1, characterized in that: Based on the set of image blocks, using the first model to determine the first lesion analysis result of the object specifically includes: based on the set of image blocks, using the first model to perform analysis image block by image block, and then obtaining the first lesion analysis result through false positive suppression processing.

10. The method according to claim 1, characterized in that Based on the other group of image blocks, using the second model to determine the second lesion analysis result of the object specifically includes: based on the other group of image blocks, using the second model to perform analysis image block by image block, and then obtaining the second lesion analysis result through false positive suppression processing.

11. The method according to any one of claims 1, 9 and 10, characterized in that The first model is trained using image blocks containing lesion annotation information as training samples, and the second model is trained using image blocks containing lesion annotation information together with false positive image blocks analyzed by the first model as training samples.

12. The method according to any one of claims 1 to 4, characterized in that: After the 3D medical image containing the object is acquired, the method further includes: adjusting the window and normalizing the 3D medical image by preset window width and window position.

13. The method according to claim 12, characterized in that The steps for setting the window width and window position are as follows: Obtain the grayscale value of a known 3D medical image with a lesion annotation mask; Obtaining a histogram of grayscale values ​​of the known 3D medical image, and then obtaining a total cumulative value of a vertical axis in the histogram; The horizontal axis of the histogram is the grayscale value, and the vertical axis is the grayscale cumulative value; The vertical axis of the histogram is accumulated from the left and right sides of the histogram respectively. If the accumulated value exceeds the preset threshold, the corresponding left and right horizontal axis coordinates are taken as the statistical minimum and maximum grayscale values ​​min_v and max_v respectively; The window width and window position are calculated by the minimum and maximum grayscale values ​​min_v and max_v.

14. The method according to claim 13, characterized in that The formula for calculating the window width and window position by the minimum and maximum grayscale values ​​min_v and max_v is as follows: Window width ww(window width)=max_v-min_v, Window level wl(window level)=(max_v-min_v) / 2.

15. The method according to any one of claims 1 to 4, characterized in that: The 3D medical image is a CTA image including blood vessels, a CTA image including ribs, or a CTA image including lungs.

16. A device for performing object analysis on a medical image, characterized in that: include: an interface configured to acquire a 3D medical image containing an object; as well as A processor configured to: execute the method for object analysis on a medical image according to any one of claims 1-15.

17. A non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor implement the method for object analysis of a medical image according to any one of claims 1-15.

Images