A computer-based method for classifying organ masses as cysts

Abstract

The present invention relates to a computer-based method for characterizing a mass of an organ, such as a cyst, and providing a category indicating the presence of serum and solid in the cyst and a subcategory indicating the presence of one or more serous or solid cavities in the cyst. The present invention finds advantageous application in diagnosing ovarian cysts.

Description

Technical Field

[0001] The present invention relates to the field of computer-based methodologies for processing ultrasound images of cysts and providing diagnostic support.

[0002] In particular, the present invention relates to computer-based methods for characterizing and classifying masses present in human organs. Background Art

[0003] Ultrasound imaging methods are known to provide data to support diagnostic preparation. Automatic processing of the data is needed to detect harmful asymptomatic pathologies such as ovarian tumors at an early stage, which are often diagnosed too late for effective minimally invasive treatment. Summary of the invention

[0004] The present invention solves the above-mentioned deficiencies at least in part by a computer-based method for finding and characterizing an anatomical mass (e.g., a cyst) of an organ and providing a class indicating the presence of serum in the cyst and a subclass indicating the presence of one or more serous cavities in the cyst, the method comprising the steps of:

[0005] (a) acquiring an ultrasound image of at least one anatomical entity under examination including an anatomical mass;

[0006] (b) processing the image using a first neural network-based algorithm to generate a representation such as a cyst and / or a class;

[0007] (c) Using a second algorithm of the computational graph, to:

[0008] i. identifying internal regions of the image of the anatomical mass, in particular fluid regions such as low echogenicity, or boundary regions such as high echogenicity, based on at least two different thresholds of a mathematical function applied to the echogenicity of the image;

[0009] ii. when verifying the proximity condition between two regions, classifying the internal region as a slurry region or a solid region based on at least one quantitative morphological parameter of the internal region and the aggregation model, generating an association between at least one fluid internal region and a boundary internal region; in particular, referring to the slurry classification, the morphological factor of the internal region and / or the roughness of the region boundary and / or the position of the center of mass (e.g., the center of gravity), identifying an aggregate with a boundary region whose semi-axis size is similar to that of the semi-axis of the internal fluid region; referring to the solid classification, for example, the aggregation model provides an association between the contours of the fluid region and the respective axes with significantly different dimensions, and inferring that the intermediate region between the boundary region and the fluid region is solid;

[0010] iii. Calculating the number of fluid regions (eg, slurry cavities) and / or the number of solid regions;

[0011] (d) providing a classification including categories and subcategories based on a combination of the outputs from the first and second algorithms.

[0012] The main steps of ultrasound image processing according to the present invention are to identify whether there is a cyst in the image; and to identify the morphological or geometric features of the cyst. For example, because the ultrasound image shows a healthy ovary or an ultrasound image with healthy and non-pathological structures, a cyst may not exist.

[0013] The inventors have determined that neural networks are particularly effective in identifying cysts, regardless of the type and form of cysts, whether in a completely healthy configuration or in functional and pathological morphology. For example, according to a specific non-limiting embodiment, the neural network identifies whether a cyst is present, whether it is serous or solid, and the number of cavities equal to or greater than 1 is identified by an imaging algorithm. In fact, processing the geometric parameters of the ultrasound image by known computational graphics algorithms, in particular the delimitation lines based on the contours of the serous areas, can easily provide more detailed parameters. Any combination of these tools in any order provides classification and sub-classification that helps the physician to evaluate any further in-depth examination or diagnosis of normal conditions.

[0014] Furthermore, the method is also suitable for identification by means of a computational graphics algorithm of geometrical features, such as fluid areas, for example serum, blood, etc., or solid border areas, centroids, short and long axes of serous areas; the subsequent calculation of quantitative parameters, such as the extension of the serous or border areas, i.e. the number of pixels, the position of the centroid, the extension and tilt relative to the image reference system and the irregularity index of the edge area (e.g. roughness in ultrasound images) and the subsequent association of the data with the result of the identification process by means of a neural network, which allows to associate the quantitative data with the actual presence of a cyst or an indication that the image does not include a cyst. In particular, the use of a reference convolutional neural network can also generate classifications and subclassifications, but without quantifying the number of locules and without obtaining relevant redundant data by means of a computational graphics algorithm, i.e. solid or serous and unilocular or multilocular based on the percentage of the serous area relative to the total area of ​​the cyst are compared with the results obtained by means of a computational graphics algorithm. Preferably, the classifications and subclassifications are presented as a result of the method of the invention when the results of the processing by means of the neural network are consistent with the results of the computational graphics algorithm. According to a preferred embodiment, the obtaining step includes the step of obtaining a plurality of frame segments of an ultrasound image of an examined anatomical entity including an anatomical mass; and includes the step of providing an overall category or subcategory of the segment such that multilocular is superior to unilocular and / or solid is superior to serous when a first subcategory of a first frame of the segment and a second subcategory of a second frame of the segment are different.

[0015] The method of the present invention can be applied to a single frame and must then provide an overall classification of the anatomical mass of the segment. Since it is possible that one frame shows two cavities and another shows a single cavity or that a serous classification is provided in one frame and a solid classification in another, a universal criterion must be provided so that the method provides a single classification of the anatomical mass in the presence of frames with different classifications.

[0016] The invention will be described below by means of some preferred embodiments given as non-limiting examples with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Further advantageous features will become more apparent from the following description of preferred but non-exclusive embodiments provided purely as non-limiting examples. The drawings illustrate different aspects and examples of the invention and, where appropriate, structures, components, materials and / or similar elements in the different figures are indicated by similar reference numerals.

[0018] Figure 1 It is a series of ultrasound images for different types of ovarian cysts;

[0019] Figure 2 It is an ultrasound image processed to calculate the centroid of the internal area of ​​the image;

[0020] Figure 3 is a block diagram of a method according to a non-limiting embodiment of the present invention;

[0021] Figure 4 is a simplified scheme of the neural network used in the method of the present invention;

[0022] Figure 5 is a simplified block diagram of the computer graphics processing performed by the method of the present invention;

[0023] Figure 6 A is the ultrasound image, Figure 6 B is correct Figure 6 The result of applying the entropy filter to the image of A;

[0024] Figure 7 A and 7B are images binarized by first and second slightly different thresholds, respectively. Figure 7 C is through Figure 6 The binarized image of the third threshold of B;

[0025] Figure 8 Shows Figure 7 A, two subsequent processing steps of the image of 7B;

[0026] Fig. 9 Two subsequent processing steps are shown for Image 7C;

[0027] Fig.10 yes Figure 6 A magnified view of A;

[0028] Fig.11 Is applied to Figure 6 A's Figure 5 An example of the processed output. DETAILED DESCRIPTION

[0029] While the invention is susceptible to various modifications and alternative constructions, certain preferred embodiments are shown in the drawings and will be described in detail below.

[0030] refer to Figure 1 , shows an image related to a non-restrictive classification, which is inspired by the IOTA classification and is used to classify ovarian cysts and support doctors' diagnosis. It should be noted that this method is also applicable to the identification and classification of kidney and pancreatic cysts.

[0031] In more detail, according to a non-limiting example of classification, a unilocular mass has a single area filled with fluid of various properties, which determines its echogenicity, i.e. the intensity of the echoes reflected by the material. A non-echoic mass will appear completely black. A hypoechoic mass will have a fuzzy and little diffuse gray within the mass itself. An isoechoic mass will present a gray content comparable to the tissue outside the mass. And a hyperechoic mass will be a particularly bright mass on ultrasound.

[0032] If the fluid is serum, the echogenicity level will be low or absent (see Figure 1 If there is blood spillage from the cyst or a mucinous mass is present, the echogenicity level will be hypoechoic or at best isoechoic (see Figure 1 The presence of a solid mass within a cyst can make it appear iso-echoic or hyperechoic. Extravasation of serum and blood is not associated with a significant risk level, whereas the presence of a solid mass is one of the main signs of potential malignancy, even if it is not directly suggestive.

[0033] A solid mass is one that protrudes from the cyst wall and grows to fill the cyst cavity. If it is greater than 3 mm in height, it is called a papilla. Figure 1 In the present study, the solid unilocular mass had an irregular structure with numerous papillae distributed along the entire cystic border.

[0034] Multilocular cysts can have all the above features, but the cystic spaces are not unique but are separated by septa in multiple chambers. Thus, septa are parts of tissue that develop through the contents of the cyst and divide it into multiple spaces.

[0035] Intersections between septa or between septa and cyst wall are preferential sites for solid mass formation and must be adequately monitored.

[0036] A mass is defined as solid when the solid component occupies more than 80% of the cystic space of the mass itself. An example of how a computational graph algorithm can associate binary segments of an image with a cyst to define the total area of ​​the cyst and calculate the serous percentage is described below. In addition, the neural network can also be enabled to classify images associated with solid or serous unilocular cysts by appropriate training of the neural network. Basically, through classification, the risk level associated with the mass increases, from a level of less than 1% for a unilocular serous mass to a high risk level for a solid mass (which can therefore be defined as a clear carcinoma).

[0037] Finally, for frames of segments that cannot be associated with any of the above categories, a "Miscellaneous" category is preferably associated, called background for example. This therefore includes not only ultrasounds about other organ tissues and completely "grey", but also areas that recall cystic shapes and echogenicity, such as intestinal loops, blood vessels, etc., or transition areas between cystic masses and adjacent tissues, where the quality of the cystic image is therefore not high enough to allow accurate training of the algorithm and may even affect it. In this regard, the selection Figure 1 The background image shown is representative of the category.

[0038] The two-dimensional ultrasound image, whether selected by the operator performing the ultrasound or stored with the aid of a special ultrasound function or a frame of a stored ultrasound segment, is initially processed to allow a better operation of the method according to the invention. In particular, the image is cropped around a region of interest (ROI - automatically identified, for example by template matching or by cropping / cropping based on a frame of preset size and position without manual intervention) and segmented by a global threshold. These two steps respectively make it possible to remove unnecessary parts from the image (in particular metadata as a watermark imprint on the image) and to simplify the application of computational graphics algorithms, which will be explained in more detail below.

[0039] Furthermore, the images are processed to reduce speckle noise, i.e. despeckle. Despeckle is necessary to prevent brightness variations associated with the acoustic interference phenomenon from covering or blending significant aspects of the image. The filter of choice is median filtering. Other alternatives are, but are not limited to, algorithms such as anisotropic diffusion or wavelet denoising.

[0040] like Figure 2 As shown in the first example in , the goal of the subsequent stage of edge detection is to extract the contours of the region of interest within the image in order to derive relevant geometric features (cyst region, contour).

[0041] In general, it is assumed that different regions of an ultrasound image are characterized by different entropy levels: low levels correspond to uniform regions, such as the contents of a serous cyst, and high levels correspond to abrupt changes, typically corresponding to boundaries. These algorithms will be described in more detail later.

[0042] According to the present invention, the main tumor features of the ovary (such as small cavity and papilla) are automatically classified by a feedforward neural network. The feedforward network is an acyclic graph composed of a series of layers. This architecture makes the network particularly suitable for performing the role of a classifier.

[0043] In a classification problem, given a set of objects, such as several ultrasound images of cysts, the network must be able to classify each of them into its respective class, for example to determine whether the cyst in a given ultrasound image belongs to the class of unilocular cysts or the class of multilocular cysts.

[0044] To accomplish this task, the network generates increasingly abstract and complex representations of the objects (to be classified) along the layers, emphasizing features that make the object a member of a given category rather than other categories ( Figure 4 ). However, the special feature of neural networks that distinguishes them from other types of classifiers is their ability to abstract these features from a set of examples, rather than having to specify them in advance.

[0045] In this step, called training, the network assigns each example a probability that it belongs to a specific class and not to another class. If the classification is successful, the class associated with the highest probability will coincide with the actual class to which the object belongs.

[0046] However, this situation is only possible at the end of the learning step, during which the synaptic connections of the network are modified to minimize a cost function that contains information about the number of misclassified examples. In the specific case of cyst classification, one of the most commonly used in machine learning was chosen as the cost function, namely the cross entropy function:

[0047]

[0048] where M represents the number of examples, but the effective probability p that example m belongs to category c and the probability q that example m belongs to category c are assigned by the network according to the current configuration of its synaptic connections W.

[0049] The cost function is optimized by an optimization algorithm, such as gradient descent (GD) and its variants. These algorithms provide a prescription or learning rule by which the network connections must be modified so that it can correctly classify the entire set of examples. In particular, in the case of GD and its variants, the synaptic connections are updated in the direction of the gradient:

[0050]

[0051] where η determines the update step, and is therefore called the learning rate. In particular, the use of a variant known as Adam as an optimization algorithm was tested, in which the learning rate is not kept constant throughout the training process and is the same for each synaptic connection, but is adjusted based on the first and second momentum of the gradient.

[0052] Generally speaking, to improve the performance of the network, there are functions that prevent overfitting of training samples, such as dropouts.

[0053] Convolutional feed-forward networks can also be used. In the initial stages of the project, it was decided to successfully implement a simpler feed-forward network, namely a perceptron multilayer. In a preferred but non-limiting case, in the case where an ultrasound scan is classified as containing a serous cyst, it is possible to localize the serous cyst with obvious results and automatically extract some geometric features. In particular, the two steps of localization and geometric features are preferably processed by computational graphics techniques. Alternatively, localization can be performed by a neural network, in particular by a convolutional neural network. Preferably, with regard to localization, in order to locate the position of a serous cyst, several complexities must be taken into account, including:

[0054] Multilocular cysts: Cysts often present with multiple cavities, and the presence of multiple serous areas should be considered;

[0055] · Serum and mucin: Serum must also be identified in the presence of mucus;

[0056] Visually similar ultrasound areas that do not represent serous cysts. It is necessary to distinguish between general dark areas and dark areas that actually represent cystic seromas.

[0057] Taking all these factors into account, an approach to the cyst from both the inside and the outside was applied ( Figure 5 ), that is, it aims to identify the serous (inner) component and the border tissue. This is useful because the serous component alone may not be completely indicative of pathology and, in general, the border tissue represents an important factor in assessing the status of the cyst.

[0058] In a pre-processing step 100, ultrasound cropping is performed to remove unnecessary parts of the ultrasound, such as rasterized metadata with the actual ultrasound image (e.g., patient registration, echo scale in millimeters). The image is then filtered by an entropy filter ( Figure 6 ) calculates the entropy of the image to make those areas of uniform grayscale more distinct. This indicatively makes the serous areas and surrounding tissue more distinct, while in fact visually homogeneous. Optionally, a filter can be applied to the Hessian matrix.

[0059] Subsequently, the crop of the entropy image is binarized by a specific threshold and then segmented 110 to extract three different segments. Two of them ( Figure 7 A, 7B) represent approximations of serous areas with different but low thresholds to identify areas with high contrast and low entropy, while the remainder represents boundary tissue by applying a high threshold ( Figure 7 C), i.e. identifying regions with high entropy. The two approximations of the serous region inside the cyst thus have slightly different visual features and can be used to validate the results. In particular, the two internal segments differ in that the respective thresholds used for binarization have similar but not identical values.

[0060] Specifically, two segments are taken from the interior and the centroid of the connected dark component is calculated, i.e., the closed region representing the slurry region of the image ( Figure 8 A) and check if they are contained in connected components of other segments with a low threshold. When this and other conditions occur, such as the connected component area is above a predetermined and adjustable threshold, connected components that are far from the border of the echographic image, such as outside a predetermined adjustable limit distance, and a predetermined and adjustable shape factor based on the relationship between the major and minor axes of the connected component approximating an ellipse, the component is considered to be connected because it actually represents a serous area ( Figure 8 B).

[0061] When the perimeter roughness of the identified region is also considered, then a relatively high value (e.g., exceeding an adjustable threshold) is associated with the presence of papillae. It will be appreciated that the roughness and related quantities (e.g., local peaks, local valleys, reference lines, peaks and valleys) can be calculated at the perimeter of the connected region (preferably of a binarized image), just as it occurs on the surface of an object.

[0062] In this way, multiple serous areas can be localized as desired, the maximum number of which is defined by specific modifiable parameters; thus, the method of the invention is able to identify multilocular cysts and localize their individual lumens.

[0063] The outer segment 130, i.e. the segment identifying the cyst contour, has to be compared with the serous area actually identified and selected in step 120. Preferably, only contours that are close enough to the serum are selected. To calculate the distance between the boundary tissue and the serous area, the centroid of the serous area ( Figure 8 B) and the boundaries of each segment ( Fig. 9 A): If this distance is less than a predetermined adjustable threshold, the checked boundary is associated with the region and is therefore selected ( Fig. 9 B) The default threshold can be fixed or adaptive, i.e. expressed as a function of the size of the slurry zone, e.g. one or both of the semi-axes. In this way, each frame is associated with its boundaries.

[0064] Once both the slurry regions and relative boundaries are extracted, this information is merged into a single image 140. This simplifies visual identification of regions of interest and enables easier reading of the ultrasound scan.

[0065] In order to further simplify the reading of ultrasound images, the method of the invention comprises a step of calculating some geometrical properties of the cyst. In particular, all metrics are calculated, which are themselves calculated in a known manner and are not further described. For the sake of clarity, the most useful and important are:

[0066] - The minor and major axes of the ellipse are most similar to the identified area: they represent good estimates of the measurements (in pixels) of the cyst's width and height.

[0067] -Direction: This is the angle between the horizontal reference axis and the main axis of the image.

[0068] - Centroid: The center of gravity of the identified area.

[0069] Cyst Area: Measured in pixels, it approximates the effective extent of the cyst.

[0070] Fig.10 and 11 An acquired ultrasound image and a result of the graphic processing according to the method of the present invention are respectively illustrated.

[0071] The preceding paragraphs relate to the application of the method according to the invention to a single ultrasound image.

[0072] According to the present invention, frames of an ultrasound segment may also be processed as follows.

[0073] Each frame of the segment is labeled so that a numerical label is associated with each individual frame that makes up the segment. The segment is then associated with a list that reports the numerical label of each frame that makes up the video, and each image is processed as described in the previous paragraph so that the list provides the relevant category and subcategory for each frame identified by the label.

[0074] This list is then processed to provide an overall classification of the cysts represented by the frames, while some frames may not include cysts that are present in other frames.

[0075] For this reason, for classifying the cyst referred to by the segment, the multilocular category was preferred over the single-locular category, i.e., if at least one frame of the segment was multilocular, the cyst was classified as multilocular. Similarly, if at least one frame was classified as solid, this category was preferred for describing the cyst, even if the other frames were classified as serous.

[0076] In addition, the counting of frames can be performed by storing the centroid position of each small cavity in each frame. If the centroid of a small cavity in a frame has a distance greater than a predetermined threshold compared to other centroids of other small cavities, the small cavity is regarded as an additional small cavity instead of a small cavity that already exists in other frames, and the total number of small cavities is updated accordingly.

Claims

1. A computer-based method for finding and characterizing an anatomical mass of an organ and for providing a class indicating the presence of serum in the anatomical mass and a subclass indicating the presence of one or more serous cavities and / or the number of serous cavities in the anatomical mass, the method comprising the following steps: (a) acquiring at least one ultrasound image of the anatomical entity under examination including the anatomical mass; (b) interpreting the image using a first neural network-based algorithm to output a representation of an anatomical mass and / or a class of an anatomical mass; (c) Using a second algorithm of the computational graph, to: i. identifying an interior region of the image, the interior region comprising a fluid region or a boundary region, based on at least first and second thresholds applied to a mathematical function of the echogenicity of the image; ii. when verifying the proximity condition between the regions, classifying the interior region as a serous region or a solid region based on the quantitative morphological parameters of the interior region and the aggregation model, and generating an association between at least the fluid interior region and the boundary interior region; iii. Calculating the number of serous areas such as serous cavities and / or the number of solid areas; (d) providing a classification of the image comprising the categories and the subcategories based on a combination of outputs from the first algorithm and the second algorithm.

2. The method according to claim 1, wherein: The anatomical mass is a cyst.

3. The method according to claim 1 or 2, wherein: The identifying of the interior region of the image based on at least first and second thresholds of a mathematical function applied to the echogenicity of the image further comprises associating low echo level regions to high echo level regions to the same anatomical entity based on distances between associated centroids.

4. The method according to claim 3, wherein: The method of acquiring at least one ultrasound image of the examined anatomical entity including the anatomical mass further includes the step of acquiring a plurality of frame segments of the ultrasound image of the anatomical mass; and the method further includes the step of providing an overall category or subcategory of the segment to favor multilocular over unilocular and / or solid over serous when a second subcategory of a second frame image of the segment and a first subcategory of a first frame have different outputs from the step of providing image classification.

5. The method according to claim 4, wherein: The step of using the second algorithm includes the step of outputting the number of cavities as part of the sub-category.

6. The method according to claim 1 or 2, wherein: The step of using said second algorithm includes the step of comparing the total area of ​​the grout region and the total area of ​​the solid region, and the step of providing a classification includes the step of indicating a sub-category based on the comparing step.

7. The method according to claim 3, wherein: The step of using said second algorithm enables said serous region and a boundary region around said serous region to be identified as respective geometrical features, said associated centroids to be calculated as respective further geometrical features and the positions of the centroids to be calculated as respective quantitative parameters.

8. The method according to claim 7, wherein: The step of using the second algorithm results in associating the centroid to a designated slurry region based on a predetermined maximum distance between the centroid of the slurry region and the centroid of the boundary region.

9. The method according to claim 1 or 2, wherein: The step of using said second algorithm provides for applying an entropy filter or a Hazen matrix filter to said image to increase the distinction between low echogenic and high echogenic fluid regions surrounding boundary solid regions.

10. The method according to claim 1 or 2, wherein: The neural network is a feedforward convolutional network or a feedforward multi-layer network.

11. The method according to claim 1 or 2, wherein: The first algorithm and the second algorithm both output a grout or solid classification and a single-house or multi-house classification, and the steps provided are based on the outputs of the first algorithm and the second algorithm being the same.

12. The method according to claim 1 or 2, wherein: The second algorithm is selected from an edge extraction algorithm, a region extraction algorithm and an image characteristic description algorithm.

Images