Cardiothoracic ratio calculation method, method and device for determining the probability of cardiac atrioventricular enlargement

Abstract

The present invention relates to the field of medical image processing technology, and specifically to a cardiothoracic ratio calculation method, a method and device for determining the probability of cardiac atrioventricular enlargement, aiming to avoid missed diagnosis. The cardiothoracic ratio calculation method proposed in the present invention comprises: inputting an anteroposterior chest radiograph image into a trained cardiopulmonary segmentation model and a spinal segmentation model respectively to obtain cardiopulmonary segmentation results and spinal segmentation results; and removing noise from the cardiopulmonary segmentation results; obtaining the spinal centerline according to the spinal segmentation results; obtaining the maximum length of the line connecting the outermost segmentation points of the left and right lung fields on the same horizontal plane as the maximum transverse diameter of the thorax based on the cardiopulmonary segmentation results; calculating the farthest distance between the left and right lateral edges of the heart and the spinal centerline on the same horizontal plane based on the cardiopulmonary segmentation results, and calculating the sum of the farthest distance of the left edge and the farthest distance of the right edge as the maximum transverse diameter of the heart; and calculating the cardiothoracic ratio based on the maximum transverse diameter of the heart and the maximum transverse diameter of the thorax. The present invention reduces dependence on doctor's experience and improves accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for calculating a cardiothoracic ratio, and a method and device for determining the probability of occurrence of cardiac atrioventricular enlargement. Background Art

[0002] Chest radiography is one of the most common and cost-effective medical imaging procedures, with high clinical demand. Although not as accurate as tomography, it still holds a unique and important role due to its simplicity, low cost, and repeatability.

[0003] Chest X-ray examination can be used to calculate the cardiothoracic ratio, which is the ratio of the maximum transverse diameter of the heart to the maximum transverse diameter of the thorax. As a simple and easy indicator to describe the overall size of the heart, the cardiothoracic ratio is an important basis for clinical judgment of heart size.

[0004] Cardiac enlargement is a physiological condition of the patient's heart. Prompt diagnosis and treatment are crucial for reducing complications and improving patient outcomes. Various heart diseases, including coronary artery disease, pulmonary hypertension, mitral stenosis, and atrial septal defects, can lead to heart enlargement. For example, a general enlarged heart is often seen in combined valvular disease and dilated cardiomyopathy, and its chest X-ray features include enlarged atria and ventricles. A mitral heart is often seen in mitral valve disease, pulmonary stenosis, and cor pulmonale, and its chest X-ray features include a bulging pulmonary artery and right ventricular enlargement. An aortic heart is often seen in aortic valve disease, hypertensive heart disease, and coronary artery disease, and its chest X-ray features include a relatively concave pulmonary artery, an enlarged left ventricle, and a widened aorta. Therefore, the diagnosis of cardiac abnormalities is closely related to atrioventricular enlargement, and therefore, diagnosing changes in atrioventricular size on chest X-rays has high clinical value.

[0005] The current method for obtaining the cardiothoracic ratio is relatively crude and does not accurately capture the heart's maximum transverse diameter, resulting in a calculated cardiothoracic ratio that is smaller than the true value. This leads to low diagnostic sensitivity for patients with intermediate cardiothoracic ratios. Furthermore, considering that doctors need both prior knowledge and specific chest radiographs to make an accurate diagnosis, and that the professional and technical capabilities of different doctors vary, different doctors may reach different conclusions on the same patient. Summary of the Invention

[0006] In order to solve the above problems in the prior art, the present invention proposes a cardiothoracic ratio calculation method, a method and device for determining the probability of atrioventricular enlargement, which reduces dependence on doctor's experience and improves accuracy.

[0007] A first aspect of the present invention provides a method for calculating a heart-to-chest ratio, the method comprising:

[0008] Input the frontal chest X-ray image into the trained cardiopulmonary segmentation model and output the cardiopulmonary segmentation result;

[0009] removing noise from the heart and lung segmentation result;

[0010] Inputting the anteroposterior chest X-ray image into a trained spine segmentation model and outputting a spine segmentation result;

[0011] Acquire a spine centerline according to the spine segmentation result;

[0012] Based on the two closed areas corresponding to the left and right lung fields in the heart and lung segmentation results, the maximum length of the line connecting the outermost segmentation points of the left and right lung fields on the same horizontal plane is obtained as the maximum transverse diameter of the thorax;

[0013] Based on the closed area corresponding to the heart in the heart-lung segmentation result, respectively calculating the maximum distance between the left and right edges of the heart and the centerline of the spine on the same horizontal plane, and obtaining the maximum distance of the left edge and the maximum distance of the right edge;

[0014] Calculate the sum of the farthest distance of the left edge and the farthest distance of the right edge as the maximum transverse diameter of the heart;

[0015] The cardiothoracic ratio is calculated according to the maximum transverse diameter of the heart and the maximum transverse diameter of the thorax.

[0016] Preferably, the step of "obtaining the spine centerline according to the spine segmentation result" includes:

[0017] Based on the spine segmentation results, the left and right sides of the spine are equally divided into M segments and M+1 key points are extracted;

[0018] Connect the key points at corresponding positions on the left and right sides to form M+1 line segments, and calculate the perpendicular bisector of each line segment respectively;

[0019] The intersection points of the M+1 line segments and the corresponding perpendicular midlines are fitted using the least squares method to obtain the spinal centerline;

[0020] Wherein, M is a preset value.

[0021] Preferably, the cardiopulmonary segmentation model adopts a VNet model;

[0022] The loss function of the cardiopulmonary segmentation model is: ;

[0023] in, ; ;

[0024] N is the total number of pixels; y i and Represents the label value and predicted value of the i-th pixel respectively; p(y i) indicates that the output belongs to y i The probability of the label.

[0025] A second aspect of the present invention provides a method for determining the probability of occurrence of cardiac atrioventricular enlargement, the method comprising:

[0026] According to the cardiothoracic ratio calculation method described above, the cardiothoracic ratio is calculated on the anteroposterior chest radiograph image to be evaluated, and a cardiopulmonary segmentation result is obtained;

[0027] Refining the edge points of the heart according to the heart-lung segmentation result, and then calculating the dense edge point matrix of the heart;

[0028] Inputting the dense edge point matrix and the anteroposterior chest X-ray image to be evaluated into a convolutional neural network to extract visual features of the anteroposterior chest X-ray image to be evaluated;

[0029] Mining the relationship data of simultaneous enlargement of each chamber of the heart based on the historical dataset of frontal chest radiograph reports, and then generating a feature matrix based on prior knowledge;

[0030] Performing a dot product of the feature matrix based on prior knowledge and the visual features of the anteroposterior chest radiograph image to be evaluated to calculate a multi-category probability of enlargement of each atrium and chamber of the heart;

[0031] The multi-category probabilities include: a probability of left atrium enlargement, a probability of left ventricle enlargement, a probability of right atrium enlargement, and a probability of right ventricle enlargement.

[0032] Preferably, the step of “finely extracting the edge points of the heart according to the heart-lung segmentation result, and then calculating a dense edge point matrix of the heart” includes:

[0033] Extracting the outermost pixels of the left and right edges of the heart respectively according to the cardiopulmonary segmentation result, and constructing a sparse pixel probability matrix based on the probability values ​​corresponding to the outermost pixels in the segmentation result of the cardiopulmonary segmentation model;

[0034] Applying a two-dimensional Gaussian function to the sparse pixel probability matrix to obtain a rendered heart edge point probability matrix;

[0035] The heart edge point probability matrix is ​​clustered by combining the distance and the probability value to form a dense edge point matrix.

[0036] Preferably, the step of “applying a two-dimensional Gaussian function to the sparse pixel probability matrix to obtain a rendered heart edge point probability matrix” includes:

[0037] The parameters μ and σ in the two-dimensional Gaussian function are set to preset values, and a 3*3 Gaussian matrix G is generated based on the Gaussian kernel position matrix;

[0038] Wherein, the Gaussian kernel position matrix is:

[0039] ;

[0040] Divide each element in the Gaussian matrix G by the sum of all elements in the matrix to obtain a normalized matrix G';

[0041] Divide each element in the matrix G' by the element at the center of the matrix to obtain a transformed matrix G'';

[0042] Multiply the center value of each sparse pixel in the sparse pixel probability matrix by the matrix G'' to obtain a Gaussian matrix distribution matrix centered on the sparse pixel, and superimpose the repeated pixels between the multiple Gaussian matrix distribution matrices to obtain the rendered heart edge point probability matrix.

[0043] Preferably, the step of “performing a clustering operation on the heart edge point probability matrix by combining the distance and the probability value to form a dense edge point matrix” includes:

[0044] The sparse pixel center point at the highest point on the left side of the rendered heart edge point probability matrix is ​​used as the starting core point, and the point with the largest linear sum of distance and probability is found within the neighborhood of the current core point as the next core point. The left tracing method of the heart segmentation map is set from top to bottom, and the right tracing method is set from bottom to top. Finally, clustering is performed to form the dense edge point matrix.

[0045] Preferably, the step of “inputting the dense edge point matrix and the orthotopic chest X-ray image to be evaluated into a convolutional neural network to extract visual features of the orthotopic chest X-ray image to be evaluated” includes:

[0046] splicing the to-be-evaluated anteroposterior chest radiograph image and the dense edge point matrix into two-channel matrix data;

[0047] The matrix data is input into a convolutional neural network to extract visual features of the frontal chest X-ray image to be evaluated.

[0048] Preferably, the step of "mining the relationship data of simultaneous enlargement of cardiac chambers based on the historical data set of anteroposterior chest radiograph reports, thereby generating a feature matrix based on prior knowledge" includes:

[0049] Mining the relationship data of simultaneous enlargement of each atrium and chamber of the heart based on the historical data set of anteroposterior chest radiograph reports and the label categories of cardiac atrium and chamber enlargement, thereby constructing a relationship data matrix; the label categories include: left atrial enlargement, left ventricular enlargement, right atrial enlargement, and right ventricular enlargement;

[0050] Calculating the probability of each tag category appearing when other tag categories appear based on the relationship data matrix, thereby obtaining a conditional probability matrix;

[0051] Binarizing the conditional probability matrix and introducing weights to obtain a correlation matrix;

[0052] The feature matrix based on prior knowledge is calculated according to the correlation matrix and the category word embedding vector.

[0053] According to a third aspect of the present invention, a computer-readable storage device is provided, storing a computer program that can be loaded by a processor and execute the method described above.

[0054] According to a fourth aspect of the present invention, a processing device is provided, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method described above.

[0055] The present invention has the following beneficial effects:

[0056] The cardiothoracic ratio calculation method proposed in the present invention calculates the maximum distance between the left and right edges of the heart and the centerline of the spine on the same horizontal plane, obtaining the maximum distances to the left and right edges. The sum of these distances is then calculated as the maximum transverse diameter of the heart. Compared to existing methods that determine the maximum transverse diameter of the heart by measuring the distance between the left and right edges, or by calculating the sum of the maximum distances between the left and right edges and the midline of the chest, the cardiothoracic ratio obtained by the present method is more accurate and less likely to be missed.

[0057] The method for determining the probability of atrioventricular enlargement proposed in the present invention achieves a two-way fusion of clinical prior knowledge and chest X-ray image information by performing a dot product between a feature matrix based on prior knowledge and the visual features of the frontal chest X-ray image to be evaluated, thereby making full use of multi-dimensional information to improve the recognition rate of atrioventricular enlargement.

[0058] In addition, the edge points of the heart are refinedly extracted based on the cardiopulmonary segmentation results, and a distance and probability edge clustering algorithm is proposed, which can fully mine the edge information of the heart and further improve the recognition rate of cardiac atrioventricular enlargement.

[0059] In summary, the cardiothoracic ratio quantification calculation method and the method for determining the probability of atrioventricular enlargement proposed by the present invention based on chest X-ray can assist doctors in diagnosis, reduce dependence on doctors' experience, improve diagnostic accuracy, reduce doctors' duplication of work, and reduce diagnostic differences between different doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 1 is a schematic diagram of the main steps of an embodiment of a method for calculating a cardiothoracic ratio according to the present invention;

[0061] Figure 2 2 is a schematic diagram of a method for measuring distance when calculating the heart-to-chest ratio in an embodiment of the present invention;

[0062] Figure 3 Schematic diagram of the main steps of the method for processing historical data in an embodiment of the present invention;

[0063] Figure 4 Schematic diagram of the main steps of an embodiment of a method for determining the probability of occurrence of cardiac atrioventricular enlargement according to the present invention;

[0064] Figure 5 is an example of the Gaussian matrix G, the normalized matrix G', and the transformed matrix G'' in an embodiment of the present invention;

[0065] Figure 6 is a pixel position numbering diagram of the sparse pixel probability matrix E in an embodiment of the present invention;

[0066] Figure 7 This is an example of the sparse pixel probability matrix E and the rendered heart edge point probability matrix E' in the embodiment of the present invention. DETAILED DESCRIPTION

[0067] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0069] It should be noted that, in the description of the present invention, the terms "first" and "second" are merely for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore should not be understood as limiting the present invention. In addition, the term "and / or" in the present invention is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document, unless otherwise specified, generally indicates that the associated objects are in an "or" relationship.

[0070] Currently, there are numerous methods for calculating the cardiothoracic ratio, differing primarily in how the heart and lung fields are captured. These methods can be summarized into three approaches: fitting the elliptical equation of the lung field using mathematical modeling; identifying the heart and lung fields using filter functions based on traditional image algorithms; and segmenting the heart and lung fields using deep learning segmentation models. Each approach then calculates the cardiothoracic ratio based on the lung and heart fields. However, when calculating the transverse cardiac diameter, current methods do not accurately measure the sum of the maximum distances from the left and right cardiac margins to the midline. Instead, they measure only the maximum transverse cardiac diameter, resulting in a smaller cardiothoracic ratio. This can result in patients with a slightly larger cardiothoracic ratio being considered normal, and can easily lead to missed diagnosis of patients with an intermediate cardiothoracic ratio.

[0071] Figure 1 FIG. 1 is a schematic diagram of the main steps of an embodiment of the method for calculating the heart-chest ratio of the present invention. Figure 1 As shown, the calculation method of this embodiment includes steps A10-A80:

[0072] Step A10: Input the frontal chest X-ray image into the trained cardiopulmonary segmentation model and output the cardiopulmonary segmentation result.

[0073] The input of the cardiopulmonary segmentation model is the frontal chest DICOM film, and the output is a three-channel result image, which is the cardiopulmonary segmentation result. The cardiopulmonary segmentation result is a matrix with a size of 3*1024*1024, where 1024 corresponds to the length and width of the chest film. Each point in the matrix represents a pixel of the cardiopulmonary segmentation result. The value of each point in the matrix is ​​0, 1, or 2, where 0 represents the background, 1 represents the lung field, and 2 represents the heart.

[0074] Step A20: removing noise from the heart and lung segmentation results.

[0075] Due to the complexity of real-world scenarios, the heart and lung segmentation results are generally rough, with some noise or holes. A contour detection algorithm (in this example, the Suzuki contour tracking algorithm) is used to denoise the heart and lung segmentation results, obtaining multiple closed regions. All closed contours are then filled. Closed regions are extracted from the lung field segmentation results. If multiple closed regions are obtained, only the two largest regions are retained as the left and right lung fields. The rest are noise regions. Similarly, only the largest region is retained in the heart segmentation results, as the closed region corresponding to the heart.

[0076] The subsequent steps A50 and A60 are both calculated using the heart and lung segmentation results after noise removal.

[0077] Step A30: Input the AP chest X-ray image into the trained spine segmentation model and output the spine segmentation result.

[0078] The input of the spine segmentation model is the frontal view of the chest DICOM film, and the output is a result image of one channel, which is the spine segmentation result. The spine segmentation result is a one-dimensional matrix. Each point in the matrix represents a pixel of the spine segmentation result. The value of each point in the matrix is ​​0 or 1, 0 represents the background, and 1 represents the spine.

[0079] Step A40, obtaining the spine centerline based on the spine segmentation result. This step may specifically include steps A41-A43:

[0080] Step A41: Based on the spine segmentation results, the left and right sides of the spine are equally divided into M segments and M+1 key points are extracted.

[0081] Among them, M is a preset value and can be any value. In this embodiment, in order to better fit the spinal structure, the value of M is set to 16, and 17 key points are extracted on both sides of the spine, which just correspond to 12 thoracic vertebrae and 5 lumbar vertebrae.

[0082] Step A42: Connect the key points at corresponding positions on the left and right sides to form M+1 line segments, and calculate the perpendicular bisector of each line segment respectively.

[0083] In this embodiment, 17 line segments are obtained, and the perpendicular bisector of each line segment is calculated respectively. Each line segment and its corresponding perpendicular bisector have an intersection point.

[0084] Step A43: Use the least squares method to fit the intersection points of M+1 line segments and the corresponding perpendicular midlines to obtain the spine centerline.

[0085] Step A50 , based on the two closed areas corresponding to the left and right lung fields in the cardiopulmonary segmentation result, obtain the maximum length of the line connecting the outermost segmentation points of the left and right lung fields on the same horizontal plane as the maximum transverse diameter of the thorax.

[0086] Step A60, based on the closed area corresponding to the heart in the heart-lung segmentation result, respectively calculate the maximum distance between the left and right edges of the heart and the centerline of the spine on the same horizontal plane to obtain the maximum distance of the left edge and the maximum distance of the right edge.

[0087] Figure 2 Schematic diagram of the method for measuring distance when calculating the heart-chest ratio in an embodiment of the present invention. Figure 2 As shown, the middle vertical line is the center line of the spine, H1 and H2 are the farthest distances to the right edge and the farthest distance to the left edge respectively, and L is the maximum transverse diameter of the thorax.

[0088] Step A70: Calculate the sum of the farthest distance of the left edge and the farthest distance of the right edge as the maximum transverse diameter of the heart.

[0089] In step A80, the cardiothoracic ratio is calculated based on the maximum transverse diameter of the heart and the maximum transverse diameter of the thorax. Specifically, the cardiothoracic ratio is the ratio of the maximum transverse diameter of the heart (H1 + H2) to the maximum transverse diameter of the thorax (L).

[0090] The cardiothoracic ratio calculation method proposed in the present invention calculates the maximum distance between the left and right edges of the heart and the centerline of the spine on the same horizontal plane, obtaining the maximum distances to the left and right edges. The sum of these distances is then calculated as the maximum transverse diameter of the heart. Compared to existing methods that determine the maximum transverse diameter of the heart by measuring the distance between the left and right edges, or by calculating the sum of the maximum distances between the left and right edges and the midline of the chest, the cardiothoracic ratio obtained by the present method is more accurate and less likely to be missed.

[0091] In order to implement the above-mentioned cardiothoracic ratio calculation method, it is necessary to process the historical data in advance to generate a training set, and to train the cardiopulmonary segmentation model and the spine segmentation model. Figure 3 FIG. 1 is a schematic diagram of the main steps of the method for processing historical data in an embodiment of the present invention. Figure 3 As shown, in addition to constructing a training set for the cardiopulmonary segmentation model and a training set for the spine segmentation model, the embodiment of the present invention also constructs a historical data set of AP chest radiograph reports (which will be used in the subsequent embodiment of the method for determining the probability of atrioventricular enlargement). The method for processing historical data in this embodiment includes steps B10-B60:

[0092] Step B10: Collect AP chest radiographs and corresponding chest radiograph reports, and exclude chest radiographs of patients aged <18 years and those with severe image artifacts (such as typical scoliosis, right heart mirror, etc.).

[0093] Step B20: Mark three closed areas on each frontal chest radiograph, namely, the left lung, right lung, and heart area.

[0094] Specifically, the lung field area is annotated along the chest cavity boundary of the frontal chest X-ray obtained in step B10. The label value can be set as needed, as long as the lung field area can be distinguished from other areas. In this embodiment, the lung field label value is 1; the label is performed along the outer contour of the heart, from the clavicle position to the end of the heart. The label value can be set as needed, as long as the outer contour of the heart can be distinguished from other areas. In this embodiment, the heart label value is 2. The annotated boundary is as smooth as possible, and there should be no voids inside to ensure that the lung field is two complete areas on the left and right, and the heart is one complete area.

[0095] Step B30: constructing a training set for the cardiopulmonary segmentation model based on the frontal chest radiograph image with the cardiopulmonary regions marked.

[0096] Specifically, the labeled data was divided into training, validation, and test sets (in a ratio of 6:2:2). The training set was augmented through geometric and color transformations to increase the quantity and diversity of the data. Image information of the frontal chest radiograph was extracted and resized to a fixed size for easy model input. The data was also normalized (the image pixel values ​​were processed to between 0 and 255).

[0097] Data augmentation is used to enable training even when fewer labeled samples are available, thus reducing the burden of labeling. Geometric transformations include operations such as flipping, rotation, cropping, deformation, and scaling; color transformations include noise and blurring.

[0098] Step B40: Mark the left and right boundaries of the spine on each frontal chest radiograph based on the brightness and darkness changes of the pixels.

[0099] Specifically, for the frontal chest radiograph obtained in step B10, the left and right boundaries of the spine are annotated along the pixels where the spine changes in light and dark (starting from the junction of the two rib cages on the spine and ending at the bottom of the chest radiograph). The annotation label value is 1, and a complete area is annotated without any holes inside, thereby obtaining the annotated spine area.

[0100] Step B50: Based on the annotated anteroposterior chest radiograph, a training set for the spine segmentation model is constructed. The specific method can be found in the above step B30.

[0101] Step B60: Extract cardiac atrioventricular enlargement features for each chest X-ray report, return a category word embedding vector as label information, and then construct a historical dataset of frontal chest X-ray reports.

[0102] Specifically, if a report contains a description of left / right atrial / ventricular enlargement, fields such as "left atrial enlargement," "right atrial enlargement," "left ventricular enlargement," and "right ventricular enlargement" are extracted from the report. In particular, "apex bulging and upturned" is equivalent to "right ventricular enlargement," and "apex dislocation" is equivalent to "left ventricular enlargement." For each report, a category word embedding vector is returned as label information. This label information will be used to generate a feature matrix based on prior knowledge in the subsequent embodiment of the method for determining the probability of atrioventricular enlargement.

[0103] The category word embedding vector is simply defined using one-hot encoding, which represents the relationship between categories using a single encoding format. The vector information corresponds to the left atrium, left ventricle, right atrium, and right ventricle, respectively. Left atrial enlargement, left ventricular enlargement, right atrial enlargement, and right ventricular enlargement correspond to the vectors [1,0,0,0], [0,1,0,0], [0,0,1,0], and [0,0,0,1], respectively. If both left atrial enlargement and right ventricular enlargement are extracted in the report, the returned category word embedding vector is [1,0,0,1].

[0104] In an optional embodiment, the cardiopulmonary segmentation model can adopt a VNet model. The cardiopulmonary segmentation model is trained using the training set obtained in step B30, using the sum of DICE loss (DICE loss) and cross entropy loss (CELoss) as the loss function, and using IOU as the evaluation criterion. The larger the IOU value, the better the cardiopulmonary segmentation model effect.

[0105] The loss function of the cardiopulmonary segmentation model in this embodiment is shown in the following formulas (1)-(3): (1);

[0106] in, (2);

[0107] (3);

[0108] N is the total number of pixels; y i and Represents the label value and predicted value of the i-th pixel respectively; p(y i ) indicates that the output belongs to label y i probability.

[0109] In this embodiment, a hybrid loss function is adopted, that is, the sum of DICE (DICE loss) and cross entropy loss (CELoss), which takes into account both category accuracy and pixel position accuracy, making the segmentation effect more accurate.

[0110] In another optional embodiment, VNet is selected as the spine segmentation model. The model input is a DICOM chest X-ray, and the output is a single-channel image. This image is a set of vectors, each value of which is either 0 or 1, where 0 represents the background and 1 represents the spine. The spine segmentation model is trained using the sum of DICE and cross entropy as the loss function, and IoU as the evaluation criterion.

[0111] Figure 4 Schematic diagram of the main steps of the embodiment of the method for determining the probability of cardiac atrioventricular enlargement of the present invention. Figure 4 As shown, the method of this embodiment includes steps C10-C50:

[0112] Step C10 : performing a cardiothoracic ratio calculation on the anteroposterior chest radiograph image to be evaluated according to a cardiothoracic ratio calculation method, and obtaining a cardiopulmonary segmentation result.

[0113] In this embodiment, the heart-to-chest ratio may be calculated by referring to the heart-to-chest ratio calculation method described in the previous embodiment, and the heart-lung segmentation result may be obtained.

[0114] Step C20 : performing refined extraction of the heart's edge points based on the heart-lung segmentation result, and then calculating a dense edge point matrix of the heart.

[0115] In this embodiment, considering that the heart edge morphology is crucial for diagnosing atrioventricular enlargement, a refined heart edge extraction is performed based on the heart and lung segmentation results, thereby fully extracting the heart edge morphology information. This step may specifically include steps C21-C23:

[0116] In step C21, the outermost pixels of the left and right edges of the heart are extracted based on the heart and lung segmentation results, and a sparse pixel probability matrix is ​​constructed based on the probability values ​​corresponding to the outermost pixels in the heart and lung segmentation model. In this embodiment, the sparse pixel probability matrix is ​​denoted as E.

[0117] Specifically, only one pixel is extracted from each row on the left and right sides of the cardiopulmonary segmentation result matrix. The left and right pixel points where the distance between the left and right pixels is the largest are used to locate the farthest positions of the left and right edges of the heart, and the probability values ​​corresponding to the segmented pixels are retained. Finally, a sparse matrix of pixel probabilities, namely the sparse pixel probability matrix E, is obtained. In step C22, a two-dimensional Gaussian function is applied to the sparse pixel probability matrix to obtain a rendered heart edge point probability matrix. Considering that the gradient change of the outer edge of the heart on the chest X-ray is gradual, edge rendering with a 2D Gaussian function can better fit this change.

[0118] This step may specifically include steps C221-C224:

[0119] In step C221 , the parameters μ and σ in the two-dimensional Gaussian function are respectively set to preset values, and a 3*3 Gaussian matrix G is generated based on the Gaussian kernel position matrix.

[0120] Among them, the two-dimensional Gaussian function G(x,y) is shown in formula (4):

[0121] (4)

[0122] The Gaussian kernel position matrix is ​​shown in the following formula (5):

[0123] (5)

[0124] In this embodiment, μ = 0 and σ = 0.8 in the Gaussian function. In formula (4), (x, y) represents the coordinates of each kernel position in the Gaussian kernel position matrix. Each kernel position coordinate in the Gaussian kernel position matrix is ​​input into the two-dimensional Gaussian function to obtain multiple elements, all of which form the Gaussian matrix G.

[0125] Step C222 : Divide each element in the Gaussian matrix G by the sum of all elements in the matrix to obtain a normalized matrix G′.

[0126] In general, the sum of the matrix G is not 1, so G needs to be normalized. Each value is divided by the sum of the values ​​of the matrix G (that is, the sum of all the values ​​in the matrix) to obtain G'. At this time, the obtained G' is consistent with the Gaussian function (that is, the meaning and effect are consistent). It decays around the center point of G', and the weight changes in the same trend as the Gaussian function.

[0127] Since the matrix E in step C21 is a sparse pixel probability matrix, to prevent the sparse pixel probability matrix E obtained in step C21 from being too small after being directly applied to G', step C223 is used to convert G' into G'', and then step C224 applies the sparse pixel probability matrix E to G''. In step C223, each element in matrix G' is divided by the element at the center of the matrix to obtain the converted matrix G''.

[0128] Figure 5 , which are examples of the Gaussian matrix G, the normalized matrix G', and the converted matrix G'' in an embodiment of the present invention.

[0129] In step C224, the center value of each sparse pixel in the sparse pixel probability matrix is ​​multiplied by the matrix G'' to obtain a Gaussian matrix distribution matrix centered on the sparse pixel, and the repeated pixels between the obtained multiple Gaussian matrix distribution matrices are superimposed to obtain the rendered heart edge point probability matrix E'.

[0130] Figure 6 : is a diagram of pixel position numbers of the sparse pixel probability matrix E in an embodiment of the present invention. Figure 7 is an example of the sparse pixel probability matrix E and the rendered heart edge point probability matrix E' in the embodiment of the present invention. Figure 7As shown, the part of the matrix E with a black background is the center of each sparse pixel. Multiply the value of each sparse pixel center with the matrix G'' to obtain a Gaussian matrix distribution matrix centered on the sparse pixel. If there are repeated pixels between the multiple Gaussian matrix distribution matrices obtained, the repeated pixels are superimposed. The specific process is: the first sparse pixel center value E10 is 0.89. After 2D Gaussian rendering: E00 = 0.45 (0.89*0.5), E10 = 0.89 (0.89*1), E11 = 0.45 (0.89*0.5), E20 = 0.45 (0.89*0.5), E21 = 0.23 (0.89*0.25) E01 = 0.23 (0.89*0.25); the second sparse value E21 is 0.91. After 2D Gaussian rendering: E10 =0.23(0.91*0.25)+0.89(the value after the first rendering)=1.12, E11=0.43(0.91*0.5)+0.45(the value after the first rendering)=0.88, E12=0.23(0.91*0.25), E20=0.43(0.91*0.5)+0.45(the value after the first rendering)=0.88, E21 =0.91(original value)+0.23(the value after the first rendering)=1.14, E22=0.43(0.91*0.5), E30 =0.23(0.91*0.25), E31 =0.43(0.91*0.5), E32=0.23(0.91*0.25); and so on, the matrix E' is finally generated.

[0131] Step C23 , performing a clustering operation on the heart edge point probability matrix by combining the distance and the probability value to form a dense edge point matrix.

[0132] To truly characterize the heart segmentation boundary information, this embodiment combines distance and probability values ​​to form a new metric, specifically expressed as d = a * probability value + b * Euclidean distance (the true distance between two points in m-dimensional space, or the natural length of a vector, i.e., the distance from the point to the origin). a and b are hyperparameters used to measure the contribution of probability and Euclidean distance, respectively. For example, a is set to 0.8 and b is set to 0.2. The specific clustering steps are as follows: The sparse pixel center point at the highest point on the left side of the rendered heart edge point probability matrix E' (point E10 in E') is used as the starting core point. Within the neighborhood of the current core point (here, the range is set to 3*3), the point with the largest linear sum of distance and probability (i.e., d value) is searched as the next core point. The left side of the heart segmentation map is traced from top to bottom, and the right side is traced from bottom to top. Finally, clustering is performed to form a dense edge point matrix.

[0133] Step C30: Input the dense edge point matrix and the anteroposterior chest X-ray image to be evaluated into a convolutional neural network to extract visual features of the anteroposterior chest X-ray image to be evaluated. This step may specifically include steps C31-C32:

[0134] Step C31 : splicing the to-be-evaluated anteroposterior chest radiograph image and the dense edge point matrix into two-channel matrix data.

[0135] Specifically, to reduce model retraining, the network model from the cardiothoracic ratio calculation stage can be reused. The output of step C20 (the dense edge point matrix) and the anteroposterior chest X-ray image to be evaluated are input into the convolutional neural network. Preprocessing is performed before input. Since the primary focus is on the heart, the dense edge matrix calculated in step C20 can be used to provide edge information for subsequent steps. Therefore, the anteroposterior chest X-ray image to be evaluated and the dense edge matrix are concatenated and preprocessed into two-channel matrix data. This serves as the input to the convolutional neural network in step C32, with an input size of [2, 1024, 1024].

[0136] In step C32, the matrix data is input into a convolutional neural network to extract visual features of the frontal chest X-ray image to be evaluated.

[0137] Specifically, the input of the convolutional neural network is the two-channel matrix data 2*1024*1024 obtained in step C31, with a channel number of 2. The two channels are the orthotopic chest X-ray image and the dense edge point matrix. The size of these two matrices is 1024*1024. 1024 16*16 feature maps can be obtained from the last layer of convolution. The output uses global maximum pooling to obtain a globally pooled feature matrix with a size of 1024*1, which is used as the visual feature of the orthotopic chest X-ray image to be evaluated.

[0138] Step C40, mining the relationship data of simultaneous enlargement of each chamber of the heart based on the historical data set of the frontal chest radiograph report, and then generating a feature matrix based on prior knowledge. This step may specifically include steps C41-C44:

[0139] Step C41 , based on the historical data set of anteroposterior chest radiograph reports and the label categories of cardiac atrium and ventricle enlargement, mining the relational data of simultaneous enlargement of cardiac atria and ventricles, thereby constructing a relational data matrix.

[0140] The method for constructing the historical data set of the AP chest radiograph report may refer to the detailed description of step B60 in the previous method for processing historical data. The label categories include: left atrial enlargement, left ventricular enlargement, right atrial enlargement, and right ventricular enlargement.

[0141] Considering the prior knowledge of heart disease on chest X-rays, for example, for a normal large heart, chest X-rays show that all chambers are enlarged; for a mitral valve heart, chest X-rays show that the right ventricle is enlarged; for an aortic heart, chest X-rays show that the left ventricle is enlarged, etc. It can be seen that the enlargement of each chamber has a certain correlation. Therefore, based on the existing chest X-ray reports, the relationship between category labels in the historical data set is mined, based on the number of heart chamber enlargement label pairs that appear in the historical data set of the frontal chest X-ray reports, the relationship data matrix based on chest X-rays is constructed. . Where C is the number of categories, here C=4. M is a 4*4 matrix, where each element is M ij, , where i is the number of rows and j is the number of columns.

[0142] Specifically, the element M in the matrix M ij Represents the base label L i With other label categories L j The number of times it appears at the same time, where the values ​​of i and j are both 0, 1, 2, and 3. For example, the 4 categories are left atrial enlargement, left ventricular enlargement, right atrial enlargement, and right ventricular enlargement. The i-th row of the M matrix, where i=0, the base label L i Corresponding to left atrial enlargement. When j = 0, the statistical label includes the number of times left atrial enlargement occurs; when j = 1, the statistical label includes the number of times left atrial enlargement and left ventricular enlargement occur together; when j = 2, the statistical label includes the number of times left atrial enlargement and right atrial enlargement occur together; when j = 3, the statistical label includes the number of times left atrial enlargement and right ventricular enlargement occur together. The above is the meaning of the first row of the M matrix when i = 0, j = 0, 1, 2, 3. Similarly, we can obtain the relationship data matrix M.

[0143] Step C42 , calculating the probability of each tag category appearing when other tag categories appear based on the relationship data matrix, thereby obtaining a conditional probability matrix.

[0144] Specifically, the conditional probability matrix P is calculated using the relational data matrix M. ij =P(L j / L i ) represents the label category L i When the label category L appears j Probability of occurrence.

[0145] For example, the relational data matrix M is shown in formula (6):

[0146] (6)

[0147] The methods for calculating the elements in the 0th row of the conditional probability matrix P are: 200 / (200+30+0+10), 30 / (200+30+0+10), 0 / (200+30+0+10) and 10 / (200+30+0+10); the methods for calculating the elements in the 1st row of the matrix P are: 80 / (80+100+23+8), 100 / (80+100+23+8), 23 / (80+100+23+8) and 8 / (80+100+23+8), and so on, the elements in the 2nd and 3rd rows of the matrix P can be calculated.

[0148] Step C43 : Binarize the conditional probability matrix P and introduce weights to obtain the correlation matrix A′.

[0149] In particular, in actual scenarios, it can be found that the above conditional probability matrix P may have problems. For example, some rare heart diseases may cause the cardiac atrioventricular relationship matrix to have a long-tail distribution. In order to meet the general scenario, the atrioventricular relationship noise caused by rare diseases can be processed. In this embodiment, the threshold To filter out noisy edges, the matrix P is binarized to obtain a binary matrix A. However, the disadvantage of A is that it is overly smooth, which affects the true relationship expression of the heart's atria and chambers. Therefore, the weight p that determines the assignment of the node itself and other related nodes is introduced and optimized into the correlation matrix A'. The specific optimization method is shown in formulas (7)-(8):

[0150] (7)

[0151] (8)

[0152] Using formulas (7)-(8), the binary correlation matrix A is transformed into the correlation matrix A'.

[0153] In this embodiment, Take 0.1 and p as 0.2. ij Represents the element in row i and column j of the binary matrix A; A' ij Represents the element in row i and column j of the correlation matrix A'.

[0154] Step C44, calculating a feature matrix based on prior knowledge according to the correlation matrix and the category word embedding vector.

[0155] Specifically, to better learn the results of step C43 and considering the four categories of feature relationships within the heart's atria and ventricles, a two-layer graph convolutional network is constructed in this embodiment. The first layer inputs the correlation matrix A' obtained in step C43 and the category word embedding vector (using one-hot encoding). The first layer outputs [d, C], which serves as the input to the second layer, which outputs [C, D]. d represents the dimension of the category word embedding vector, C represents the number of categories, and D represents the output dimension of the convolutional neural network in step C32. The output of the second layer is a C*D feature matrix, which serves as a feature matrix based on prior knowledge.

[0156] Step C50 , performing a dot product between the feature matrix based on prior knowledge and the visual features of the anteroposterior chest radiograph image to be evaluated, and calculating the multi-category probability of enlargement of each chamber of the heart.

[0157] The multi-category probabilities include: the probability of left atrial enlargement, the probability of left ventricular enlargement, the probability of right atrial enlargement, and the probability of right ventricular enlargement.

[0158] Specifically, to reflect the equal importance of clinical prior knowledge and image feature information, this embodiment sets the output dimension D of the graph convolutional network in step C44 and the output dimension of the convolutional neural network in step C32 to the same value, both 1024. That is, the output of the graph convolutional network is 1024*C, and the output of the convolutional neural network is 1024*1. Finally, in step C50, the visual features of the evaluated frontal chest radiograph output by the convolutional neural network are fused with the feature matrix based on prior knowledge output by the graph convolutional network through a dot product operation to output the multi-category probability of atrioventricular enlargement.

[0159] It should be noted that the convolutional neural network involved in step C30, the graph convolutional network involved in step C40, and the point multiplication algorithm involved in step C50 in this embodiment together constitute the probability judgment model for atrioventricular enlargement. Before executing this embodiment, the atrioventricular enlargement probability judgment model must be trained using historical data. The input of the atrioventricular enlargement probability judgment model consists of two parts: the first part is a 2*1024*1024 matrix concatenated from the frontal chest radiograph image and the dense edge matrix; the second part is the correlation matrix A' and the category word embedding vector extracted from the historical data. The output is a 1*C vector representing the multi-category probability of atrioventricular enlargement.

[0160] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0161] The present invention further provides an embodiment of a computer-readable storage device. The device of this embodiment stores a computer program that can be loaded by a processor and execute the method described above.

[0162] The computer-readable storage device may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other devices that can store program codes.

[0163] Furthermore, the present invention also provides an embodiment of a processing device. The processing device of this embodiment includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor to execute the method described above.

[0164] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0165] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is clearly not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent modifications or substitutions to the relevant technical features, and the technical solutions after such modifications or substitutions will fall within the scope of protection of the present invention.

Claims

1. A method for determining the probability of occurrence of cardiac atrioventricular enlargement, characterized in that: The method comprises: Input the anteroposterior chest X-ray image to be evaluated into the trained cardiopulmonary segmentation model and output the cardiopulmonary segmentation result; Refining the edge points of the heart according to the heart-lung segmentation result, and then calculating the dense edge point matrix of the heart; Inputting the dense edge point matrix and the anteroposterior chest X-ray image to be evaluated into a convolutional neural network to extract visual features of the anteroposterior chest X-ray image to be evaluated; Mining the relationship data of simultaneous enlargement of each chamber of the heart based on the historical dataset of frontal chest radiograph reports, and then generating a feature matrix based on prior knowledge; Performing a dot product of the feature matrix based on prior knowledge and the visual features of the anteroposterior chest radiograph image to be evaluated to calculate a multi-category probability of enlargement of each atrium and chamber of the heart; The multi-category probabilities include: a probability of left atrium enlargement, a probability of left ventricle enlargement, a probability of right atrium enlargement, and a probability of right ventricle enlargement.

2. The method for determining the probability of occurrence of cardiac atrioventricular enlargement according to claim 1, characterized in that: The step of “finely extracting the edge points of the heart according to the heart and lung segmentation result, and then calculating a dense edge point matrix of the heart” includes: Extracting the outermost pixels of the left and right edges of the heart respectively according to the cardiopulmonary segmentation result, and constructing a sparse pixel probability matrix based on the probability values ​​corresponding to the outermost pixels in the segmentation result of the cardiopulmonary segmentation model; Applying a two-dimensional Gaussian function to the sparse pixel probability matrix to obtain a rendered heart edge point probability matrix; The heart edge point probability matrix is ​​clustered by combining the distance and the probability value to form a dense edge point matrix.

3. The method for determining the probability of occurrence of cardiac atrioventricular enlargement according to claim 2, characterized in that: The step of “applying a two-dimensional Gaussian function to the sparse pixel probability matrix to obtain a rendered heart edge point probability matrix” includes: The parameters μ and σ in the two-dimensional Gaussian function are set to preset values, and a 3*3 Gaussian matrix G is generated based on the Gaussian kernel position matrix; Wherein, the Gaussian kernel position matrix is: Divide each element in the Gaussian matrix G by the sum of all elements in the matrix to obtain a normalized matrix G′; Divide each element of the matrix G′ by the element at the center of the matrix to obtain a transformed matrix G″; Multiply the center value of each sparse pixel in the sparse pixel probability matrix by the matrix G″ to obtain a Gaussian matrix distribution matrix centered on the sparse pixel, and superimpose the repeated pixels between the multiple Gaussian matrix distribution matrices to obtain the rendered heart edge point probability matrix.

4. The method for determining the probability of occurrence of cardiac atrioventricular enlargement according to claim 2, characterized in that: The step of “combining the distance and the probability value to perform a clustering operation on the heart edge point probability matrix to form a dense edge point matrix” includes: The sparse pixel center point at the highest point on the left side of the rendered heart edge point probability matrix is ​​used as the starting core point, and the point with the largest linear sum of distance and probability is found within the neighborhood of the current core point as the next core point. The left tracing method of the heart segmentation map is set from top to bottom, and the right tracing method is set from bottom to top. Finally, clustering is performed to form the dense edge point matrix.

5. The method for determining the probability of occurrence of cardiac atrioventricular enlargement according to claim 1, characterized in that: The step of “inputting the dense edge point matrix and the orthotopic chest X-ray image to be evaluated into a convolutional neural network to extract visual features of the orthotopic chest X-ray image to be evaluated” includes: splicing the to-be-evaluated anteroposterior chest radiograph image and the dense edge point matrix into two-channel matrix data; The matrix data is input into a convolutional neural network to extract visual features of the frontal chest X-ray image to be evaluated.

6. The method for determining the probability of occurrence of cardiac atrioventricular enlargement according to claim 1, characterized in that: The steps of "mining the relationship data of simultaneous enlargement of cardiac chambers based on the historical dataset of anteroposterior chest radiograph reports, and then generating a feature matrix based on prior knowledge" include: Mining the relationship data of simultaneous enlargement of each atrium and chamber of the heart based on the historical data set of anteroposterior chest radiograph reports and the label categories of cardiac atrium and chamber enlargement, thereby constructing a relationship data matrix; the label categories include: left atrial enlargement, left ventricular enlargement, right atrial enlargement, and right ventricular enlargement; Calculating the probability of each tag category appearing when other tag categories appear based on the relationship data matrix, thereby obtaining a conditional probability matrix; Binarizing the conditional probability matrix and introducing weights to obtain a correlation matrix; The feature matrix based on prior knowledge is calculated according to the correlation matrix and the category word embedding vector.

7. A computer-readable storage device, characterized in that: The computer program is stored and can be loaded by a processor to execute the method according to any one of claims 1 to 6.

8. A processing device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method according to any one of claims 1 to 6.

Images