Image processing method, device and storage medium

Abstract

Embodiments of the present application provide an image processing method and device and a storage medium. In the embodiments of the present application, a coronary vessel region and a lesion sub-region containing a plaque in the coronary vessel region in a medical image can be automatically identified, feature extraction and quantification are respectively performed on the coronary vessel region and the lesion sub-region, first medical features and second medical features are obtained, and the first medical features and the second medical features are sent into a pre-trained image classification model to obtain a classification result of the medical image. In the whole process, potential information in the medical image can be deeply mined, the accuracy of identifying the classification result of the medical image is improved, and the demand of actual application is met.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus and storage medium. Background Technology

[0002] Coronary computed tomography angiography (CCTA) is a commonly used medical imaging technique in clinical practice. Specifically, after intravenous injection of an appropriate contrast agent, multi-slice spiral CT is used to scan the coronary arteries, acquiring images containing coronary artery information. Based on these CCTA images, the condition of coronary artery lesions can be understood. Currently, the interpretation of CCTA images relies heavily on the subjective judgment of radiologists. CCTA images contain a large amount of medical information, and because radiologists can only utilize visual information, the accuracy of identification is relatively low. Summary of the Invention

[0003] This application provides an image processing method, apparatus, and storage medium to improve the accuracy of classification results for recognizing medical images.

[0004] This application provides an image processing method, comprising: receiving a medical image containing coronary vessels sent by a medical imaging device; identifying and marking the coronary vessel region in the medical image based on the structural features of the coronary vessels; identifying lesion sub-regions containing plaques within the coronary vessel region using an adversarial generative network model; extracting and quantifying features from the coronary vessel region and the lesion sub-region respectively to obtain a first medical feature and a second medical feature; and feeding the first medical feature and the second medical feature into a pre-trained image classification model to obtain a classification result for the medical image.

[0005] This application also provides a medical image processing system, including: a medical imaging device, an image processing device, and a medical terminal; the medical imaging device is used to scan a specified object to obtain a medical image containing coronary vessels, and upload the medical image to the image processing device and the medical terminal; the image processing device is used to identify and mark the coronary vessel region in the medical image based on the structural features of the coronary vessels; to identify lesion sub-regions containing plaques in the coronary vessel region using an adversarial generative network model; to extract and quantify features of the coronary vessel region and the lesion sub-region respectively to obtain a first medical feature and a second medical feature; to input the first medical feature and the second medical feature into a pre-trained image classification model to obtain the classification result of the medical image, and return the classification result to the medical terminal; the medical terminal is used to receive the medical image and the classification result, and to display the classification result and the medical image together on its display screen.

[0006] This application embodiment also provides an image processing device, including: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program to: receive a medical image containing coronary vessels sent by a medical imaging device; identify and label the coronary vessel region in the medical image based on the structural features of the coronary vessels; identify lesion sub-regions containing plaques in the coronary vessel region using an adversarial generative network model; extract and quantify features from the coronary vessel region and the lesion sub-region respectively to obtain a first medical feature and a second medical feature; and input the first medical feature and the second medical feature into a pre-trained image classification model to obtain the classification result of the medical image.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the image processing method provided in this application.

[0008] In this embodiment, the system can automatically identify coronary artery regions and lesion sub-regions containing plaques in medical images. Features are extracted and quantified from both the coronary artery regions and the lesion sub-regions to obtain first and second medical features. These first and second medical features are then fed into a pre-trained image classification model to obtain the classification result of the medical image. Throughout this process, the system can deeply mine the potential information in medical images, improving the accuracy of the classification results and meeting the needs of practical applications. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0010] Figure 1 A schematic flowchart of an image processing method provided for an exemplary embodiment of this application;

[0011] Figure 2 A schematic diagram of the structure of a medical image processing system provided for an exemplary embodiment of this application;

[0012] Figure 3 A schematic diagram of the structure of a target patch segmentation model provided for an exemplary embodiment of this application;

[0013] Figure 4 A schematic diagram of the structure of an adversarial generative network model provided as an exemplary embodiment of this application;

[0014] Figure 5 This is a schematic diagram of the structure of an image processing device provided for an exemplary embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To address the low accuracy of medical information recognition by radiologists in existing technologies, this application embodiment identifies coronary artery regions and lesion sub-regions containing plaques in medical images. Features are extracted and quantified from both the coronary artery regions and the lesion sub-regions to obtain first and second medical features. These first and second medical features are then fed into a pre-trained image classification model to obtain the classification result of the medical image. Throughout this process, the potential information within the medical image can be deeply mined, improving the accuracy of the classification results and meeting the needs of practical applications.

[0017] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0018] Figure 1 This is a schematic flowchart illustrating an image processing method provided for an exemplary embodiment of this application. Figure 1 As shown, the method includes:

[0019] 101. Receive medical images containing coronary arteries sent by medical imaging equipment;

[0020] 102. Based on the structural characteristics of coronary arteries, identify and label coronary artery regions in medical images;

[0021] 103. Using a generative adversarial network model, identify lesion sub-regions containing plaques in the coronary artery region;

[0022] 104. Features were extracted and quantified from the coronary artery region and the lesion sub-region to obtain the first medical feature and the second medical feature.

[0023] 105. Input the first medical feature and the second medical feature into the pre-trained image classification model to obtain the classification result of the medical image.

[0024] In this embodiment, the coronary arteries are also called coronary vessels, which are the blood vessels that supply blood to the heart. In some scenarios, coronary vessels may contain plaques, which are substances containing components such as calcification, fibrosis, or lipids. Plaques in the coronary vessels may affect the blood supply to the heart.

[0025] In this embodiment, the medical imaging device is a device capable of acquiring medical images containing coronary arteries. The medical imaging device can be a camera or a computed tomography (CT) scanner, etc. For example, a medical imaging device implemented as a camera can acquire medical images containing coronary arteries from a simulated medical model; another example is a medical imaging device implemented as a CT scanner, which can acquire medical images containing coronary arteries.

[0026] In this embodiment, regardless of the type of medical imaging device, it can receive medical images containing coronary vessels sent by the medical imaging device. After receiving the medical image, the coronary vessel regions in the medical image can be identified and marked based on the structural features of the coronary vessels. These structural features include: the left anterior descending branch, diagonal branch, ventricular septal branch, circumflex branch, and obtuse marginal branch of the left coronary artery; and the right acute marginal branch, left ventricular posterior branch, and posterior descending branch, etc. For example, the medical image can be compared with the structural features of the coronary vessels to identify the coronary vessel regions in the medical image. After identifying the coronary vessel regions, they can be marked with labeling information. For example, the coronary vessel regions in the medical image can be marked with some visual information, making it more intuitive and convenient to understand the coronary vessel regions in the medical image. The labeling information can be textual information, such as the names of the coronary vessel structural features. In addition, the labeling information can also be lines with different visual attributes, where the visual attributes of the lines include at least one of color, line type, and line width.

[0027] In this embodiment, a medical image marked with coronary artery regions can be fed into an adversarial generative network (GCN) model. The GCN model is used to identify lesion sub-regions containing plaques within the coronary artery regions. Specifically, the GCN model is used for automatic identification of lesion sub-regions containing plaques within the vascular region. Details regarding the GCN model can be found in subsequent embodiments and will not be elaborated here. After identifying the coronary artery regions and lesion sub-regions in the medical image, feature extraction and quantification can be performed on both regions to obtain a first medical feature and a second medical feature. The first medical feature is a feature extracted from the coronary artery region. Optionally, the first medical feature is a feature related to the coronary artery. For example, the first medical feature may be a morphological feature such as the shape or curvature of the coronary artery; or, for example, the first medical feature may be the grayscale feature, texture feature, etc., of the coronary artery region. The second medical feature is a feature extracted from the lesion sub-region. Optionally, the second medical feature is a feature related to the plaque. For example, the second medical feature may be the compositional feature, shape feature, density feature, or texture feature of the plaque. After obtaining the first and second medical features, the first and second medical features can be fed into a pre-trained image classification model to obtain the classification results of the medical image.

[0028] Image classification models can be machine learning models, such as Support Vector Machines (SVMs), Decision Tree Analysis, Random Forests, or Logistic Regression. SVMs improve the generalization ability of image classification models by seeking the minimum structured risk, minimizing empirical risk and confidence range, thus achieving good statistical regularity even with a small sample size. Hinge loss is used to significantly improve the generalization performance of the prediction model by selecting the hyperplane with the largest margin, while radial basis functions are used as kernel functions to establish a nonlinear model. Random forests randomly build multiple decision trees to determine the classification of input samples. Random forests have good generalization ability, can detect interactions between features, and can rank the importance of features.

[0029] The image classification model can classify medical images based on a first medical feature and a second medical feature, obtaining a classification result. The classification result is not limited. For example, it can be determined based on the classification objective (classification criterion). The objective could be whether a plaque in a coronary artery can be successfully opened via interventional surgery or not; it could also be whether the plaque is dense or non-dense. Furthermore, the classification result is categorized as a first-class medical image or a second-class medical image. For each medical image corresponding to the first and second medical features, the machine learning model can also output the probability that the medical image belongs to the first-class image and the probability that it belongs to the second-class image. For example, the probability that the medical image is a first-class image is 10%, and the probability that it is a second-class image is 90%.

[0030] In this embodiment, coronary artery regions and lesion sub-regions containing plaques in medical images can be identified. Features are extracted and quantified from both the coronary artery regions and the lesion sub-regions to obtain first and second medical features. These first and second medical features are then fed into a pre-trained image classification model to obtain the classification result of the medical image. Throughout this process, potential information in the medical image can be deeply mined (e.g., medical features are extracted), improving the accuracy of the classification results and meeting the needs of practical applications. Furthermore, it can improve the accuracy of predicting the difficulty of surgical recanalization of plaque-containing blood vessels.

[0031] In one optional embodiment, the generative adversarial network model includes a target plaque segmentation model capable of image segmentation, such as segmenting lesion sub-regions containing plaques within a coronary artery region in a medical image. The target plaque segmentation model can be any model capable of image segmentation; for example, it may include, but is not limited to, fully convolutional networks (FCNs), segmentation networks (SegNets), or conditional random fields as recurrent neural networks. Based on this, an implementation method for identifying lesion sub-regions containing plaques within a coronary artery region using the generative adversarial network model includes: inputting a medical image with labeled coronary artery regions into the generative adversarial network model; and using the target plaque segmentation model to perform image segmentation on the labeled coronary artery regions in the medical image to identify lesion sub-regions containing plaques within the coronary artery region. Figure 3As shown, the target patch segmentation model consists of a convolutional coding network layer and a convolutional decoding network layer. The convolutional coding network layer includes: a convolutional layer, a batch normalization (BN) layer, a non-linear activation (ReLU) layer, and a pooling layer; the convolutional decoding network layer includes: a convolutional layer, a BN layer, a ReLU layer, an upsampling layer, and an activation function (Softmax) layer.

[0032] Alternatively, the adversarial generative network model can be obtained through training, such as... Figure 4 As shown, an initial patch segmentation model and an adversarial model can be trained to obtain an adversarial generative network model. During model training, the training of the initial patch segmentation model and the adversarial model can be viewed as a game. The initial patch segmentation model strives to segment lesion sub-regions as accurately as possible, while the adversarial model continuously evaluates and provides feedback on the image segmentation results of the initial patch segmentation model, thus improving the initial patch segmentation model's ability to segment lesion sub-regions. In this way, the adversarial generative network model can continuously improve its segmentation performance. Specifically, reference sample images and adversarial sample images containing coronary arteries are acquired. The reference sample image refers to a sample image in which plaque sub-regions within the coronary artery region have been labeled. For example, the reference sample image could be the gold standard segmentation result, i.e., a sample image in which the physician has labeled the plaque sub-regions within the coronary artery region. The adversarial sample image refers to a sample image in which plaque sub-regions within the coronary artery region have not been labeled. The initial plaque segmentation model in the adversarial generative network (GNN) model is used to segment the adversarial sample image to label the plaque sub-regions contained in the adversarial sample image. The adversarial model in the GNN model is then used to perform a difference analysis on the plaque sub-regions in the segmented sample image output by the initial plaque segmentation model and the reference sample image. The difference analysis results are fed back to the initial plaque segmentation model to iteratively update the initial plaque segmentation model until the difference analysis results meet the set requirements, thus obtaining the target plaque segmentation model. This achieves accurate segmentation and identification of lesion sub-regions containing plaques. The entire process can be viewed as a game between the initial patch segmentation model and the adversarial model. Through continuous iteration and updates, the two models eventually reach a dynamic equilibrium, meaning the adversarial model cannot determine whether the data provided by the initial patch segmentation model is a real image (i.e., a reference sample image), with a discrimination accuracy of approximately 50%, approximating random guessing. At this point, the initial patch segmentation model, after continuous iteration and updates, becomes the target patch segmentation model. Figure 4As shown, the initial patch segmentation model performs image segmentation on the adversarial sample image to obtain adversarial patch sub-regions containing patches in the adversarial sample image; the adversarial patch sub-regions and reference sample sub-regions containing patches in the reference sample image are evaluated, and the difference analysis is performed by combining the adversarial sample image and the reference sample image to obtain the difference analysis results.

[0033] In an optional embodiment, after acquiring the coronary artery region and the lesion sub-region containing plaques, a first medical feature related to the coronary artery and a second medical feature related to the plaque can be extracted from the coronary artery region and the lesion sub-region, respectively. Specifically, medical features related to the classification result can be pre-selected, referred to as selected medical features. Selected medical features are those medical features that can assist in classifying medical images. Based on the pre-selected medical features, feature extraction and quantization are performed on the coronary artery region and the lesion sub-region, respectively, to obtain the first medical feature and the second medical feature.

[0034] Optionally, an implementation method for extracting and quantifying features of a coronary artery region according to a pre-selected medical feature type to obtain a first medical feature under the selected medical feature type includes: extracting morphological features and imaging features of the coronary artery from the coronary artery region, and quantifying the morphological features and imaging features to obtain quantified morphological features and imaging features; selecting a first medical feature from the quantified morphological features and imaging features according to the selected medical feature, wherein the first medical feature includes at least one of the quantified morphological features and imaging features.

[0035] Optionally, the morphological features of the coronary arteries include at least one of the diameter and tortuosity of the coronary arteries. The diameter can be measured based on the coronary artery region in a medical image; for example, the size of the diameter in the medical image can be measured, and then the actual diameter size can be obtained according to the transformation relationship between the image coordinate system and the world coordinate system. When measuring the tortuosity of the coronary arteries, the coronary artery is divided into two segments at the corner, and the angle between the centerlines of the two segments is measured as the tortuosity of the coronary artery. The image features of the coronary arteries include at least one of statistical features and texture features. Statistical features can be obtained by using gray-level histogram statistics to extract the maximum, minimum, median, mean, variance, skewness, or kurtosis of gray levels within the coronary artery region in the medical image. Texture features mainly refer to the texture heterogeneity of blood vessels that are difficult to quantify with the naked eye. There are two ways to acquire texture features. One is to obtain the gray-level spatial correlation characteristics of the coronary artery region in medical images to obtain the gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), and gray-level size zone matrix (GLSZM), and then obtain texture features based on these matrices. The other is to use the Laws filtering method to perform Laws filtering on medical images containing the coronary artery region to obtain a filtering matrix that reflects texture information such as points, lines, and ripples, and then obtain texture features based on this filtering matrix.

[0036] Optionally, an implementation method for extracting and quantifying features of a lesion sub-region according to a pre-selected medical feature type to obtain a second medical feature under the selected medical feature type includes: extracting plaque component features and plaque image features from the lesion sub-region, and quantifying the plaque component features and plaque image features to obtain quantized plaque component features and plaque image features; selecting a second medical feature from the quantized plaque component features and plaque image features according to the selected medical feature, wherein the second medical feature includes at least one of the quantized plaque component features and plaque image features.

[0037] Plaque composition features mainly refer to characteristics that reflect plaque components, such as at least one of calcification, fibrosis, and lipids. In the process of extracting and quantifying plaque composition features from a lesion sub-region, the grayscale distribution within the lesion sub-region can be used for quantitative component analysis to obtain the proportions of calcification, fibrosis, and lipids in the plaque as plaque composition features. Plaque image features are extracted from the lesion sub-region using radiomics analysis methods, including shape features, statistical features, or wavelet domain features. The process of extracting and quantifying plaque image features from a lesion sub-region involves three main scenarios: First, extracting plaque shape information, such as plaque edges or spiculations. Based on this shape information, shape features can be extracted, such as plaque length, volume, surface area, or edge smoothness. After obtaining the shape features, these features can be quantified to obtain the quantified shape features. The second method involves obtaining plaque density information, i.e., plaque statistical characteristics, based on the gray-level statistics of the lesion sub-region. These statistical characteristics are then quantified to obtain the quantified statistical features. Gray-level statistics include, but are not limited to, the mean, variance, skewness, or kurtosis of the plaque gray levels. Density information primarily reflects the density of the plaque's components. For example, if a plaque is entirely composed of calcification, it is considered to have high density; if a plaque comprises 50% fat and 50% calcification, it is considered to have low density. The third method involves performing multi-scale wavelet filtering on the lesion sub-region to obtain images in different wavelet domains. Feature extraction is then performed on these images to obtain wavelet domain features that are not readily apparent to the naked eye. These wavelet domain features are then quantified to obtain the quantified wavelet domain features.

[0038] In an optional embodiment, the method provided in this application further includes a process of pre-selecting medical features. Specifically, multiple sample medical images containing coronary arteries can be pre-acquired, and the sample medical images can be divided into two categories: a first category of sample medical images and a second category of sample images. The first and second categories of sample medical images can be used as classification results of an image classification model. For example, the first category of sample medical images consists of plaques that can be opened, and the second category of sample images consists of plaques that cannot be opened. Feature extraction and quantization are performed on each sample medical image to obtain quantized morphological features and image features. The medical features extracted from the first category of sample medical images can form a first feature set, and the medical features extracted from the second category of sample medical images can form a second feature set. In this embodiment, plaques in the vascular region divide the blood vessel into a proximal and distal end. Blood flow in the blood vessel flows from the proximal end to the distal end. Based on the above, for any feature in the quantified morphological and imaging features, a third difference value is calculated between a first difference value between the proximal and distal ends of the blood vessel in the first feature set and a second difference value between the proximal and distal ends of the blood vessel in the second feature set. If the third difference value meets the statistical difference condition, the feature is selected as the medical feature. Specifically, for any feature A1 in the quantified morphological and imaging features, the difference between the feature value B1 corresponding to the proximal end of feature A1 and the feature value B2 corresponding to the distal end of the blood vessel in the first feature set is the first difference value B3, and the difference between the feature value C1 corresponding to the proximal end of feature A1 and the feature value C2 corresponding to the distal end of the blood vessel in the second feature set is the second difference value C3. If the third difference value D1 between the first difference value B3 and the second difference value C3 satisfies the statistical difference condition, then feature A1 is considered helpful for subsequent image classification, and therefore can be selected as a medical feature. If the third difference value D1 between the first difference value B3 and the second difference value C3 does not satisfy the statistical difference condition, then feature A1 is considered not to affect image classification, and therefore can not be selected as a medical feature. The statistical difference condition refers to the conditions that must be met when using statistical methods to analyze the difference in the third difference value D1. The statistical difference condition can be that the third difference value D1 is greater than a set threshold, or that the third difference value D1 is less than a set threshold. Alternatively, the statistical difference condition can also be reflected by the significance P-value. Statistical analysis is performed on the third difference value D1 using statistical methods to obtain the significance P-value. If the significance P-value is less than 0.05, then the third difference value D1 is considered sufficiently large to meet the statistical difference condition, and this feature A1 can influence image classification. Therefore, feature A1 is selected as the medical feature.It should be noted that an image classification model can be pre-trained based on the selected medical features and the classification results of the medical images corresponding to those selected medical features.

[0039] In this embodiment, during the process of pre-determining the selected medical features, a feature screening method with comparative properties can be used. For example, methods such as difference-in-differences analysis can be used to screen the quantified morphological and imaging features to obtain the selected medical features. The process of determining the selected medical features is explained below using a difference-in-differences regression model as an example.

[0040] Specifically, the difference-in-differences regression model is defined as follows:

[0041] Y(T,G)=α+β1*T+β2*G+β3*T*G+δ

[0042] In this model, Y represents the quantified vascular characteristics (morphological and imaging features), T represents the dummy variable for the vascular segment (0 for proximal and 1 for distal), G represents the dummy variable for the group (0 for non-recanalized plaques and 1 for recanalized plaques); α is the constant term in the difference-in-differences regression model; β1 is the regression coefficient of the dummy variable for the vascular segment in the difference-in-differences regression model; β2 is the regression coefficient of the grouping variable in the difference-in-differences regression model; β3 is the regression coefficient of the product term of the grouping dummy variable and the dummy variable for the vascular segment in the difference-in-differences regression model; and δ is the estimation error of the difference-in-differences regression model. It should be noted that dummy variables are virtual variables, also known as nominal variables or dummy variables. They are quantified independent variables (e.g., proximal or source of the vessel, non-recanalized or recanalized plaques), and typically take values ​​of 0 or 1. Introducing dummy variables can make the regression model more complex, but it provides a simpler description of the problem. One equation can achieve the effect of two equations, and it is closer to reality, thus improving the accuracy of regression analysis.

[0043] The first type of sample medical images represents the plaque recanalization group, and the second type of sample medical images represents the plaque non-recanalization group. For any feature, the third difference value between the first difference value of that feature in the proximal and distal segments of the vessel within the plaque recanalization group and the second difference value in the proximal and distal segments of the vessel within the non-recanalization group can be estimated as follows:

[0044] DID=(Y(1,1)-Y(0,1))-(Y(1,0)-Y(0,1))=β3

[0045] Statistical analysis of the third difference value β3 is used to determine whether any feature is a selected medical feature. For example, statistical analysis of β3 is performed to obtain the significance p-value of any feature. If the significance p-value is less than 0.05, then any feature is determined to be a selected medical feature.

[0046] In an optional embodiment, the selected medical features include not only the morphological and imaging features corresponding to coronary arteries, but also the plaque composition features and plaque image features corresponding to plaques. Based on this, the method provided in this application further includes: for each sample medical image, obtaining quantified plaque composition features and plaque image features corresponding to the sample medical image; and using a lasso regression analysis method to select key features from the quantified plaque composition features and plaque image features as the selected medical features. The key features can be quantified plaque composition features and plaque image features with feature coefficients less than a set threshold, or quantified plaque composition features and plaque image features with feature coefficients non-zero; there is no limitation on this.

[0047] Lasso-Logistic Regression analysis is defined as follows:

[0048]

[0049] Where RSS stands for Residual Sum of Squares. In this embodiment, patch component features and patch image features are referred to as patch features, y i Let x be the predicted label for the i-th sample, where the sample is a medical image containing coronary arteries. There are two types of predicted labels: the first indicates the plaque is operable, and the second indicates the plaque is not operable; ij For the j-th patch feature of the i-th sample, β j Let be the feature coefficient of the j-th patch feature, d be the number of patch features, ε be the estimation bias, N be the total number of samples, and the sigmoid function, also called the logistic function, can map a real number to the interval (0,1).

[0050] In the Lasso-Logistic regression analysis, the least squares method is used to minimize the sum of squared residuals (RSS). The least squares method is used to minimize the characteristic coefficients β in the regression model. j A common method for estimation is lasso regression analysis, which incorporates the L1 norm as a constraint penalty term into the RSS. In lasso regression analysis, the sum of the RSS and the lasso penalty term needs to be minimized.

[0051]

[0052] Wherein, the L1 norm is The sparsity parameter λ, the L1 norm, can be adjusted by changing the size of λ to reduce the feature coefficients β of redundant and irrelevant features in the Lasso-Logistic regression model. jConstraining λ to 0 adjusts the sparsity of the model. A larger λ allows more feature coefficients to be set to zero, resulting in greater model sparsity, but may decrease predictive performance. A smaller λ retains more secondary medical features in the model, but also increases the likelihood of overfitting. Therefore, choosing an appropriate λ allows for achieving good predictive performance while maintaining good model sparsity and avoiding overfitting.

[0053] In addition to the image processing methods described above, this application also provides a medical image processing system, such as... Figure 2 As shown, the medical image processing system 200 includes: a medical imaging device 201, an image processing device 202, and a medical terminal 203.

[0054] In this embodiment, the medical imaging device 201 can be implemented as various sensors such as a monocular camera, a binocular camera, a lidar, a microwave radar, or an infrared radar; the medical imaging device 201 can also be implemented as a CT scanner. The image processing device 202 can be implemented as a server-side device such as a conventional server, a cloud server, or a server array; it can also be implemented as a terminal device such as a desktop computer, a laptop computer, or a smartphone. The medical terminal 203 can be implemented as a desktop computer, a laptop computer, a smartphone, or a self-service terminal. Figure 2 The illustration uses the Sino-Israeli medical imaging device 201 (camera), image processing device 202 (cloud server), and medical terminal 203 (autonomous service terminal) as examples, but is not limited to these.

[0055] In this embodiment, the medical imaging device 201 can scan a designated object to obtain a medical image containing coronary vessels, and upload the medical image to the image processing device 202 and the medical terminal 203. The designated object can be a simulated medical model containing coronary vessels or a 3D model containing coronary vessels. The image processing device 202 can identify and label the coronary vessel region in the medical image based on the structural features of the coronary vessels; use an adversarial generative network model to identify lesion sub-regions containing plaques within the coronary vessel region; extract and quantify features from the coronary vessel region and the lesion sub-region respectively to obtain a first medical feature and a second medical feature; input the first and second medical features into a pre-trained image classification model to obtain a classification result for the medical image, and return the classification result to the medical terminal 203. The medical terminal 203 can receive the medical image and the classification result, and display the classification result and the medical image together on its display screen.

[0056] In this embodiment, the medical imaging device 201 includes a wireless communication module. Through this module, the medical imaging device 201 connects to the Internet and uploads medical images containing coronary arteries to a cloud server (image processing device 202). Figure 2As shown. Additionally, the medical imaging device 201 can also send medical images containing coronary arteries to the medical terminal 203 via the Internet. Or, as... Figure 2 As shown, the medical imaging device 201 and the medical terminal 203 are located on the same local area network (LAN). The medical imaging device includes a Wi-Fi module, through which the medical imaging device 201 connects to the LAN and sends medical images containing coronary arteries to the medical terminal 203. Alternatively, the medical imaging device 201 uploads medical images containing coronary arteries to a cloud server via the LAN.

[0057] In addition, such as Figure 2 As shown, the image processing device 202 can provide the classification results of medical images to the medical terminal 203 via the Internet. Alternatively, the image processing device 202 can provide the classification results of medical images to the medical terminal 203 via a local area network.

[0058] The medical image processing system provided in this application can automatically identify coronary artery regions and lesion sub-regions containing plaques in medical images. It extracts and quantifies features from both the coronary artery regions and lesion sub-regions to obtain first and second medical features. These first and second medical features are then fed into a pre-trained image classification model to obtain the classification result of the medical image. Throughout this process, the system can deeply mine the potential information in medical images, improving the accuracy of the classification results and meeting the needs of practical applications.

[0059] In this embodiment, the above-described image processing method or system is not only used to process medical images containing coronary vessels, but also applicable to any crisscrossing linear or tubular objects similar to coronary vessels, and images containing such linear or tubular objects can be used in this embodiment.

[0060] For example, for a traffic map containing main roads acquired through satellite systems or cameras, the main road areas in the traffic map can be identified and marked based on their structure. Traffic congestion may occur on these main road areas due to traffic control or accidents; these congested areas can be considered as sub-regions of interest within the main road areas. Generative adversarial networks (GANs) are used to identify these sub-regions of interest. Features are extracted and quantified from both the main road areas and the sub-regions of interest to obtain road features and congestion features. These features are then fed into a pre-trained image classification model to obtain the classification result of the traffic map. The classification result could indicate that the congested area in the traffic map can be cleared within half an hour, or that a traffic accident occurred in the congested area, or that no traffic accident occurred in the congested area.

[0061] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 103 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0062] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0063] Figure 5 This is a schematic diagram of the structure of an image processing device provided for an exemplary embodiment of this application. For example... Figure 5 As shown, the image processing device includes a memory 54 and a processor 55.

[0064] Memory 54 is used to store computer programs and can be configured to store various other data to support operation on the image processing device. Examples of this data include instructions for any application or method used to operate on the image processing device. Furthermore, memory 54 may also store medical image data, etc.

[0065] The memory 54 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0066] The processor 55, coupled to the memory 54, is used to execute the computer program in the memory 54 for: receiving a medical image containing coronary vessels sent by a medical imaging device; identifying and labeling the coronary vessel region in the medical image based on the structural features of the coronary vessels; identifying lesion sub-regions containing plaques in the coronary vessel region using an adversarial generative network model; extracting and quantifying features from the coronary vessel region and the lesion sub-region respectively to obtain a first medical feature and a second medical feature; and feeding the first medical feature and the second medical feature into a pre-trained image classification model to obtain the classification result of the medical image.

[0067] In an optional embodiment, when the processor 55 identifies lesion sub-regions containing plaques in the coronary artery region using an adversarial generative network model, it specifically performs the following: inputting a medical image of the marked coronary artery region into the adversarial generative network model, which includes a trained target plaque segmentation model; and using the target plaque segmentation model to perform image segmentation on the marked coronary artery region in the medical image to mark the lesion sub-regions containing plaques in the coronary artery region.

[0068] In an optional embodiment, the processor 55 is further configured to: acquire a reference sample image and an adversarial sample image containing coronary arteries, wherein the reference sample image refers to a sample image in which plaque sub-regions contained in the coronary artery region have been marked; and the adversarial sample image refers to a sample image in which plaque sub-regions contained in the coronary artery region have not been marked; segment the adversarial sample image using an initial plaque segmentation model in the adversarial generative network model to mark the plaque sub-regions contained in the adversarial sample image; perform difference analysis on the segmented sample image and the plaque sub-regions in the reference sample image output by the initial plaque segmentation model using the adversarial model in the adversarial generative network model, and feed back the difference analysis results to the initial plaque segmentation model so that the initial plaque segmentation model can be iteratively updated until the difference analysis results meet the set requirements, thereby obtaining the target plaque segmentation model.

[0069] In an optional embodiment, when the processor 55 extracts and quantizes features from the coronary artery region and the lesion sub-region to obtain the first medical feature and the second medical feature, it is specifically used to: extract and quantize features from the coronary artery region and the lesion sub-region according to the pre-selected medical features to obtain the first medical feature and the second medical feature.

[0070] In an optional embodiment, when the processor 55 extracts and quantizes features of the coronary artery region according to a pre-selected medical feature type to obtain a first medical feature under the selected medical feature type, it is specifically used to: extract morphological features and imaging features of the coronary artery from the coronary artery region, and quantize the morphological features and imaging features to obtain quantized morphological features and imaging features; select a first medical feature from the quantized morphological features and imaging features according to the selected medical feature, wherein the first medical feature includes at least one of the quantized morphological features and imaging features.

[0071] In one optional embodiment, the morphological features of the coronary vessels include at least one of the diameter and tortuosity of the coronary vessels; the imaging features of the coronary vessels include at least one of the statistical features and texture features of the coronary vessels.

[0072] In an optional embodiment, when the processor 55 extracts and quantizes features from the lesion sub-region according to a pre-selected medical feature type to obtain a second medical feature under the selected medical feature type, it is specifically used to: extract plaque component features and plaque image features from the lesion sub-region, and quantize the plaque component features and plaque image features to obtain quantized plaque component features and plaque image features; select a second medical feature from the quantized plaque component features and plaque image features according to the selected medical feature, wherein the second medical feature includes at least one of the quantized plaque component features and plaque image features.

[0073] In an optional embodiment, plaques in the vascular region divide the blood vessel into proximal and distal ends. The processor 55 is further configured to: pre-acquire multiple sample medical images containing coronary vessels; for each sample medical image, acquire quantified morphological features and image features corresponding to the sample medical image; for any feature in the quantified morphological features and image features, calculate a third difference value between a first difference value between the proximal and distal ends of the blood vessel in a first feature set and a second difference value between the proximal and distal ends of the blood vessel in a second feature set; if the third difference value satisfies the statistical difference condition, select any feature as the selected medical feature; wherein the first feature set contains quantified medical features extracted from a first type of sample medical images, and the second feature set contains quantified medical features extracted from a second type of sample medical images.

[0074] In an optional embodiment, the processor 55 is further configured to: for each sample medical image, acquire quantified plaque component features and plaque image features corresponding to the sample medical image; and use a lasso regression analysis method to select key features from the quantified plaque component features and plaque image features as selected medical features.

[0075] The image processing device provided in this application embodiment can automatically identify coronary artery regions and lesion sub-regions containing plaques in medical images. It extracts and quantifies features from both the coronary artery regions and the lesion sub-regions to obtain first and second medical features. These first and second medical features are then fed into a pre-trained image classification model to obtain the classification result of the medical image. Throughout this process, the device can deeply mine the potential information in medical images, improving the accuracy of the classification results and meeting the needs of practical applications.

[0076] Furthermore, such as Figure 5 As shown, the image processing device also includes other components such as a communication component 56, a display 57, a power supply component 58, and an audio component 59. Figure 5 The image processing device only shows some components schematically and does not mean that it includes only a portion of the components. Figure 5 The components shown. It should be noted that... Figure 5 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the image processing equipment.

[0077] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the image processing method provided in embodiments of this application.

[0078] The above Figure 5 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0079] The above Figure 5 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0080] The above Figure 5 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0081] The above Figure 5 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0082] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0087] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0088] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0089] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0090] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. An image processing method, characterized in that, include: Receive medical images containing coronary arteries sent by medical imaging equipment; Based on the structural characteristics of coronary arteries, the coronary artery regions in the medical images are identified and marked; Using an adversarial generative network model, lesion sub-regions containing plaques are identified in the coronary vascular region; The coronary artery region and the lesion sub-region are respectively subjected to feature extraction and quantification to obtain the first medical feature and the second medical feature; The first medical feature and the second medical feature are fed into a pre-trained image classification model to obtain the classification result of the medical image; Specifically, feature extraction and quantification are performed on the coronary artery region and the lesion sub-region to obtain a first medical feature and a second medical feature, including: based on pre-selected medical features, feature extraction and quantification are performed on the coronary artery region and the lesion sub-region to obtain a first medical feature and a second medical feature; The method further includes: pre-acquiring multiple sample medical images containing coronary arteries; for each sample medical image, acquiring quantified morphological features and imaging features corresponding to the sample medical image; for any feature in the quantified morphological features and imaging features, calculating a third difference value between a first difference value between the proximal and distal ends of the blood vessel in a first feature set and a second difference value between the proximal and distal ends of the blood vessel in a second feature set; if the third difference value meets the statistical difference condition, selecting the feature as the selected medical feature; wherein the first feature set contains quantified medical features extracted from a first type of sample medical images, and the second feature set contains quantified medical features extracted from a second type of sample medical images. The method further includes: for each sample medical image, obtaining the quantified plaque component features and plaque image features corresponding to the sample medical image; and using a lasso regression analysis method to select key features from the quantified plaque component features and plaque image features as selected medical features.

2. The method according to claim 1, characterized in that, Using a generative adversarial network model, lesion sub-regions containing plaques are identified in the coronary artery region, including: The medical image with the marked coronary artery region is input into the adversarial generative network model, which includes a trained target plaque segmentation model. The target plaque segmentation model is used to segment the marked coronary artery region in the medical image to mark the lesion sub-region containing plaque in the coronary artery region.

3. The method according to claim 2, characterized in that, Also includes: Acquire reference sample images and adversarial sample images containing coronary arteries. The reference sample images are sample images in which plaque sub-regions contained in the coronary artery region have been labeled; the adversarial sample images are sample images in which plaque sub-regions contained in the coronary artery region have not been labeled. The adversarial sample image is segmented using an initial patch segmentation model in an adversarial generative network model to mark the patch sub-regions contained in the adversarial sample image; The adversarial model in the adversarial generative network model is used to perform difference analysis on the segmented sample image output by the initial patch segmentation model and the patch sub-regions in the reference sample image. The difference analysis results are fed back to the initial patch segmentation model so that the initial patch segmentation model can be iteratively updated until the difference analysis results meet the set requirements, and the target patch segmentation model is obtained.

4. The method according to claim 1, characterized in that, Based on a pre-selected medical feature type, features are extracted and quantified from the coronary artery region to obtain a first medical feature under the selected medical feature type, including: The morphological and imaging features of the coronary vessels are extracted from the coronary vessel region, and the morphological and imaging features are quantified to obtain quantified morphological and imaging features. Based on the selected medical features, a first medical feature is selected from the quantified morphological features and imaging features; The first medical feature includes at least one of quantified morphological features and imaging features.

5. The method according to claim 4, characterized in that, The morphological features of the coronary vessels include at least one of the diameter and tortuosity of the coronary vessels; the imaging features of the coronary vessels include at least one of the statistical features and texture features of the coronary vessels.

6. The method according to claim 1, characterized in that, Based on a pre-selected medical feature type, feature extraction and quantification are performed on the lesion sub-region to obtain a second medical feature under the selected medical feature type, including: Plaque component features and plaque image features are extracted from the lesion sub-region, and the plaque component features and plaque image features are quantified to obtain quantified plaque component features and plaque image features; Based on the selected medical features, a second medical feature is selected from the quantified plaque composition features and plaque image features; The second medical feature includes at least one of quantified plaque composition features and plaque imaging features.

7. A medical image processing system, characterized in that, include: Medical imaging equipment, image processing equipment, and medical terminals; The medical imaging device is used to scan a specified object to obtain a medical image containing coronary vessels, and to upload the medical image to an image processing device and a medical terminal. The image processing device is used to perform the method as described in claim 1 and return the classification result to the medical terminal; The medical terminal is used to receive the medical image and the classification result, and to display the classification result and the medical image together on its display screen.

8. The system according to claim 7, characterized in that, The image processing device is a cloud server; The medical imaging device includes a wireless communication module, through which the medical imaging device connects to the Internet and uploads medical images containing coronary vessels to the cloud server and the medical terminal via the Internet; Alternatively, the medical imaging device and the medical terminal are located on the same local area network. The medical imaging device includes a Wi-Fi module, through which the medical imaging device accesses the local area network and uploads medical images containing coronary arteries to the cloud server and the medical terminal via the local area network.

9. An image processing device, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor, coupled to the memory, is configured to execute the computer program for performing the method as described in claim 1.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-6.

Images