Method, system and computer readable storage medium for generating a pathological distribution of medical images

Abstract

The application provides a pathological distribution generation method and system of a medical image and a computer readable storage medium. The pathological distribution generation method comprises the following steps: obtaining a to-be-processed matrix set of image data of a medical image, and preprocessing the to-be-processed matrix set to obtain a region of interest of the medical image and a category label corresponding to the region of interest; standardizing the region of interest, and performing feature extraction on the standardized region of interest based on a classifier to form a feature vector; performing nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates; and drawing a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates. After the above technical solution is adopted, the features between nonlinear two-dimensional matrix data samples are distinguished and similarity measurement is performed, important structural information in the data is retained, and the result of data processing is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a pathological distribution generation method and system of medical images and a computer readable storage medium. BACKGROUND

[0002] In recent years, when dealing with intelligent diagnosis of medical images, a convolutional neural network is mainly used, which aims to convert medical image information into a few diagnostic indicators by using learning theory and statistical ideas, but the deep mechanism is not clear. On the other hand, the convolutional neural network classifier currently used for auxiliary medical diagnosis often only outputs classification probability values, and the rich image information about the image itself and its relationship with other images is missing.

[0003] In high-dimensional visualization technology, the dimension reduction method DRT has better scalability in sample size and input dimension, and has become the preferred tool for exploring data structure. In the past 10 years, T-SNE is one of the most famous and popular techniques, which is a dimension reduction method based on machine learning, which constructs a conditional probability based on the similarity of objects in high-dimensional probability distribution, and reasonably reduces the KL divergence between similar distributions in low dimension for low-dimensional visualization. However, it has speed limitations for large-scale data, and the output results are unstable. In order to overcome these limitations, people have proposed uniform manifold approximation projection (UMAP), which constructs a fuzzy topological representation for high-dimensional and low-dimensional data embedding, and uses cross-entropy optimization for low-dimensional embedding to make its fuzzy topology similar to high-dimensional, which is based on Riemannian geometry and algebraic topology, and has more rigorous theoretical demonstration. This algorithm has no dimension calculation limit and becomes an excellent representative method in general DRT.

[0004] However, for medical images with inter-class similarity and intra-class difference, using general nonlinear dimension reduction methods cannot form an image similarity distribution field with pathological significance, so as to distinguish the nature of the lesions in medical images. Using convolutional neural networks to encode image features and then performing nonlinear dimension reduction is a solution, but its two-stage operation mode does not effectively combine the advantages of convolutional neural networks and dimension reduction methods, but causes information loss and cannot really improve the effect of identifying the nature of the lesions based on medical images by radiologists.

[0005] Therefore, a new visualization auxiliary method and system for medical image processing to obtain pathological distribution is needed to improve the efficiency of automatic identification of lesion nature. SUMMARY

[0006] In order to overcome the above technical defects, the purpose of the present application is to provide a pathological distribution generation method, system and computer readable storage medium of medical images, so as to realize the feature distinction and similarity measurement between non-linear two-dimensional matrix data samples, retain important structure information in the data, and optimize the result of data processing.

[0007] The present application discloses a pathological distribution generation method of medical images, comprising the following steps:

[0008] Obtain a set of to-be-processed matrices of image data of medical images, and pre-process the set of to-be-processed matrices to obtain a region of interest of the medical images and a category label corresponding to the region of interest;

[0009] Standardize the region of interest, and perform feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector;

[0010] Perform non-linear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates;

[0011] Draw a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates.

[0012] Preferably, the step of obtaining a set of to-be-processed matrices of image data of medical images, and pre-processing the set of to-be-processed matrices to obtain a region of interest of the medical images and a category label corresponding to the region of interest comprises:

[0013] Obtain pixel value matrix data corresponding to pixel points in the medical images as the set of to-be-processed matrices, or obtain experimental data belonging to different categories as the set of to-be-processed matrices;

[0014] Pre-process the set of to-be-processed matrices to obtain the region of interest of the medical images;

[0015] Analyze the region of interest, set N classification problems based on pathological properties, and label the region of interest as 0 to N-1 as the category label corresponding to the region of interest.

[0016] Preferably, the step of standardizing the region of interest, and performing feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector comprises:

[0017] Obtain a set of regions of interest formed by the regions of interest, and perform size standardization and pixel normalization processing on the regions of interest of the set of regions of interest based on a preset pixel size;

[0018] Select a convolutional neural network trained by supervised learning using the classification label, input the standardized region of interest into the convolutional neural network, and calculate the feature vector before the last fully connected layer of the convolutional neural network and pathological category probability value p i ;

[0019] For eigenvectors Normalization treatment and the pathology category probability value p i Merge them to form a set of feature vectors.

[0020] Preferably, the step of performing nonlinear dimensionality reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates includes:

[0021] Obtain the feature vector set X = {x1, x2, ..., x} n},x i ∈R M ,in For x i Relative to the metric d CReUMAP The set of points consisting of the k nearest neighbors;

[0022] Construct a Gaussian blur representation of the feature vector set based on the following formula.

[0023]

[0024] Where V = X, E represents the edge set, and ω is the weight matrix corresponding to the point set, and ω is at position {(i,i... h The values ​​of ) on |1≤h≤k,1≤i≤n} are determined according to the following formula:

[0025]

[0026] Where σ i For the normalizer, ρ i The nearest neighbor distance is calculated based on the following formula:

[0027]

[0028]

[0029] Where the metric d CReUMAP Calculated based on the following formula:

[0030]

[0031] Where λ∈[0,1] are hyperparameters. With d probability (p i ,p j ) is a metric function;

[0032] Gaussian blur representation The adjacency matrix of X is defined as A, which is symmetrized based on the following formula:

[0033]

[0034] where T denotes matrix transpose, denotes matrix multiplication, and matrix B is defined as is a high-dimensional representation of X;

[0035] Assume D is D = diag(d1, d2, …, d n ), L is

[0036] Take the first K dimensions of the eigenvectors of L, normalize them, and superimpose noise from a normal distribution to form Y, Y = {y1, y2, …, y n}, use Y as the initialized low-dimensional representation, and assume the graph structure corresponding to Y is G y , B' is G y the adjacency matrix with respect to the Euclidean distance;

[0037] Optimize the low-dimensional representation Y using stochastic gradient descent, and minimize the cross-entropy between the eigenvector set X and the adjacency matrix of the low-dimensional representation Y based on the following formula:

[0038]

[0039] The Y obtained after minimizing the cross-entropy is recorded as the low-dimensional embedding with low-dimensional coordinates.

[0040] Preferably, the method further comprises the following steps:

[0041] Obtaining another batch of medical image regions of interest and class labels corresponding to the regions of interest, processing the regions of interest of the other batch of medical images based on the convolutional neural network to obtain new feature vectors and new pathological class probability values, and merging the new feature vectors and the new pathological class probability values to obtain feature data;

[0042] Based on the existing low-dimensional embedding, performing nonlinear dimensionality reduction on the feature data to generate a new low-dimensional embedding;

[0043] Generating a distribution area based on the new low-dimensional embedding and the class labels, and drawing the distribution area in the pathological distribution field image.

[0044] Preferably, the step of obtaining another batch of medical image regions of interest and class labels corresponding to the regions of interest, processing the regions of interest of the other batch of medical images based on the convolutional neural network to obtain new feature vectors and new pathological class probability values, and merging the new feature vectors and the new pathological class probability values to obtain feature data comprises:

[0045] Obtain the regions of interest (ROIs) and their corresponding category labels from another batch of medical images;

[0046] The region of interest of another batch of medical images is processed using a convolutional neural network to obtain new feature vectors and probability values ​​of new pathological categories.

[0047] The feature data X is obtained by merging the new feature vector and the probability value of the new pathological category. test ;

[0048] Obtain the existing feature vector X and the existing low-dimensional embedding Y.

[0049] Preferably, the step of performing nonlinear dimensionality reduction on the feature data to generate a new low-dimensional embedding based on the existing low-dimensional embedding includes:

[0050] Merge feature data and existing feature vectors X to construct a high-dimensional vector set;

[0051] Select m nearest neighbors of each region of interest in the high-dimensional vector set and connect them with edges to form an edge set E;

[0052] Define d CReUMAP In For the selected metric function, d probability (p i ,p j Select a specific norm; and record the data indices of another batch of regions of interest;

[0053] Spectral embedding is performed on the edge set E using the standard spectral method on the symmetric normalized Laplacian matrix to initialize the low-dimensional representation of another batch of regions of interest, and to initialize the low-dimensional spatial coordinates of the existing eigenvector X.

[0054] Fix the existing low-dimensional embedding Y, and use stochastic gradient descent to optimize the low-dimensional representation of another batch of regions of interest.

[0055] Preferably, the step of generating a distribution region based on the new low-dimensional embedding and category labels, and plotting the distribution region within the pathological distribution field image, further includes:

[0056] When the target dimension is two-dimensional, obtain the existing low-dimensional embedding Y and category label;

[0057] Fit a classification equation and region lines to the existing low-dimensional embedding Y and category labels, and draw the distribution regions separated by adjacent regions with different colors;

[0058] The classification results for another batch of regions of interest are obtained by classifying the low-dimensional representation of another batch of regions of interest based on the region lines.

[0059] This invention also discloses a system for generating pathological distributions in medical images, comprising:

[0060] an image acquisition module, acquiring a to-be-processed matrix set of image data of a medical image, and preprocessing the to-be-processed matrix set to obtain a region of interest of the medical image and a category label corresponding to the region of interest;

[0061] a feature extraction module, normalizing the region of interest, and performing feature extraction on the normalized region of interest based on a classifier to form an n-dimensional feature vector;

[0062] a dimension reduction module, performing nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates;

[0063] a drawing module, drawing a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates.

[0064] The application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the pathological distribution generation method.

[0065] After the above technical scheme is adopted, the following beneficial effects are obtained compared with the prior art:

[0066] 1. The generation method combines the advantages of a convolutional neural network and manifold learning to establish a parameterized CReUMAP algorithm, and makes up for the insufficient speed and accuracy of the prior art in dimension reduction of ultrasound image big data;

[0067] 2. The generation method can use medical image data to obtain a pathological distribution field reflecting different classification distribution trends, and has high adaptability to new medical image sample data;

[0068] 3. The generation method solves the problems of the prior art in an auxiliary diagnosis and treatment scene, such as less output information, non-specific medical image information report, and difficulty in querying and comparing similar images, and realizes automatic processing of medical image preprocessing, feature extraction, distribution field generation, data embedding and interactive query. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 Fig. 1 is a flowchart of a pathological distribution generation method of a medical image according to a preferred embodiment of the application;

[0070] Figure 2a Fig. 2 is a schematic diagram of an original ultrasound image of a malignant thyroid nodule and its annotation box according to a preferred embodiment of the application, and the numbers in the schematic diagram are the malignant probability values output by the convolutional neural network.

[0071] Figure 2bIn order to meet the original ultrasound image of the benign thyroid nodule and the annotation box schematic diagram in a preferred embodiment of the present application, the numbers in the schematic diagram are the malignant probability values output by the convolutional neural network;

[0072] Figure 3 In order to meet the pathological distribution field schematic diagram in a preferred embodiment of the present application;

[0073] Figure 4 In order to meet the flowchart of the pathological distribution generation method of the medical image in a further preferred embodiment of the present application;

[0074] Figure 5 In order to meet the pathological distribution field schematic diagram with distribution lines in a preferred embodiment of the present application. DETAILED DESCRIPTION

[0075] The advantages of the present application will be further described below in combination with the accompanying drawings and specific embodiments.

[0076] Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. In the following description, unless otherwise expressly specified, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments are not representative of all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0077] The terms used in the present disclosure are merely for the purpose of describing particular embodiments and are not intended to limit the present disclosure. As used in the present disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0078] It should be understood that although the terms first, second, third, etc. can be employed in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one piece of information from another piece of information of the same type. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0079] In the description of the present application, it needs to be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0080] In the description of the present application, unless otherwise specified and limited, it needs to be explained that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication inside two elements, it can be direct connection, or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0081] In the subsequent description, the suffix such as "module", "component" or "unit" used to represent elements is only for the convenience of the description of the present application, and has no specific meaning. Therefore, "module" and "component" can be used mixedly.

[0082] Referring to Figure 1 , a flowchart of a pathological distribution generation method of a medical image according to a preferred embodiment of the present application is shown, in which the pathological distribution generation method comprises the following steps:

[0083] S100: obtaining a set of to-be-processed matrices of image data of a medical image, and pre-processing the set of to-be-processed matrices to obtain a region of interest of the medical image and a class label corresponding to the region of interest

[0084] The set of to-be-processed matrices can be some experimental data belonging to different categories, or pixel value matrix data corresponding to pixel points in the image. Exemplarily, in medical data such as ultrasound images, the image data is two-dimensional matrix data composed of processed pixel values. The matrix data in the set of to-be-processed matrices can be a nonlinear matrix.

[0085] S200: standardizing the region of interest, and performing feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector

[0086] The image matrix of the standardized region of interest is input into a convolutional neural network for feature extraction, and a classification probability is output. The convolutional neural network can be a pre-trained classification neural network based on any architecture. The feature vector is formed by the extracted combined features and the probability, and the set of to-be-processed matrices is changed into a set of vectors.

[0087] S300: Non-linear dimensionality reduction on the n-dimensional feature vectors to generate a low-dimensional embedding with low-dimensional coordinates

[0088] After having a set of vectors (feature vectors) to be processed, a high-dimensional fuzzy representation of the set of vectors is constructed. Specifically, an edge weight matrix of the set of vectors is computed based on a probability aided metric, also called CReUMAP metric, from which the high-dimensional fuzzy representation is constructed. Then an initialized low-dimensional representation is made, specifically, a spectral embedding is performed on a symmetric normalized Laplacian matrix using a standard spectral method to initialize the low-dimensional representation. Finally, the low-dimensional representation is optimized using stochastic gradient descent to minimize the cross-entropy between the high-dimensional data X and the adjacency matrix of the low-dimensional embedding Y.

[0089] S400: Drawing a pathology distribution field image based on the low-dimensional embedding with low-dimensional coordinates

[0090] The low-dimensional embedding with low-dimensional coordinates is drawn as an interactive point plot, the interactive point plot is colored using the class labels, and the image information is loaded at each point so that the corresponding medical image can be viewed by clicking, forming a pathology distribution field image.

[0091] In a preferred embodiment, step S100 comprises:

[0092] S110: Obtaining a pixel value matrix data corresponding to the pixel points in the medical image as a set of matrices to be processed, or obtaining experimental data belonging to different categories as a set of matrices to be processed;

[0093] S120: Preprocessing the set of matrices to be processed to obtain a region of interest of the medical image;

[0094] S130: Analyzing the region of interest, setting N kinds of classification problems (such as benign and malignant, etc.) based on pathological properties, and labeling the region of interest as 0 to N-1 as the class label corresponding to the region of interest.

[0095] In the face of medical images, it is often necessary to preprocess them to obtain the region of interest (ROI) and then analyze the ROI. For example, in a complete thyroid ultrasound image, there may be more than one nodule, and the rectangular region containing the corresponding nodule needs to be cut off. The matrix set to be processed needs to be encoded according to the classification category corresponding to the matrix data of the medical image. For example, in the ultrasound thyroid nodule and other oncology data, the pathological benignity and malignancy can be used for encoding, such as encoding the class label of malignant nodules as 1 and the class label of benign nodules as 0. Specifically, assuming that the original medical image set is a matrix set containing 2000 1024x768x3 pixel matrices, after the image is grayed to a matrix set containing 2000 1024x768x1 pixel matrices, considering that it contains 2614 regions of interest, the ROI set is constructed by using the bounding rectangle of the lesion region label and expanding 20 pixels to crop out 2614 regions of interest, and the benign and malignant labels corresponding to the regions of interest are taken to form a 2614-dimensional benign and malignant label vector, wherein each value is 0 or 1, i.e. l=(l1, l2, …, ln), l n ),l i ∈{0,1}, where n=2614.

[0096] Further, step S200 includes:

[0097] S210: obtaining a region of interest set formed by the regions of interest, and performing size standardization and pixel normalization on the regions of interest of the region of interest set based on a preset pixel size;

[0098] In the face of a region of interest set composed of N classes (2 classes in this embodiment, benign and malignant), N is a positive integer not less than 2, and n images (n≥Nx1000) in the preprocessed region of interest set are selected, including more than 1000 ultrasound images of different categories, and the sizes are standardized to 256x256. For example, according to the assumption in step S130, the size of the ROI matrix in the ROI set is standardized to 256x256, and the pixel value is normalized to obtain 2614 256x256 matrices. The benign and malignant labels corresponding to all regions of interest form a 2614-dimensional benign and malignant label vector.

[0099] S220: selecting a convolutional neural network for supervised learning training using a classification label, and inputting the standardized region of interest into the convolutional neural network to calculate the feature vector and the pathological category probability value p i before the last fully connected layer of the convolutional neural network.

[0100] The normalized region of interest is input into a convolutional neural network for feature extraction and output of classification probability. Specifically, a vector output before the last fully connected layer of the classification neural network is selected and a prediction probability vector of N classes p i ∈ [0, 1] N-1 where i ∈ {1,..., n}. The convolutional neural network here can be any architecture that has been trained using supervised learning with classification labels to achieve a certain classification accuracy (e.g. as shown in Figure 2a and 2b In an example, the present embodiment uses AI-SONIC TM The thyroid system software development kit (version 2.3.1.5) extracts ultrasound nodule features, and the software version is 5.3.0.2. The benign and malignant accuracy based on single image statistics on the data set containing 2614 ROIs described in S100 step reaches 86.5%.

[0101] The resulting 2614 256x256 matrices in the region of interest set are input into the benign and malignant diagnosis convolutional neural network. For the i-th matrix, a 2304-dimensional feature vector before the last fully connected layer of the convolutional neural network model is calculated and the last output one-dimensional malignant probability value p i ∈ [0, 1]. The parameters are assumed here, M = 2305, N = 2.

[0102] S230: Normalize the feature vector to and merge with the pathological category probability value p i to form a feature vector set. In an embodiment, p i represents the pathological malignant probability.

[0103] The feature vector is normalized, i.e. the features of each group of ROIs are extracted using the convolutional neural network, and a linear transformation is performed on the extracted features, mapping the (M-N-1) values in the vector to [0, 1], and the normalized is denoted as is used and p i to form the feature vector of the i-th sample. In an example, the obtained in the previous step is normalized and combined with p iThe merged, 2305-dimensional feature vectors constitute the set of feature vectors of the region of interest S100, and the 2614 256x256 matrices become 2614 2305-dimensional feature vectors, denoted as a 2614x2305 feature matrix, each row of which is a feature vector corresponding to a region of interest.

[0104] Further, step S300 includes:

[0105] S310: Obtain a set of feature vectors X={x1, x2, …, xn} based on the following formula: n}, x i ∈R M , where is a set of k nearest neighbors of x i with respect to the metric d CReUMAP .

[0106] For example, according to the assumption in step S200, the 2614x2305 feature vector matrix is obtained, n=2614, M=2305, and the feature vector x i corresponding to the ith row, and all the region of interest samples constitute a set of high-dimensional vectors X.

[0107] S320: Construct a Gaussian blur representation of the set of feature vectors based on the following formula:

[0108]

[0109] where V=X, E represents an edge set, and ω is a weight matrix corresponding to the set of points, and ω is not 0 at positions {(i, i h )|1≤h≤k,1≤i≤n}, and is determined according to the following formula:

[0110]

[0111] where σ i is a normalization sub, and ρ i is a nearest neighbor distance, which is calculated based on the following formula:

[0112]

[0113]

[0114] where the metric d CReUMAP is calculated based on the following formula:

[0115]

[0116] where x i and x jThese are the identifiers of the two feature vectors used to calculate the distance. The p is the standardized feature vector extracted by the convolutional neural network. i Let be the output classification probability vector, and λ∈[0,1] be an adjustable hyperparameter. With d probability (p i ,p j λ is an adjustable metric function; the larger λ is, the more the dimensionality reduction result will depend on the classification probability value of the convolutional neural network, and vice versa.

[0117] S330: Represent Gaussian blur The adjacency matrix is ​​defined as A(n×n dimensional, where the element at position (i,j) is w). ij A is symmetric based on the following formula:

[0118]

[0119] Where T represents matrix transpose. To represent matrix multiplication, matrix B is used as the definition. For a higher-dimensional representation of X;

[0120] For example, following the assumption in step S200, 2614 feature vectors of 2305 dimensions form a high-dimensional vector set X. Assuming k = 10, this means selecting the 10 nearest neighbors of a sample in the region of interest and connecting them with edges to form an edge set. Furthermore, exemplarily assuming metric d CReUMAP In Selected as the correlation coefficient, d probability (p i ,p j The L2 norm is selected, and the specific formula is as follows:

[0121]

[0122] d probability (p i ,p j )=(p i -p j ) 2

[0123] Furthermore, in this embodiment, it is assumed that λ = 0.5.

[0124] S340: Assume D is D = diag(d1, d2, ..., d n ), L is

[0125] S350: Take the first K dimensions of the eigenvectors of L, normalize them, and add normally distributed noise to form Y, Y = {y1, y2, …, y n , Take Y as the initialized low-dimensional representation, and assume that the graph structure corresponding to Y is G y , B' is the adjacency matrix of G y with respect to the Euclidean distance; that is, take the first K dimensions of the eigenvectors of the symmetric normalized Laplacian matrix L, form Y = {y1, y2, …, y n , Y is the obtained spectral embedding, and K is the target dimension of dimension reduction.

[0126] That is, take the first K dimensions of the eigenvectors of L, normalize them by 10, and add normally distributed noise with a variance of 0.0001 and a mean of 0 to form Y = {y1, y2, …, y n , Y is the initialized low-dimensional coordinate. Assume that the graph structure corresponding to Y is G y , B' is the adjacency matrix thereof with respect to the Euclidean distance.

[0127] For example, according to the assumption in step S200, K = 2 and n = 2614.

[0128] S360: Use stochastic gradient descent to optimize the low-dimensional representation Y, and minimize the cross-entropy between the eigenvector set X and the adjacency matrix of the low-dimensional representation Y based on the following formula:

[0129]

[0130] After minimizing the cross-entropy, Y obtained is recorded as a low-dimensional embedding with low-dimensional coordinates.

[0131] For example, according to the assumption in step S200, the set of 2614 thyroid nodule ROI matrices is mapped to a 2614x2 matrix, where each row is the position coordinates of a ROI sample in two dimensions. Such a coordinate graph presents a distribution trend related to pathology.

[0132] Referring to Figure 3 , a distribution trend related to pathology is shown, which is distributed according to the trend from benign pathology to malignant pathology. The technical solution of the embodiment uses a convolutional neural network to extract features and probabilities, and uses a probability auxiliary measurement function to construct a fuzzy set CReUMAP as a dimension reduction means, to reduce the dimension of multi-dimensional matrix data, specifically ultrasound thyroid data. Important structural information related to categories in matrix data can be preserved, so that non-linear data sample features can also be separated and applied to data classification and image diagnosis to more directly show the classification results.

[0133] Referring to Figure 4 In a further embodiment, based on the above-mentioned method of generating a medical image distribution field for a large batch of different categories of medical images, by fixing the existing medical image distribution field and individually optimizing the new medical image, the new data can be embedded in the existing medical image distribution field while preserving the category and similarity, and subject to the classification trend distribution. Specifically, the pathological distribution generation method further comprises the following steps:

[0134] S500: Obtain the region of interest of another batch of medical images and the category label corresponding to the region of interest, process the region of interest of another batch of medical images based on a convolutional neural network to obtain a new feature vector and a new pathological category probability value, and combine the new feature vector and the new pathological category probability value to obtain feature data;

[0135] In other words, the feature set of the existing region of interest sample in the above-mentioned partial embodiment is obtained as a reference embedding, and the step of extracting the feature set is performed on the new region of interest.

[0136] S600: Based on the existing low-dimensional embedding, perform non-linear dimensionality reduction on the feature data to generate a new low-dimensional embedding;

[0137] S700: Generate a distribution region based on the new low-dimensional embedding and the category label, and draw the distribution region in the pathological distribution field image.

[0138] Referring to Figure 5 In an example, the higher the background value, the closer to 1, the greater the probability of malignancy; and the lower the value, the closer to 0, the greater the probability of benign. The black curve is the contour line of the surface at 0.5, and the classification process in S700 classifies all new sample points with higher values relative to this line as malignant, and the new sample points other than this are benign, achieving a benign and malignant classification accuracy of 90.8%, of which 87.1% benign means that the pathological benign samples account for 87.1% in the new samples determined as benign by the S700 method, and 92.7% malignant means that the pathological malignant samples account for 92.7% in the new samples determined as malignant by the S700 method.

[0139] Specifically, step S500 comprises:

[0140] S510: Obtain the region of interest of another batch of medical images and the category label corresponding to the region of interest;

[0141] S520: Process the region of interest of another batch of medical images based on a convolutional neural network to obtain a new feature vector and a new pathological malignant category value;

[0142] S530: Combine the new feature vector and the new pathological category probability value to obtain feature data Xtest ;

[0143] S540: Obtain existing feature vector X and existing low-dimensional embedding Y

[0144] That is, exemplary, in this step, assuming that the new data generated by the region of interest of another batch of medical images is 303 ultrasound thyroid regions of interest, after passing through the same convolutional neural network as described above, the feature vector and the probability of malignancy value are extracted, and they are combined to obtain X test ={testx1,testx2,…,testx 303}, In addition, the feature data X train =X={x1,x2,…,x n}, and its low-dimensional embedding Y train =Y={y1,y2,…,y n}, where n=2614, M=2305, K=2.

[0145] Further, step S600 includes:

[0146] S610: Merge feature data and existing feature vector X to construct a high-dimensional vector set;

[0147] Step 1: Construct a high-dimensional fuzzy representation of the vector set, and record the index corresponding to the new data;

[0148] S620: Select m nearest neighbors of each region of interest in the high-dimensional vector set to form an edge set E;

[0149] S630: Define the metric d CReUMAP in d as a selected metric function (which can be the same as the metric function in the above embodiment), d probability (p i ,p j ) is selected as a specific norm, and the data index of another batch of regions of interest is recorded;

[0150] The specific construction process is the same as the above steps, exemplary, according to the assumption in the foregoing, form a high-dimensional vector set consisting of 2917 2305-dimensional feature vectors, and the first 303 samples of this set are new samples. Assuming k=10, i.e. in the range of all samples, select 10 nearest neighbors of each sample to form an edge set Further, according to the assumption in the foregoing, exemplary, the metric d CReUMAP in d is selected as the correlation coefficient, and dprobability (p i ,p j ) is chosen as the two-norm, and λ = 0.5 is chosen. The indices 1 to 303 of the new data are recorded.

[0151] S640: Perform spectral embedding on the symmetric normalized Laplacian matrix using the standard spectral method on the edge set E to initialize the low-dimensional representation of the other batch of regions of interest and initialize the low-dimensional space coordinates of the existing feature vectors X;

[0152] Specifically, the adjacency matrix is initialized according to the initialization method provided in the previous embodiment, and spectral embedding is performed on the symmetric normalized Laplacian matrix using the standard spectral method to initialize the low-dimensional representation of the new data. For the existing data, the obtained low-dimensional space coordinates of the existing data are used for initialization.

[0153] S650: Fix the existing low-dimensional embedding Y, and use random gradient descent to optimize the low-dimensional representation of the other batch of regions of interest.

[0154] For example, the indices 1 to 303 corresponding to the new data are obtained, and after calculating the gradient using random gradient descent, only the new data is updated and optimized. Assuming that 100 rounds of updating are performed, after 100 rounds of updating of the new data embedding, the embedding coordinates Y test of the new data are obtained. 303 ,

[0155] In a preferred embodiment, step S700 comprises:

[0156] S710: When the target dimension is two-dimensional, obtain the existing low-dimensional embedding Y and the class label;

[0157] Obtain the low-dimensional embedding coordinates Y = {y1, y2,..., y n} of the existing samples, K = 2, and the class labels l = {l1, l2,..., l n} of the existing samples, where the value of l i is an integer from 1 to N {1,..., N} representing which class the ith sample belongs to. For example, the low-dimensional embedding coordinates Y = {y1, y2,..., y train} of 2614 existing samples and the pathological benign and malignant labels l = (l1, l2,..., l n ) are obtained. train l n = (l1, l2,..., l i ), l n×5 ∈ {0, 1}, where n = 2614, 0 is benign, and 1 is malignant.

[0158] ​S720: fitting a classification equation and region line (as shown) to the existing low-dimensional embedding Y and class labels, and drawing the distribution regions separated by the adjacent regions in different colors; Figure 5

[0159] In this step, for each class, construct a quadratic equation that needs to be solved on the k_type class to make:

[0160]

[0161] Where Construct the following linear equation system, and solve the parameters a, b, c, d, e, f using Gaussian elimination method:

[0162] A n×5 v = l, v = (a, b, c, d, e, f),

[0163] Then use the contour of the quadratic equation to divide the space region, and perform accuracy evaluation on the existing low-dimensional embedding coordinates. When the accuracy is the highest, the region of the surface greater than the corresponding threshold is the region of the current class.

[0164] Repeat the above steps N times to obtain the distribution regions of N classes, and use different colors to draw and display.

[0165] For example, when it comes to benign and malignant thyroid nodules, N = 2, only the malignant class needs to be constructed into a surface. Specifically, using the obtained low-dimensional space coordinates and pathological benign and malignant classification, the parameters of the benign and malignant trend surface obtained by solving are: a = 0.006, b = 0.003, c = 0.009, d = -0.165, e = -0.201, f = 1.730;

[0166] It should be noted here that the fitting method is based on existing data fitting partition, and the purpose is to predict new data. When using, the classification label of new data is not required, so new data does not participate in the fitting process.

[0167] S730: Based on the region line, the low-dimensional representation of another batch of regions of interest is classified to obtain the classification results of another batch of regions of interest.

[0168] For example, the low-dimensional embedding coordinates Y of the new sample are obtained test = {testy1, testy2, …, testy 303}, Substitute the calculation ​When considering whether the pathology is malignant, if the calculated value is greater than 0.5, it means that the corresponding sample is diagnosed as malignant, otherwise it is diagnosed as benign. Thus, the accuracy of the classification of new samples based on the fitted classification curve of the existing data can be obtained. In this embodiment, the accuracy of the classification of benign and malignant samples is 90.8%.

[0169] The application further discloses a pathological distribution generation system of a medical image, comprising: an image acquisition module, which acquires a to-be-processed matrix set of image data of a medical image, and pre-processes the to-be-processed matrix set to obtain a region of interest of the medical image and a category label corresponding to the region of interest; a feature extraction module, which standardizes the region of interest, and performs feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector; a dimension reduction module, which performs nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates; and a drawing module, which draws a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates.

[0170] The application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the pathological distribution generation method.

[0171] It should be noted that the embodiments of the application have better implementation, and do not limit the application in any form, and any skilled person in the art can change or modify the equivalent effective embodiments by using the disclosed technical content, as long as the content of the technical scheme of the application is not deviated, and any modification or equivalent change and modification of the above embodiments according to the technical essence of the application are still within the scope of the technical scheme of the application.

Claims

1. A method of generating a pathological distribution of a medical image, characterized by, The method comprises the following steps: obtaining a set of to-be-processed matrices of image data of a medical image, and preprocessing the set of to-be-processed matrices to obtain a region of interest of the medical image and a category label corresponding to the region of interest; standardizing the region of interest, and performing feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector; performing nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates; drawing a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates. The step of performing nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates comprises: a set of eigenvectors of n-dimensional eigenvectors wherein is a point set of k nearest neighbors with respect to a metric ​ A Gaussian blur representation of the set of feature vectors is constructed based on the following equation : ， where V = X, E denotes the edge set, and , is the weight matrix corresponding to the point set, and The value at position is given by the following formula: ， wherein is the normalization sub, is the nearest neighbor distance, calculated based on the following formula: where the metric is calculated based on the following formula: ， wherein is a hyperparameter, with is a metric function; A Gaussian blur representation is defined as A, which is symmetrized based on the following equation: where T denotes matrix transposition, denotes matrix multiplication, and matrix B is used as a definition is a high-dimensional representation of X; Assume D is , L is ; Take the first K dimensions of the eigenvectors of L, normalize them, and add normally distributed noise to form Y, , Use Y as the initialized low-dimensional representation and assume that the graph structure corresponding to Y is , Adjacency matrix with respect to Euclidean distance;​ The low-dimensional representation Y is optimized using stochastic gradient descent and is minimized based on the following equation cross-entropy between the adjacency matrix of the low-dimensional representation Y: minimizing the cross-entropy to obtain Y as the low-dimensional embedding with low-dimensional coordinates.

2. The pathological profile generation method of claim 1, wherein, The step of obtaining a set of to-be-processed matrices of image data of a medical image, and preprocessing the set of to-be-processed matrices to obtain a region of interest of the medical image and a category label corresponding to the region of interest comprises: obtaining pixel value matrix data corresponding to pixel points in the medical image as the set of to-be-processed matrices, or obtaining experimental data belonging to different categories as the set of to-be-processed matrices; preprocessing the set of to-be-processed matrices to obtain the region of interest of the medical image; analyzing the region of interest, setting N classification problems based on pathological properties, and labeling the region of interest as 0 to N-1 as the category label corresponding to the region of interest.

3. The pathological profile generation method of claim 2, wherein, The step of standardizing the region of interest, and performing feature extraction on the standardized region of interest based on a classifier to form an n-dimensional feature vector comprises: obtaining a set of regions of interest formed by the regions of interest, and performing size standardization and pixel normalization processing on the regions of interest of the set of regions of interest based on a preset pixel size; The convolutional neural network using the classification label is selected for supervised learning training, and the standardized region of interest is input into the convolutional neural network, and a feature vector before the last fully connected layer of the convolutional neural network is calculated , and a pathological category probability value . The feature vector is normalized as and merged with the pathological category probability value to form a feature vector set.

4. The pathological profile generation method of claim 1, wherein, The method further comprises the following steps: obtaining a region of interest of another batch of medical images and a category label corresponding to the region of interest, processing the region of interest of the another batch of medical images based on a convolutional neural network to obtain a new feature vector and a new pathological category probability value, and merging the new feature vector and the new pathological category probability value to obtain feature data; performing nonlinear dimension reduction on the feature data based on the existing low-dimensional embedding to generate a new low-dimensional embedding; generating a distribution region based on the new low-dimensional embedding and the category label, and drawing the distribution region in the pathological distribution field image.

5. The pathological profile generation method of claim 4, wherein, The step of obtaining a region of interest of another batch of medical images and a category label corresponding to the region of interest, processing the region of interest of the another batch of medical images based on a convolutional neural network to obtain a new feature vector and a new pathological category probability value, and merging the new feature vector and the new pathological category probability value to obtain feature data comprises: obtaining a region of interest of another batch of medical images and a category label corresponding to the region of interest; processing the region of interest of the another batch of medical images based on a convolutional neural network to obtain a new feature vector and a new pathological category probability value; combining the new feature vector and the new pathological category probability value to obtain feature data ; obtaining an existing feature vector X and an existing low-dimensional embedding Y.

6. The pathological profile generation method of claim 5, wherein, The step of performing nonlinear dimension reduction on the feature data based on the existing low-dimensional embedding to generate a new low-dimensional embedding comprises: merging the feature data and the existing feature vector X to construct a high-dimensional vector set; selecting m nearest neighbors of each region of interest of the high-dimensional vector set to form an edge set E; Definitions in is a selected metric function, selected as a particular norm; and recording data indices of another batch of regions of interest; performing spectral embedding on the edge set E using standard spectral method on the symmetric normalized Laplacian matrix to initialize the low-dimensional representation of another batch of regions of interest and the low-dimensional space coordinates of the existing feature vectors X; fixing the existing low-dimensional embedding Y and optimizing the low-dimensional representation of another batch of regions of interest using stochastic gradient descent.

7. The pathological profile generation method of claim 4, wherein, The step of generating distribution regions based on the new low-dimensional embedding and the class labels and drawing the distribution regions in the pathological distribution field image further comprises: when the target dimension is two-dimensional, obtaining the existing low-dimensional embedding Y and the class labels; fitting a classification equation and a region line to the existing low-dimensional embedding Y and the class labels, and drawing the distribution regions separated by adjacent regions in different colors; classifying the low-dimensional representation of another batch of regions of interest based on the region line to obtain the classification results of another batch of regions of interest.

8. A pathological distribution generation system of a medical image, characterized by, comprises: an image acquisition module, which acquires a set of to-be-processed matrices of image data of a medical image, and pre-processes the set of to-be-processed matrices to obtain a region of interest of the medical image and a class label corresponding to the region of interest; a feature extraction module, which normalizes the region of interest, and extracts features from the normalized region of interest based on a classifier to form an n-dimensional feature vector; a dimension reduction module, which performs nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates; a drawing module, which draws a pathological distribution field image based on the low-dimensional embedding with low-dimensional coordinates. The step of performing nonlinear dimension reduction on the n-dimensional feature vector to generate a low-dimensional embedding with low-dimensional coordinates comprises: a set of eigenvectors of n-dimensional eigenvectors wherein is relative to a point set of k nearest neighbors of the metric A Gaussian blur representation of the set of feature vectors is constructed based on the following equation : ， where V = X, E denotes the edge set, and , is the weight matrix corresponding to the point set, and The value at position is given by the following formula: ， wherein is the normalized sub, is the nearest neighbor distance, calculated based on the following formula: where the metric is calculated based on the following formula: ， wherein is a hyperparameter, with is a metric function; A Gaussian blur representation is defined as A, which is symmetrized based on the following equation: where T denotes matrix transposition, denotes matrix multiplication, and B is used as a definition is a high-dimensional representation of X; Assume D is , L is ; Take the first K dimensions of the eigenvectors of L, normalize them, and add normally distributed noise to form Y, , Use Y as the initialized low-dimensional representation and assume that the graph structure corresponding to Y is , Adjacency matrix with respect to Euclidean distance;​ The low-dimensional representation Y is optimized using stochastic gradient descent and is minimized based on the following equation cross-entropy between the adjacency matrix of the low-dimensional representation Y: Y obtained after minimizing the cross-entropy is recorded as the low-dimensional embedding with low-dimensional coordinates.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the pathological distribution generation method according to any one of claims 1-7.

Images