A multi-sequence image processing method, system and electronic device

Abstract

The present invention discloses a multi-sequence image processing method, belonging to the field of medical image processing, which includes steps such as image acquisition, image preprocessing, target feature extraction, feature screening, and machine learning, to extract more and richer features and improve classification accuracy. The present invention also relates to a multi-sequence image processing system and an electronic device for implementing the above multi-sequence image processing method.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and particularly to a multi-sequence image processing method, system, and electronic device. Background Art

[0002] Multi-sequence images usually refer to images of different sequences showing the same content. Since the information in a single sequence often cannot meet various requirements alone, currently, magnetic resonance imaging (MRI) technology is commonly used in medicine to detect cardiovascular and cerebrovascular diseases. Similar to ordinary multi-sequence images, the images presented by MRI multi-sequence images are the information of the same patient at the same position, only with different imaging parameters and slightly different imaging times before and after.

[0003] When processing existing multi-sequence images, the features are not rich enough, resulting in low classification accuracy. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, one of the purposes of the present invention is to provide a multi-sequence image processing method that can make the features rich and the classification accuracy high after multi-sequence image processing.

[0005] In order to overcome the deficiencies of the prior art, another purpose of the present invention is to provide a multi-sequence image processing system that can make the features rich and the classification accuracy high after multi-sequence image processing.

[0006] In order to overcome the deficiencies of the prior art, the third purpose of the present invention is to provide an electronic device that can make the features rich and the classification accuracy high after multi-sequence image processing.

[0007] One of the purposes of the present invention is achieved by adopting the following technical solutions:

[0008] A multi-sequence image processing method includes the following steps:

[0009] Obtain images: Read images of different sequences in the same lesion;

[0010] Image preprocessing: For the image data of a certain type of sequence or a certain time phase, manually label the target, perform morphological processing on the target area, and find the active area of the tumor edge; perform spatial registration and reconstruction on the remaining images to obtain the labeled images of the remaining images;

[0011] Target feature extraction, including the following steps:

[0012] Extract image features from the target area of each floating image; extract image features from the target dilation area of the floating image; extract image features from the target erosion area of the floating image; extract image features from the target edge area of the floating image;

[0013] Perform multiple filters on each floating image, and each floating image generates multiple filtered images. Extract the features of the floating images and the filtered images;

[0014] Concatenate the features of multiple modal images at the head and tail to form new features;

[0015] Calculate the feature differences of multiple temporal image features to form a collection of difference features, and concatenate the floating image features and the collection of difference features at the head and tail to form new features;

[0016] Feature screening: Screen the extracted features and select the relevant features beneficial to the learning algorithm;

[0017] Machine learning: Input the screened features into machine learning and output the corresponding classification results.

[0018] Furthermore, in the image preprocessing step, perform morphological processing on the target area, and the specific steps to find the active area of the tumor edge are:

[0019] Select the mask shape and size, and perform binary dilation operation on the target area;

[0020] Select the mask shape and size, and perform binary erosion operation on the target area;

[0021] Perform a difference set operation on the binary image.

[0022] Furthermore, in the image preprocessing step, the spatial registration of the remaining images is specifically as follows: Select several marker points in the floating image, and select marker points with corresponding anatomical significance in the reference image; Perform marker point registration on the marker points in the floating image.

[0023] Furthermore, in the image preprocessing step, the reconstruction of the remaining images is specifically as follows: Perform image interpolation on the floating image, and the interpolation method is trilinear interpolation or spline function interpolation; The deformation field of the registered image is T ms ; Use T ms Perform a spatial transformation on the floating image, and the transformed image is the binary marker image.

[0024] Furthermore, in the target feature extraction step, the filtered images include one or more of wavelet images, Log images, square images, square root images, logarithmic images, exponential images, gradient images, and local binary pattern images.

[0025] Furthermore, in the step of extracting the features of the floating images and the filtered images, the features are specifically: one or more of the first-order statistical moment, three-dimensional shape features, gray-level co-occurrence matrix, gray-level run-length matrix, gray-level region size matrix, adjacent gray-level difference matrix, and gray-level correlation matrix.

[0026] Further, in the step of calculating the feature differences of multiple temporal image features to form a difference feature set, specifically: subtracting the feature value in the floating image feature from the feature value in the reference image feature or dividing the feature value in the reference image feature by the corresponding feature value in the floating image feature.

[0027] Further, the reference image feature and the floating image feature have the same vector length.

[0028] The second object of the present invention is achieved by the following technical solution:

[0029] A multi-sequence image processing system for implementing any one of the above multi-sequence image processing methods.

[0030] The third object of the present invention is achieved by the following technical solution:

[0031] An electronic device, comprising

[0032] a processor;

[0033] a memory communicatively connected to the processor;

[0034] The memory stores instructions executable by the processor, and the instructions cause the processor to execute any one of the above multi-sequence image processing methods.

[0035] Compared with the prior art, the multi-sequence image processing method of the present invention extracts floating image features, extracts features of multiple filtered images, concatenates the features of multiple modal images at the head and tail to form new features; calculates the feature differences of multiple temporal image features to form a difference feature set, concatenates the floating image features and the difference feature set at the head and tail to form new features, extracts more and richer features, and improves the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of the multi-sequence image processing method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.

[0038] It should be noted that when a component is referred to as "fixed to" another component, it can be directly on the other component or there may also be another intermediate component through which it is fixed. When a component is considered to be "connected to" another component, it can be directly connected to the other component or there may be another intermediate component at the same time. When a component is considered to be "disposed on" another component, it can be directly disposed on the other component or there may be another intermediate component at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this article are for illustrative purposes only.

[0039] Unless otherwise defined, all technical and scientific terms used in this article have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs. The terms used in the specification of this invention are only for the purpose of describing specific embodiments and are not intended to limit the invention. The term "and / or" used in this article includes any and all combinations of one or more of the related listed items.

[0040] Figure 1 The figure is a flowchart of the multi-sequence image processing method of the present invention. The multi-sequence image processing method includes the following steps:

[0041] Obtain images: Read images of different sequences in the same lesion;

[0042] Image preprocessing: including target annotation, morphological processing of the target area, and annotation of the remaining another modality or another phase image Im'.

[0043] The target annotation is specifically to annotate the target for the image data of a certain type of sequence or a certain phase using the manual annotation method, and the annotated image is denoted as I ml , and the original image is denoted as I ms ; among them, the annotated image is a binary image, and the target area is a non-zero value.

[0044] The morphological processing of the target area is specifically: select the mask shape and size, perform binary dilation operation on the target area, and the dilated binary image is denoted as I mld , to expand the image. Select the mask shape and size, perform binary erosion operation on the target area, and the eroded binary image is denoted as I mle , to shrink the image. Perform a difference set operation on the binary image, and the difference set image is denoted as I ls , and the specific difference set calculation method is: I mld -I ml , I mld -I mle , I ml -I mle Any one of them to find the active area of the tumor edge.

[0045] For the remaining other modality or other phase image I m’ The annotation is specifically as follows: The target is annotated by manual annotation, and the annotated image is denoted as I m’l , and the original image is denoted as I m’s ; among them, the annotated image is a binary image, and the target area is a non-zero value. Or the target is annotated by the fiducial point registration method: Select several fiducial points in I ms , and select the fiducial points with corresponding anatomical significance in I m’s ; perform fiducial point registration on the fiducial points; I m’s is the reference image, and I ms is the floating image. Among I ms and I m’s , the number of fiducial points of each image is not less than 3, and the fiducial point registration method is rigid registration or elastic registration. After registration, perform image interpolation on I ms to reconstruct the spatially registered image, and the reconstructed new image is the same size as the reference image. The interpolation method is trilinear interpolation or spline function interpolation; the deformation field of the registered image is T ms ; use T ms to perform a spatial transformation on I ml , and the transformed image is the binary labeled image of I m’s , denoted as I m’l . The spatial transformation between the two images can transform one or more images to the specified reference image, so that the features of the same annotation area can be extracted in batches, reducing the manual annotation workload.

[0046] Target feature extraction: including image filtering, feature extraction, multi-modal image feature fusion, and multi-phase image feature fusion.

[0047] Specifically, the image filtering is as follows:

[0048] Extract the image features of the target area of I ms , that is, the area of non-zero pixels in I ml , denoted as F msrc ; for subsequent machine learning training (image classification). Extract the image features of the target dilation area of I ms , that is, the area of non-zero pixels in I mld , denoted as F mld . Extract the image features of the target erosion area of I ms , that is, the area of non-zero pixels in I mle , denoted as F me . Extract the image features of the target edge area of I ms , that is, the area of non-zero pixels in I mls , denoted as F msub .

[0049] Perform image filtering on I ms The filtered image includes the wavelet image I wavelet , Log image I log , squared image I square , square root image I squareroot , logarithmic image I logarithm , exponential image I exponential , gradient image I gradient , local binary pattern image I LBP or more of them. Among them, the wavelet basis of the wavelet image I wavelet is one or several of Haar, dmey, symi (i = 2, 3, …, 20), dbi (i = 1, 2, …, 20), coif i (i = 1, 2, …, 20), bior x (x = 1.1, 1.3, 1.5, 2.2, 2.4, 2.6, 2.8, 3.1, 3.3, 3.5, 3.7, 3.9, 4.4, 5.5, 6.8), rbio x (x = 1.1, 1.3, 1.5, 2.2, 2.4, 2.6, 2.8, 3.1, 3.3, 3.5, 3.7, 3.9, 4.4, 5.5, 6.8).

[0050] Feature extraction is specifically as follows: Extract image features from I ms and I wavelet , I log , I square , I squareroot , I logarithm , I exponential , I gradient or more of them. That is, the filtered image generated by each filter and I ms will extract one or more of the following features. Among them, the image features include: first-order statistical moment F fos , calculate three-dimensional shape feature F s3d , gray-level co-occurrence matrix F glcm , gray-level run-length matrix F glrlm , gray-level region size matrix F glszm , adjacent gray-level difference matrix F ngtdm , gray-level correlation matrix F gldm .

[0051] Multi-modal image feature fusion is specifically as follows: Perform the same operations on another modal image I m’s as on I ms . The features of I m’s and I ms are respectively denoted as F m’s and F ms ; For Fm’s and F ms are concatenated head-to-tail, and the concatenated feature is denoted as F merge . For images of multiple modalities, the specific operation steps are the same as the previous step, and the concatenated feature is denoted as F merge .

[0052] Feature fusion of multi-temporal images is specifically as follows: For an image I of another time phase m’s , image filtering and then feature extraction are performed on it in the same way, and its feature is denoted as F m’s ; F m’s and F ms have the same vector length. Calculate the feature difference, that is, ΔF m’s = F m’s - F ms ; The specific method is that the eigenvalue in F m’s is subtracted from the eigenvalue corresponding to F ms . For example: The first-order statistical feature F ms generated by the logarithmic image after filtering of I fos , minus the first-order statistical feature F m’s generated by the logarithmic image after filtering of I fos , that is, it constitutes the interpolation of the first-order statistical moment feature of the logarithmic image in the ΔF m’s feature, denoted as ΔF m’s-Logarithm-fos . ΔF m’s-Logarithm-fos is a subset of ΔF m’s , I m’s , or / and the I log generated by its filtering, or / and the I square generated by its filtering, or / and the I squareroot generated by its filtering, or / and the I logarithm generated by its filtering, or / and the I exponential generated by its filtering, or / and the I exponential generated by its filtering, or / and the I LBP image, one or several features extracted from the above one or several images together constitute the feature set of I m’s . ΔF m’s is a feature set composed of the differences of the eigenvalues with the same feature name after feature extraction from the filtered images generated by F m’s and F ms respectively through the same filter. The subtraction operation in calculating the feature difference can be replaced by a division operation, with the operation object and purpose remaining unchanged. Concatenate the head-to-tail features of F ms and ΔF m’s , and the concatenated feature is denoted as F merge .

[0053] Feature screening: Screen the extracted features and select the relevant features beneficial to the learning algorithm.

[0054] The feature screening specifically adopts one or more of the methods of T-test, correlation analysis, maximum correlation-minimum redundancy, and sequence feature screening.

[0055] The correlation analysis specifically adopts the peason method, spearman method, and kandall method.

[0056] The maximum correlation-minimum redundancy specifically adopts the MIQ and MID methods.

[0057] The sequence feature screening specifically includes the sequential forward selection method, sequential backward selection method, sequential floating backward selection method, and sequential floating forward selection method.

[0058] When multiple methods are adopted, the features extracted by the extraction module are successively passed through the above-mentioned T-test, correlation analysis, maximum correlation-minimum redundancy, and sequence feature screening methods to obtain a series of screened output features.

[0059] Machine learning: The machine learning inputs the screened features and outputs the corresponding classification results.

[0060] Specifically, the classifier adopts one or more of C-SVC, Nu-SVC, multinomial logistic regression, random forest, adaboost, and xgboost.

[0061] This application also relates to a multi-sequence image processing system for implementing the multi-sequence image processing method. The multi-sequence image processing system includes an image preprocessing module, a target feature extraction module, a feature screening module, and a machine learning module. The image preprocessing module is used to implement the image preprocessing step, the target feature extraction module is used to implement the target feature extraction step, the feature screening module is used to implement the feature screening step, and the machine learning module is used to implement the machine learning step.

[0062] This application also relates to an electronic device for implementing the multi-sequence image processing method. The electronic device includes a processor and a memory. The memory is communicatively connected to the processor, and the memory stores instructions executable by the processor. The instructions cause the processor to execute the above multi-sequence image processing method.

[0063] The multi-sequence image processing method of this application extracts more and richer features and improves the classification accuracy by floating image feature extraction, feature extraction of multiple filtered images, forming new features by concatenating the head and tail of the features of multiple modal images; calculating the feature differences of the features of multiple temporal images to form a difference feature set, and forming new features by concatenating the head and tail of the floating image features and the difference feature set.

[0064] The above embodiments merely illustrate several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can be made. These are all equivalent modifications and evolutions made to the above embodiments based on the essential technology of the present invention, and all of these fall within the protection scope of the present invention.

Claims

1. A multi-sequence image processing method, characterized in that, Including the following steps: Obtain images: Read images of different sequences in the same lesion; Image preprocessing: For a certain type of sequence or image data at a certain time phase, manually annotate the targets, perform morphological processing on the target regions, and find the active regions at the tumor margins; select several marker points in the floating image and select marker points with corresponding anatomical significance in the reference image; perform marker point registration on the marker points in the floating image, perform image interpolation on the floating image, and the interpolation method is trilinear interpolation or spline function interpolation; the image deformation field after registration is T ms ; Use T ms to perform a spatial transformation on the floating image, and the transformed image is the binary marker image, obtaining the marker image of the remaining image; Target feature extraction, including the following steps: Extract image features from the target region of each floating image; extract image features from the target dilated region of the floating image; extract image features from the target eroded region of the floating image; extract image features from the target edge region of the floating image; Perform multiple filters on each floating image, each floating image generates multiple filtered images, and extract the features of the floating image and the filtered images; Concatenate the features of multiple modal images head-to-tail to form new features; Calculate the feature differences of multiple temporal image features to form a difference feature set, subtract the corresponding feature value in the floating image feature from the feature value in the reference image feature or divide the feature value in the reference image feature by the corresponding feature value in the floating image feature, the reference image feature and the floating image feature have the same vector length, and concatenate the floating image feature and the difference feature set head-to-tail to form new features; Feature screening: Screen the extracted features and select the relevant features beneficial to the learning algorithm; Machine learning: Input the screened features into machine learning and output the corresponding classification results.

2. The multi-sequence image processing method according to claim 1, characterized in that: In the image preprocessing step, perform morphological processing on the target region and find the specific steps of the tumor edge active region as follows: Select the mask shape and size and perform a binary dilation operation on the target region; Select the mask shape and size and perform a binary erosion operation on the target region; Perform a difference set operation on the binary image.

3. The multi-sequence image processing method according to claim 1, wherein: In the target feature extraction step, the filtered images include one or more of wavelet images, Log images, square images, square root images, logarithmic images, exponential images, gradient images, and local binary pattern images.

4. The multi-sequence image processing method according to claim 1, characterized in that: In the step of extracting the features of the floating image and the filtered images, the features are specifically one or more of the first-order statistical moment, three-dimensional shape features, gray-level co-occurrence matrix, gray-level run-length matrix, gray-level region size matrix, adjacent gray-level difference matrix, and gray-level correlation matrix.

5. A multi-sequence image processing system, characterized in that: The multi-sequence image processing system is used to implement the multi-sequence image processing method according to any one of claims 1-4.

6. An electronic device, characterized in that: Including A processor; A memory, the memory is communicatively connected to the processor; The memory stores instructions executable by the processor, and the instructions are executed by the processor to implement the multi-sequence image processing method according to any one of claims 1-4.

Images