Image processing system based on tumor imaging omics

Abstract

The present invention relates to the technical field of image fusion, and in particular to an image processing system based on tumor imaging omics. By improving the existing medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering, after determining a fusion layer with better representation effect of graphic information and lesion information after the first fusion, the fusion weights of image fusion of different feature image layers are adaptively determined according to the image feature performance of the fusion layer and the lesion feature performance, so as to perform adaptive image fusion based on the adaptively obtained fusion weights, thereby solving the problem of poor universality and flexibility of the medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering, so that the universality, flexibility and accuracy of the final fused image are better.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and in particular to an image processing system based on tumor imaging omics. Background Art

[0002] In the field of medical teaching, learning about case diagnosis of tumors is very important; and radiomics is an auxiliary technology used to extract large amounts of perfectly reproducible information from images. It converts high-dimensional image information into mathematical data and uses objective and quantitative methods to analyze it. It has become a common method for analyzing medical image characteristics. Therefore, radiomics is also often used in the field of medical teaching.

[0003] Tumor images usually include a variety of single-modality medical images such as CT, MRT, and PET. However, single-modality medical images usually have certain limitations in the judgment of tumor cases. Therefore, in order to improve the teaching effect of tumor case judgment in medical education, multi-modality medical image fusion is often required to more comprehensively present the pathological condition.

[0004] In the prior art, a multi-scale characteristic frequency domain decomposition filtering algorithm is used to fuse medical images of different modalities through an image processing system. However, since the algorithm is only applicable to the studied parts and image types, the universality and flexibility of the algorithm are poor, resulting in poor fusion effect when multi-modal tumor examination images are fused using the prior art, which leads to poor teaching effect when the fused tumor examination images are used in medical teaching. Summary of the invention

[0005] The present application provides an image processing system based on tumor imaging omics. The system improves the existing medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering, determines the fusion weights for image fusion of different feature image layers, and then performs adaptive image fusion based on the adaptively obtained fusion weights, thereby solving the problem of poor universality and flexibility of the medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering, so that the universality, flexibility and accuracy of the final fused image are better.

[0006] The present application provides an image processing system based on tumor imaging omics, comprising:

[0007] A data acquisition module is used to obtain all image layers of the grayscale images of each modality obtained when the patient undergoes tumor examination each time; based on a feature extraction method, the grayscale images of each image feature and the feature values ​​corresponding to each image layer are obtained; wherein the types of image features include global features and regional features;

[0008] The first determination module is used to determine the global feature importance of each global feature to each image layer according to the consistency of the feature values ​​between each image layer and the corresponding inspection grayscale image and the prominence of the feature values ​​of each image layer compared with other image layers under each global feature; and determine the image representation strength of each image layer in the inspection grayscale image under each modality according to the overall size of the global feature importance of various global features corresponding to each image layer;

[0009] The second determination module is used to determine the regional feature importance of each regional feature to each image layer according to the change of the feature value of each image layer during each inspection process and the tumor index matching of the corresponding regional feature under each regional feature; and determine the lesion expression intensity of each image layer in the inspection grayscale image under each modality according to the overall size of the regional feature importance of various regional features corresponding to each image layer;

[0010] The third determination module is used to perform pixel-level fusion according to the grayscale expression of each pixel in the inspection grayscale image under various modalities and the image expression intensity and lesion expression intensity of each pixel in each image layer, and determine the fusion layer of each image layer; according to the image expression intensity and lesion expression intensity of each fusion layer, determine the fusion weight of each fusion layer;

[0011] The image fusion module is used to perform image fusion according to the fusion weights combined with multi-scale feature frequency domain decomposition filtering to determine a final fused image.

[0012] Furthermore, the process of obtaining the importance of the global features includes:

[0013] Each image layer in the inspection grayscale image under each modality is sequentially used as a target layer; other image layers other than the target layer in the inspection grayscale image under each modality are used as reference layers for the target layer;

[0014] Under each global feature, the difference between the feature value of the target layer and the feature value of each reference layer is used as the reference prominence of each reference layer; the accumulated values ​​of the reference prominence of all reference layers are normalized to determine the feature value prominence of the target layer under each global feature;

[0015] The ratio between the occurrence frequency of each global feature in all image layers in the inspection grayscale image of each modality and the number of image layers is taken as the feature frequency ratio of each global feature;

[0016] Under each global feature, the difference between the feature value corresponding to the inspection grayscale image under each modality and the feature value corresponding to the target layer is taken as the image deviation degree of the target layer under each global feature;

[0017] A negative correlation mapping is performed on the product of the feature frequency ratio and the image deviation degree to determine the consistency of the feature values ​​of the target layer under each global feature;

[0018] According to the eigenvalue prominence and the eigenvalue consistency, the global feature importance of the target layer under each global feature is determined; wherein the eigenvalue prominence and the eigenvalue consistency are both positively correlated with the global feature importance.

[0019] Furthermore, the process of determining the importance of the global features of the target layer under each global feature according to the prominence degree of the feature value and the consistency degree of the feature value includes:

[0020] The global feature importance of the target layer under each global feature is determined according to the product of the feature value prominence degree and the feature value consistency degree.

[0021] Furthermore, the process of obtaining the image performance intensity includes:

[0022] The cumulative value of the global feature importance of each image layer under all types of global features is used as the image representation strength of each image layer.

[0023] Furthermore, the process of obtaining the importance of regional features includes:

[0024] Determine all key indicators of the tumor based on prior clinical experience, and sort all key indicators in descending order of importance to obtain a key indicator sequence; match all regional features with all key indicators through a feature matching method, and use the negative correlation mapping value of the index value of the key indicator matched to each regional feature in the key indicator sequence as the feature importance of each regional feature;

[0025] Under each regional feature, the difference between the feature value of each image layer at each inspection and the feature value at the previous inspection is used as the feature change value of each image layer at each inspection under each regional feature; the mean of the feature change values ​​of each image layer at all inspections under each regional feature is normalized to determine the feature change degree of each image layer under each regional feature;

[0026] According to the feature importance and the feature variation degree, the regional feature importance of the target layer under each regional feature is determined; wherein the feature importance and the feature variation degree are both positively correlated with the regional feature importance.

[0027] Furthermore, the process of determining the importance of regional features of target layers under each regional feature according to the importance of the features and the degree of change of the features includes:

[0028] The importance of the regional features of each image layer under each regional feature is determined according to the product of the feature importance and the feature variation degree.

[0029] Furthermore, the process of obtaining the lesion manifestation intensity includes:

[0030] The mean value of the importance of regional features of each image layer under all types of regional features is taken as the lesion expression intensity of each image layer.

[0031] Furthermore, the acquisition process of the fusion layer includes:

[0032] Each pixel position in the inspection grayscale image is sequentially used as the target position; the image layer after all pixels in the column and row where the target position is located in each image layer of the inspection grayscale image under each modality are removed is used as the reference layer corresponding to the target position under each modality;

[0033] Under each modality, the image performance intensity difference between each image layer of the grayscale image and the corresponding reference layer is examined as the image performance change value corresponding to the target position in each image layer under each modality;

[0034] Under each modality, the difference in lesion manifestation intensity between each image layer of the inspection grayscale image and the corresponding reference layer is used as the lesion manifestation change value of the target position in each image layer under each modality; the sum of the image manifestation change value and the lesion manifestation change value is used as the position importance of the target position in each image layer under each modality;

[0035] Among all the position importances corresponding to the target position in each image layer under all modalities, the grayscale value of the target position in each image layer of the inspection grayscale image under the modality corresponding to the position importance with the largest value is taken as the final grayscale value of the target position in each image layer; the final grayscale values ​​of all pixel positions in each image layer are counted; and the fusion layer corresponding to each image layer is determined according to the final grayscale values ​​of all pixel positions in each image layer.

[0036] Furthermore, the process of obtaining the fusion weight includes:

[0037] The lesion expression intensity and image expression intensity of each fusion layer are calculated; the normalized value of the sum of the lesion expression intensity and the image expression intensity of each fusion layer is used as the fusion weight of each fusion layer.

[0038] Furthermore, the process of obtaining the final fused image includes:

[0039] The product of the grayscale value of each pixel position in each fusion layer and the corresponding fusion weight is used as the weighted grayscale value of each pixel position in each fusion layer; the cumulative sum of the weighted grayscale values ​​of each pixel position in all fusion layers is used as the fusion grayscale value of each pixel position; the final fusion image is determined based on the fusion grayscale values ​​of all pixel positions in the inspection grayscale image.

[0040] This application has the following beneficial effects:

[0041] For each global feature, the more consistent the features between a certain image layer under the global feature and the inspection grayscale image, the more the corresponding image layer under the global feature can restore the characteristic features in the image, that is, the higher the importance of the global feature to the corresponding image layer; at the same time, if the importance of the global feature in the corresponding image layer is high but there is no obvious performance in other image layers, the stronger the specificity of the global feature in the image layer, the higher the importance of the global feature to the corresponding image layer; therefore, the present application combines the above-mentioned characteristics to calculate the image expression intensity of each image layer in the inspection grayscale image under each modality; so that the greater the image expression intensity, the stronger the global feature representation of the original image by the corresponding image layer.

[0042] For each regional feature, if a certain image layer under the regional feature changes in each examination, it means that the corresponding image layer under the regional feature is more prominent in the representation of the disease, that is, the importance of the regional feature to the corresponding image layer is higher; at the same time, if the tumor indicator matched by the regional feature is more important, the degree of attention to the regional feature should be higher, and the corresponding importance should also be higher; therefore, the present application combines the above characteristics to calculate the lesion expression intensity of each image layer in the inspection grayscale image under each modality; so that the greater the lesion expression intensity, the stronger the lesion expression of the corresponding image layer to the regional feature.

[0043] Image fusion in multi-scale special frequency domain decomposition filtering is usually divided into two fusion processes, the first of which is pixel-level fusion, which is the process of fusing the same layers of multimodal medical images; and the same pixel in the same image layer usually has different grayscale values ​​in different modalities. The more important the corresponding pixel is in different modalities, the more reliable the grayscale performance corresponding to the pixel is, and the corresponding pixel-level fusion weight is larger. Therefore, this application first determines the fused layer corresponding to each image layer based on the principle of pixel-level fusion. Through pixel-level fusion, the fused layer can provide multimodal graphic information and lesion information, so that the image fusion effect is better.

[0044] Further, on the basis of the fusion layer obtained by pixel-level fusion, combined with the characteristics that the greater the image expression intensity, the stronger the global feature representation of the original image, and the greater the lesion expression intensity, the stronger the lesion expression of the regional feature, the fusion weight is calculated according to the image expression intensity and the lesion expression intensity corresponding to the fusion layer, and further adaptive image fusion is performed according to the fusion weight combined with the multi-scale feature frequency domain decomposition filtering, so that the information represented by the final fused image is more complete and accurate, while improving the universality and flexibility of the multi-scale feature frequency domain decomposition filtering algorithm, so that the image fusion effect is better, thereby improving the teaching effect when the final fused image is applied to medical teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A structural block diagram of an image processing system based on tumor imaging omics provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a multimodal medical image sample provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, characteristics and effects of an image processing system based on tumor imaging genomics proposed according to the present invention, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment, and the specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0050] The following is a detailed description of a specific solution of an image processing system based on tumor imaging omics provided by the present invention in conjunction with the accompanying drawings.

[0051] The present application embodiment provides an image processing system based on tumor imaging genomics, please refer to Figure 1 , which shows a structural block diagram of an image processing system based on tumor imaging omics provided by an embodiment of the present invention, the system includes a data acquisition module 101, a first determination module 102, a second determination module 103, a third determination module 104 and an image fusion module 105.

[0052] The data acquisition module 101 is used to obtain all image layers of the grayscale images of each modality obtained when the patient undergoes tumor examination each time; based on the feature extraction method, the grayscale images of each image feature and the feature values ​​corresponding to each image layer are obtained; wherein the image feature types include global features and regional features.

[0053] In a specific implementation of the embodiment of the present invention, the modality types include: Computed Tomography (CT) modality, Magnetic Resonance Imaging (MRI) modality, Positron Emission Tomography (PET) modality and Single Photon Emission Computed Tomography (SPECT) modality, which correspond to CT modality inspection grayscale image, MRI modality inspection grayscale image, PET modality inspection grayscale image and SPECT modality inspection grayscale image respectively. Please refer to Figure 2 , which shows a schematic diagram of a multimodal medical image sample provided by an embodiment of the present invention, Figure 2 Medical image samples of four modalities, CT, MRI, PET and SPECT, are given as references.

[0054] Further, the inspection grayscale image obtained in each modality during each tumor examination of the patient is decomposed into various image layers through a multi-scale feature frequency domain decomposition filtering algorithm; in a specific implementation of an embodiment of the present invention, the inspection grayscale image is decomposed into a smooth layer, a texture layer, a contour layer and an edge layer through a multi-scale feature frequency domain decomposition filtering algorithm, that is, all image layers in the embodiment of the present invention include a smooth layer, a texture layer, a contour layer and an edge layer. It should be noted that the method of decomposing an image into a smooth layer, a texture layer, a contour layer and an edge layer through a multi-scale feature frequency domain decomposition filtering algorithm is a technical means in a medical image fusion algorithm based on a multi-scale feature frequency domain decomposition filtering, and is not further limited or elaborated herein.

[0055] In a specific implementation of an embodiment of the present invention, the global features in the image feature types are specifically set to: the mean grayscale value of the entire image, the variance of the grayscale value of the entire image, the cumulative sum of the connected domain area of ​​the image, and the sum of the elements of the grayscale co-occurrence matrix of the entire image; further, the inspection grayscale image and the corresponding regions of interest under each image layer are obtained through semantic segmentation, and the regional features in the image feature types are set to: the mean grayscale value of the region of interest, the variance of the grayscale value of the region of interest, the cumulative sum of the connected domain area of ​​the region of interest, and the sum of the elements of the grayscale co-occurrence matrix of the region of interest; the implementer can choose image features of other feature types according to the specific implementation environment, such as global statistical features such as the mean, kurtosis, asymmetry and clarity of the histogram, shape features such as area and compactness, and texture features such as the sum of grayscale run matrix elements, which will not be further elaborated here; that is, the global features are the features of the inspection grayscale image and each image layer as a whole, and the regional features are the features of all regions of interest corresponding to the inspection grayscale image and each image layer.

[0056] The first determination module 102 is used to determine the global feature importance of each global feature to each image layer under each global feature according to the consistency of the feature values ​​between each image layer and the corresponding inspection grayscale image and the prominence of the feature values ​​of each image layer compared with other image layers; and determine the image expression intensity of each image layer in the inspection grayscale image under each modality according to the overall size of the global feature importance of various global features corresponding to each image layer.

[0057] For each global feature, the more consistent the features between a certain image layer under the global feature and the inspection grayscale image, the more the corresponding image layer under the global feature can restore the characteristic features in the image, that is, the higher the importance of the global feature to the corresponding image layer; at the same time, if the importance of the global feature in the corresponding image layer is high but there is no obvious performance in other image layers, the stronger the specificity of the global feature in the image layer, the higher the importance of the global feature to the corresponding image layer; therefore, the present application combines the above-mentioned characteristics to calculate the image expression intensity of each image layer in the inspection grayscale image under each modality; so that the greater the image expression intensity, the stronger the global feature representation of the original image by the corresponding image layer.

[0058] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the importance of global features includes:

[0059] Each image layer in the inspection grayscale image under each modality is taken as the target layer in turn; other image layers other than the target layer in the inspection grayscale image under each modality are taken as the reference layer of the target layer. By analyzing the target layer, the analysis process of all image layers can be characterized at the same time.

[0060] Under each global feature, the difference between the eigenvalue of the target layer and the eigenvalue of each reference layer is used as the reference prominence of each reference layer; the accumulated values ​​of the reference prominence of all reference layers are normalized to determine the prominence of the eigenvalue of the target layer under each global feature. Under the same global feature, the more discrete the eigenvalue of the target layer is compared with the characteristics of all reference layers, the more prominent the characteristics of the target layer are under the corresponding global feature, that is, the greater the prominence of the eigenvalue, the stronger the specificity of the corresponding global feature to the target layer, and the higher the corresponding importance. In a specific implementation of an embodiment of the present invention, the process of obtaining the prominence of the eigenvalue under each mode at the time of the last inspection is expressed by the formula: ;in, is the first Under the global feature The prominence of the feature values ​​of each image layer; is the first Under the global feature The feature values ​​of each image layer; The total number of layers for the image, i.e. The number of reference layers corresponding to each image layer; is the first Under the global feature The image layer corresponds to the The characteristic value of the reference layer; is the first Under the global feature The image layer corresponds to the The reference prominence of each reference layer; It is a linear normalization function. The implementer can adjust the normalization method according to the specific implementation environment.

[0061] The ratio between the frequency of occurrence of each global feature in all image layers in the inspection grayscale image under each modality and the number of image layers is taken as the feature frequency ratio of each global feature; under each global feature, the difference between the feature value corresponding to the inspection grayscale image under each modality and the feature value corresponding to the target layer is taken as the image deviation degree of the target layer under each global feature; the product between the feature frequency ratio and the image deviation degree is negatively correlated to determine the consistency degree of the feature value of the target layer under each global feature. For each global feature, if it exists in the target layer but the frequency of occurrence in other image layers is small, the stronger the specificity of the global feature to the target layer, the higher the importance. And under the corresponding global feature, the closer the feature value of the target layer is to the feature value of the inspection grayscale image, that is, the smaller the image deviation degree, the more the target layer can restore the original features of the image under the corresponding global feature, the stronger the corresponding image performance is, and the higher the importance is; therefore, the greater the consistency degree of the feature value obtained by combining the feature frequency ratio and the image deviation degree, the greater the importance of the corresponding global feature to the target layer. It should be noted that in a specific implementation method of an embodiment of the present invention, when the grayscale value variance, grayscale value mean, image overall connected domain area accumulation value or grayscale co-occurrence matrix element sum of any image layer is 0, it can be considered that the corresponding feature does not appear.

[0062] In a specific implementation of the embodiment of the present invention, the process of obtaining the degree of consistency of the characteristic values ​​in each mode at the last inspection is expressed by the formula: ;in, is the first Under the global feature The consistency of the feature values ​​of each image layer; For the The frequency of occurrence of the global features in all image layers in the inspected grayscale images in each modality; The total number of layers for the image; is the first The frequency ratio of the global features; is the first Under the global feature The feature values ​​of each image layer; is the first Check the feature values ​​of grayscale images under the global features; is the absolute value symbol; is the first Under the global feature The degree of image deviation for each image layer; is an exponential function with a natural constant as the base. Implementers can adopt other negative correlation mapping methods according to the specific implementation environment, such as , and the reciprocal, where is a hyperbolic tangent function, which will not be further described here.

[0063] Further, according to the eigenvalue prominence and eigenvalue consistency, the global feature importance of the target layer under each global feature is comprehensively determined. Since the greater the eigenvalue prominence and the greater the eigenvalue consistency, the greater the importance of the corresponding global feature to the target layer, the eigenvalue prominence and eigenvalue consistency are positively correlated with the global feature importance.

[0064] Preferably, in some possible implementations of the embodiments of the present invention, the process of determining the importance of global features of target layers under each global feature according to the prominence of the feature values ​​and the consistency of the feature values ​​includes:

[0065] According to the product between the eigenvalue prominence and the eigenvalue consistency, the global feature importance of the target layer under each global feature is determined. In a specific implementation of the embodiment of the present invention, the process of obtaining the global feature importance under each mode at the last inspection is expressed by the formula: ;in, is the first The global feature pair Global feature importance at each image level; is the first Under the global feature The prominence of the feature values ​​of each image layer; is the first Under the global feature The consistency of the feature values ​​of the image layers.

[0066] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the image representation intensity includes:

[0067] The cumulative value of the importance of global features of each image layer under all types of global features is used as the image representation strength of each image layer. For each image layer, the greater the overall importance of the image layer under all global features, the stronger the global feature representation of the original image in each feature of the image layer. In a specific implementation of an embodiment of the present invention, the process of obtaining the image representation strength under each modality at the last inspection is expressed by the formula: ;in, The last check for each mode The image representation strength of each image layer; is the number of types of global features; is the first The global feature pair It should be noted that the image expression strength of the image layers used in the subsequent analysis is the image expression strength calculated at the last examination, and no further elaboration will be given later.

[0068] The second determination module 103 is used to determine the regional feature importance of each regional feature to each image layer under each regional feature according to the change of the characteristic value of each image layer during each inspection process and the tumor index matching of the corresponding regional feature; and determine the lesion expression intensity of each image layer in the inspection grayscale image under each modality according to the overall size of the regional feature importance of various regional features corresponding to each image layer.

[0069] For each regional feature, if a certain image layer under the regional feature changes in each examination, it means that the corresponding image layer under the regional feature is more prominent in the representation of the disease, that is, the importance of the regional feature to the corresponding image layer is higher; at the same time, if the tumor indicator matched by the regional feature is more important, the degree of attention to the regional feature should be higher, and the corresponding importance should also be higher; therefore, the present application combines the above characteristics to calculate the lesion expression intensity of each image layer in the inspection grayscale image under each modality; so that the greater the lesion expression intensity, the stronger the lesion expression of the corresponding image layer to the regional feature.

[0070] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the importance of regional features includes:

[0071] According to the prior clinical experience, all the key indicators of the tumor are determined, and all the key indicators are sorted in order from large to small according to the importance to obtain a key indicator sequence; all regional features are matched with all the key indicators by a feature matching method, and the negative correlation mapping value of the index value of the key indicator matched to each regional feature in the key indicator sequence is used as the feature importance of each regional feature. In a specific implementation of an embodiment of the present invention, all the key indicators of the tumor are determined according to the prior clinical experience and are sorted in order from large to small according to the importance as tumor size, tumor edge clarity, tumor shape, and tumor density. The importance and key indicator types can be adjusted according to the specific real-time environment, and no further details are given here. Further, the open source pyradiomics software package is used to match all regional features with all key indicators, so that each regional feature can determine the corresponding key indicator. For each regional feature, the smaller the index value of the corresponding key indicator, the closer the key indicator is in the key indicator sequence, and the greater the corresponding importance; therefore, the smaller the index value of the key indicator corresponding to the regional feature in the key indicator sequence, the higher the corresponding feature importance. It should be noted that the feature importance is an importance weight determined based on prior clinical experience. The feature importance of all regional features can also be set to the same value according to the specific implementation environment, and the feature is not introduced to calculate the regional feature importance. No further elaboration will be given here.

[0072] Under each regional feature, the difference between the feature value of each image layer at each inspection and the feature value at the previous inspection is used as the feature change value of each image layer at each inspection under each regional feature; the mean of the feature change values ​​of each image layer at all inspections under each regional feature is normalized to determine the feature change degree of each image layer under each regional feature. The larger the feature change value, the greater the feature change that occurs at the corresponding number of inspections. Therefore, the greater the feature change degree obtained by the mean of the feature change values ​​at all inspections, the higher the importance of the corresponding regional feature image layer, that is, the corresponding image layer under this regional feature can better characterize the lesion manifestation.

[0073] Further, according to the feature importance and feature variation, the regional feature importance of the target stratification under each regional feature is determined; since the greater the feature importance and the greater the feature variation, the higher the importance of the corresponding regional feature to the image stratification, so the feature importance and feature variation are positively correlated with the regional feature importance.

[0074] Preferably, in some possible implementations of the embodiments of the present invention, the process of determining the importance of regional features of target layers under each regional feature according to the feature importance and the feature variation degree includes:

[0075] According to the product of the feature importance and the feature change degree, the regional feature importance of each image layer under each regional feature is determined. Regional characteristics The process of obtaining the importance of regional features of each image layer is expressed as follows: ;in, is the first The regional characteristics of Importance of regional features in each image layer; For the The index value of the key indicator matched by the regional feature in the key indicator sequence; For the The importance of the characteristics of the regional characteristics; The total number of tumor examinations for the patient; For the The first check in each mode The regional characteristics of The feature values ​​of each image layer; For the The first check in each mode The regional characteristics of The feature values ​​of each image layer; For the The first check in each mode The regional characteristics of The feature change value of each image layer; is the first The regional characteristics of The degree of feature change of each image layer; is an exponential function with a natural constant as the base. Implementers can adopt other negative correlation mapping methods according to the specific implementation environment, such as , and the reciprocal, where is the hyperbolic tangent function, which will not be further described here; It is a linear normalization function, and the implementer can adjust the normalization method according to the specific implementation environment. It should be noted that when the total number of inspections is one, the importance of the regional feature is set to a preset value of 0.5, which can be adjusted according to the specific implementation environment, and will not be further described here.

[0076] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the lesion manifestation intensity includes:

[0077] The mean of the importance of regional features of each image layer under all types of regional features is used as the lesion expression intensity of each image layer. For each image layer, the greater the overall importance of the image layer under all regional features, the stronger the regional feature characterization of the lesion expression of the image layer on each regional feature. In a specific implementation of an embodiment of the present invention, the acquisition process of the lesion expression intensity under each modality at the last inspection is expressed by the formula: ;in, The last check for each mode The intensity of lesion expression in each image layer; is the number of types of regional characteristics; is the first The regional characteristics of It should be noted that the lesion expression intensity of the image layer used in the subsequent analysis is the lesion expression intensity calculated at the last examination, and no further elaboration will be given later.

[0078] The third determination module 104 is used to perform pixel-level fusion according to the grayscale expression of each pixel in the inspection grayscale image under various modalities and the image expression intensity and lesion expression intensity of each pixel in each image layer, and determine the fusion layer of each image layer; according to the image expression intensity and lesion expression intensity of each fusion layer, determine the fusion weight of each fusion layer.

[0079] Image fusion in multi-scale special frequency domain decomposition filtering is usually divided into two fusion processes, the first of which is pixel-level fusion, which is the process of fusing the same layers of multimodal medical images; and the same pixel in the same image layer usually has different grayscale values ​​in different modalities. The more important the corresponding pixel is in different modalities, the more reliable the grayscale performance corresponding to the pixel is, and the corresponding pixel-level fusion weight is larger. Therefore, this application first determines the fused layer corresponding to each image layer based on the principle of pixel-level fusion. Through pixel-level fusion, the fused layer can provide multimodal graphic information and lesion information, so that the image fusion effect is better.

[0080] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the fusion layer includes:

[0081] Each pixel position in the inspection grayscale image is taken as the target position in turn; the image layer after removing all pixels in the column and row of the target position in each image layer of the inspection grayscale image under each modality is taken as the reference layer corresponding to the target position under each modality; under each modality, the image performance intensity difference between each image layer of the inspection grayscale image and the corresponding reference layer is taken as the image performance change value corresponding to the target position in each image layer under each modality; under each modality, the lesion performance intensity difference between each image layer of the inspection grayscale image and the corresponding reference layer is taken as the lesion performance change value of the target position in each image layer under each modality; the sum of the image performance change value and the lesion performance change value is taken as the position importance of the target position in each image layer under each modality. It should be noted that the image layer in the process of obtaining the image performance intensity and lesion performance intensity of the image layer is replaced with the corresponding reference layer, and the image performance intensity and lesion performance intensity corresponding to each reference layer can be calculated, which will not be further elaborated here.

[0082] In each image layer, if the change in the image expression intensity and the lesion expression intensity after all the pixels in the row and column where the target position is located are removed is greater, it means that the target position in the corresponding image layer can better present the image graphic expression and lesion expression under the corresponding modality, that is, the importance of the pixel of the target position in the corresponding image layer under the corresponding modality is greater, that is, among all the modalities under the image layer, the grayscale value of the target position in the modality with greater importance is more likely to belong to the fused modality value. Therefore, further, among all the position importances corresponding to the target position in each image layer under all modalities, the grayscale value of the target position in each image layer of the inspection grayscale image under the modality corresponding to the position importance with the largest value is taken as the final grayscale value of the target position in each image layer; the final grayscale values ​​of all pixel positions in each image layer are counted; and the fusion layer corresponding to each image layer is determined according to the final grayscale values ​​of all pixel positions in each image layer. That is, each pixel position in each image layer corresponds to a position importance in each modality, and the first fusion, that is, pixel-level fusion, is the fusion of modalities. Therefore, in order to make the graphic performance and lesion performance provided by the fused image layer as good as possible, for each pixel position in each image layer, the grayscale value at the corresponding pixel position in the modality corresponding to the maximum position importance is selected as the grayscale value of each pixel position in the fused layer after fusion, so that the obtained fused layer can provide more effective graphic information and lesion information.

[0083] Further, on the basis of the fusion layer obtained by pixel-level fusion, combined with the characteristics that the greater the image expression intensity, the stronger the global feature representation of the original image, and the greater the lesion expression intensity, the stronger the lesion expression of the regional feature, the fusion weight is calculated according to the image expression intensity and the lesion expression intensity corresponding to the fusion layer, and further adaptive image fusion is performed according to the fusion weight combined with the multi-scale feature frequency domain decomposition filtering, so that the information represented by the final fused image is more complete and accurate, while improving the universality and flexibility of the multi-scale feature frequency domain decomposition filtering algorithm, so that the image fusion effect is better, thereby improving the teaching effect when the final fused image is applied to medical teaching.

[0084] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the fusion weight includes:

[0085] Calculate the lesion expression intensity and image expression intensity of each fusion layer; take the normalized value of the sum of the lesion expression intensity and image expression intensity of each fusion layer as the fusion weight of each fusion layer. It should be noted that the image layer in the process of acquiring the image expression intensity and lesion expression intensity of the image layer is replaced with the corresponding fusion layer, and the image expression intensity and lesion expression intensity corresponding to each fusion layer can be calculated. For each fusion layer, the larger the corresponding lesion expression intensity and image expression intensity, the more the fusion layer can image the graphic expression of the original image features and the lesion expression of the lesion characterization, so the larger the fusion weight is given, the better the fusion effect. It should be noted that in order to make the subsequent fusion image more accurate, the sum of the fusion weights of all fusion layers in the normalized result is limited to 1. In a specific implementation method of an embodiment of the present invention, the acquisition process of the fusion weight is expressed by the formula: ;in, The last check corresponds to The fusion weights of the fusion layers; The last check corresponds to The image expression strength of each fused layer; The last check corresponds to The intensity of lesion expression in each fusion layer; is the total number of fusion layers; The last check corresponds to The image expression strength of each fused layer; The last check corresponds to The intensity of lesion expression in each fusion layer; Used for Normalization is performed so that the sum of the fusion weights of all fusion layers is 1.

[0086] The image fusion module 105 is used to perform image fusion according to the fusion weights combined with multi-scale feature frequency domain decomposition filtering to determine the final fused image.

[0087] After obtaining the fusion weight of each fusion layer, further performing final image fusion on all fusion layers based on the fusion weight; preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the final fused image includes:

[0088] The product of the gray value of each pixel position in each fusion layer and the corresponding fusion weight is used as the weighted gray value of each pixel position in each fusion layer; the cumulative sum of the weighted gray values ​​of each pixel position in all fusion layers is used as the fusion gray value of each pixel position; the final fusion image is determined based on the fusion gray values ​​of all pixel positions in the inspection gray image. In a specific implementation of the embodiment of the present invention, the process of obtaining the fusion gray value is expressed by the formula: ;in, For the The fused gray value of the pixel position; is the total number of fusion layers; The last check corresponds to The fusion weights of the fusion layers; The last time the inspection was In the fusion layer The gray value of the pixel position; The last time the inspection was In the fusion layer The weighted gray value of the pixel position.

[0089] That is, each pixel position has a corresponding grayscale value in each fusion layer. The larger the fusion weight of the corresponding fusion layer, the greater the reference value of the grayscale value of the corresponding pixel. Therefore, the grayscale value in each fusion layer is weighted with the fusion weight as the weight, and the grayscale value participating in the fusion corresponding to each fusion layer at each pixel position can be obtained, that is, the weighted grayscale value; further, the weighted grayscale values ​​in all fusion layers at each pixel position are accumulated to obtain the fusion grayscale value after fusion at each pixel position; since the cumulative sum of all fusion weights is 1, the corresponding calculation process does not affect the dimension of the fusion grayscale value, so that the final fusion image obtained is more accurate, and the corresponding fusion effect is better, thereby improving the teaching effect of the final fusion image when applied to medical teaching. After obtaining the final fusion image, the final fusion image obtained is uploaded to the database for medical teaching. It should be noted that the final fusion image in the embodiment of the present invention is the fusion image corresponding to the last examination. Since the present application fuses the image in real time, the image is uploaded once after each examination, so the final fusion image of each examination can be obtained, which will not be further described here.

[0090] In summary, an image processing system based on tumor imaging omics improves the existing medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering. After determining the fusion layer with better representation effect on graphic information and lesion information after the first fusion, the fusion weights of different feature image layers for image fusion are adaptively determined according to the image feature performance of the fusion layer and the lesion feature performance, so as to perform adaptive image fusion based on the adaptively obtained fusion weights, which solves the problem of poor universality and flexibility of the medical image fusion algorithm based on multi-scale feature frequency domain decomposition filtering, so that the universality, flexibility and accuracy of the final fused image are better.

[0091] In the embodiments provided in the present application, it should be understood that the provided computer device, computer program product and computer-readable storage medium are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the methods provided above and will not be repeated here.

[0092] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An image processing system based on tumor imaging, characterized in that: The system comprises: A data acquisition module is used to obtain all image layers of the grayscale images of each modality obtained when the patient undergoes tumor examination each time; based on a feature extraction method, the grayscale images of each image feature and the feature values ​​corresponding to each image layer are obtained; wherein the types of image features include global features and regional features; The first determination module is used to determine the global feature importance of each global feature to each image layer according to the consistency of the feature values ​​between each image layer and the corresponding inspection grayscale image and the prominence of the feature values ​​of each image layer compared with other image layers under each global feature; and determine the image representation strength of each image layer in the inspection grayscale image under each modality according to the overall size of the global feature importance of various global features corresponding to each image layer; The second determination module is used to determine the regional feature importance of each regional feature to each image layer according to the change of the feature value of each image layer during each inspection process and the tumor index matching of the corresponding regional feature under each regional feature; and determine the lesion expression intensity of each image layer in the inspection grayscale image under each modality according to the overall size of the regional feature importance of various regional features corresponding to each image layer; The third determination module is used to perform pixel-level fusion according to the grayscale expression of each pixel in the inspection grayscale image under various modalities and the image expression intensity and lesion expression intensity of each pixel in each image layer, and determine the fusion layer of each image layer; according to the image expression intensity and lesion expression intensity of each fusion layer, determine the fusion weight of each fusion layer; The acquisition process of the fusion layer includes: Each pixel position in the inspection grayscale image is sequentially used as the target position; the image layer after all pixels in the column and row where the target position is located in each image layer of the inspection grayscale image under each modality are removed is used as the reference layer corresponding to the target position under each modality; Under each modality, the image performance intensity difference between each image layer of the grayscale image and the corresponding reference layer is examined as the image performance change value corresponding to the target position in each image layer under each modality; Under each modality, the difference in lesion manifestation intensity between each image layer of the inspection grayscale image and the corresponding reference layer is used as the lesion manifestation change value of the target position in each image layer under each modality; the sum of the image manifestation change value and the lesion manifestation change value is used as the position importance of the target position in each image layer under each modality; Among all the position importances corresponding to the target position in each image layer under all modalities, the grayscale value of the target position in each image layer of the inspection grayscale image under the modality corresponding to the position importance with the largest value is taken as the final grayscale value of the target position in each image layer; the final grayscale values ​​of all pixel positions in each image layer are counted; according to the final grayscale values ​​of all pixel positions in each image layer, the fusion layer corresponding to each image layer is determined; The image fusion module is used to perform image fusion according to the fusion weights combined with multi-scale feature frequency domain decomposition filtering to determine a final fused image.

2. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the importance of the global features includes: Each image layer in the inspection grayscale image under each modality is sequentially used as a target layer; other image layers other than the target layer in the inspection grayscale image under each modality are used as reference layers for the target layer; Under each global feature, the difference between the feature value of the target layer and the feature value of each reference layer is used as the reference prominence of each reference layer; the accumulated values ​​of the reference prominence of all reference layers are normalized to determine the feature value prominence of the target layer under each global feature; The ratio between the occurrence frequency of each global feature in all image layers in the inspection grayscale image of each modality and the number of image layers is taken as the feature frequency ratio of each global feature; Under each global feature, the difference between the feature value corresponding to the inspection grayscale image under each modality and the feature value corresponding to the target layer is taken as the image deviation degree of the target layer under each global feature; A negative correlation mapping is performed on the product of the feature frequency ratio and the image deviation degree to determine the consistency of the feature values ​​of the target layer under each global feature; According to the eigenvalue prominence and the eigenvalue consistency, the global feature importance of the target layer under each global feature is determined; wherein the eigenvalue prominence and the eigenvalue consistency are both positively correlated with the global feature importance.

3. The image processing system based on tumor imaging omics according to claim 2, characterized in that: The process of determining the importance of the global features of the target layer under each global feature according to the prominence degree of the feature value and the consistency degree of the feature value includes: The global feature importance of the target layer under each global feature is determined according to the product of the feature value prominence degree and the feature value consistency degree.

4. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the image performance intensity includes: The cumulative value of the global feature importance of each image layer under all types of global features is used as the image representation strength of each image layer.

5. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the importance of regional features includes: Determine all key indicators of the tumor based on prior clinical experience, and sort all key indicators in descending order of importance to obtain a key indicator sequence; match all regional features with all key indicators through a feature matching method, and use the negative correlation mapping value of the index value of the key indicator matched to each regional feature in the key indicator sequence as the feature importance of each regional feature; Under each regional feature, the difference between the feature value of each image layer at each inspection and the feature value at the previous inspection is used as the feature change value of each image layer at each inspection under each regional feature; the mean of the feature change values ​​of each image layer at all inspections under each regional feature is normalized to determine the feature change degree of each image layer under each regional feature; According to the feature importance and the feature variation degree, the regional feature importance of the target layer under each regional feature is determined; wherein the feature importance and the feature variation degree are both positively correlated with the regional feature importance.

6. The image processing system based on tumor imaging omics according to claim 5, characterized in that: The process of determining the importance of regional features of target stratification under each regional feature according to the importance of the features and the degree of change of the features includes: The importance of the regional features of each image layer under each regional feature is determined according to the product of the feature importance and the feature variation degree.

7. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the lesion manifestation intensity includes: The mean value of the importance of regional features of each image layer under all types of regional features is taken as the lesion expression intensity of each image layer.

8. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the fusion weight includes: The lesion expression intensity and image expression intensity of each fusion layer are calculated; the normalized value of the sum of the lesion expression intensity and the image expression intensity of each fusion layer is used as the fusion weight of each fusion layer.

9. The image processing system based on tumor imaging omics according to claim 1, characterized in that: The process of obtaining the final fused image includes: The product of the grayscale value of each pixel position in each fusion layer and the corresponding fusion weight is used as the weighted grayscale value of each pixel position in each fusion layer; the cumulative sum of the weighted grayscale values ​​of each pixel position in all fusion layers is used as the fusion grayscale value of each pixel position; the final fusion image is determined based on the fusion grayscale values ​​of all pixel positions in the inspection grayscale image.

Images