Benign tumor development trend assessment system, computer device and storage medium

Abstract

The present application discloses a benign tumor development trend assessment system, a computer device and a storage medium. The benign tumor development trend assessment system includes an image output device and a servo computer device. The image output device outputs the first and second images extracted at the same position of the benign tumor. The servo computer device includes: an image receiving module receives the first and second images; an image pre-processing module performs image pre-processing on the first and second images to obtain the first and second partial images; a target object extraction module automatically detects the first and second partial images and circles the tumor position to obtain the first and second region of interest images; a feature extraction module automatically identifies the first and second region of interest images to obtain at least one first and at least one second feature; a trend analysis module analyzes at least one first and at least one second feature to obtain a tumor development trend analysis result.

Description

Technical Field

[0001] The present invention relates to a benign tumor development trend assessment system, its servo computer device and computer-readable storage medium, and in particular to a system, servo computer device and computer-readable storage medium for assessing the development trend of a benign tumor before or after radiation treatment of the benign tumor. Background Art

[0002] Although benign tumors are confined to the site of occurrence and will not metastasize or spread, they will still proliferate and grow, thereby compressing surrounding tissues, and have different effects on patients depending on the site of occurrence. For example, benign tumors of the nervous system such as vestibular schwannoma (VS, also known as "acoustic neuroma") originate from Schwann cells of the vestibulocochlear nerve and are the most common schwannomas. Although VS is a benign tumor, it may still cause damage to the vestibular nerve function, and may cause risks of hearing loss, tinnitus, dizziness, facial nerve paralysis, etc. due to tumor progression or side effects of treatment.

[0003] There are many different radiotherapy methods available for the treatment of benign tumors, such as Gamma Knife radiosurgery (GKRS). Although GKRS is an effective method for the treatment of VS, with a long-term tumor control rate of over 90% and a low risk of complications, some problems may still arise for some patients. One is the risk of treatment failure in some patients; the other is that some patients may experience temporary tumor growth due to tumor swelling after GKRS, which is called pseudoprogression. The aforementioned temporary pseudoprogression can only be distinguished from actual tumor progression by the time interval. Pseudoprogression of tumors usually occurs within 6 to 18 months after radiotherapy, while actual tumor progression can be diagnosed at least 2 years after radiotherapy. The above two problems can only be observed after radiotherapy, so they are likely to cause psychological pressure on patients and medical staff, and it is currently impossible to provide different clinical recommendations based on different patients to improve the effectiveness of treatment.

[0004] In addition, generally speaking, before performing GKRS, experienced surgeons must first manually circle the outline of the tumor area based on the tumor image to identify the tumor location, measure the tumor volume, and then determine the treatment area and treatment dose. This method is not only more subjective but also more time-consuming.

[0005] In summary, the currently known benign tumor treatment method requires surgeons to manually circle the tumor image before radiotherapy to confirm the treatment area and dose. This method is not only subjective and time-consuming, but also cannot predict the patient's response to radiotherapy and provide clinical treatment recommendations other than radiotherapy for different patients. If the patient's tumor image can be automatically evaluated for the response to radiotherapy before radiotherapy, and then the patient is recommended to undergo radiotherapy or other treatments (such as surgery) based on the evaluation results, it will be of great help to the treatment of benign tumors.

[0006] Therefore, there is still an urgent need to provide an evaluation system that can automatically evaluate the patient's future tumor development trend based on tumor images, and provide different suggestions to medical staff based on the evaluation results, so as to provide optimal treatment methods for different patients, thereby improving the treatment effect, so as to avoid patients' doubts due to poor treatment results and cause psychological pressure on medical staff. Summary of the invention

[0007] The purpose of the present invention is to provide an evaluation system that automatically evaluates the future tumor development trend of patients based on tumor images. It can provide different suggestions to medical staff based on the evaluation results, so as to provide preferred treatment methods for different patients, thereby improving the treatment effect of benign tumors. In addition, it can also be used to predict whether the patient is likely to have pseudo-progression of the tumor, so as to inform the patient in advance, which helps to reduce the patient's doubts and the pressure on medical staff.

[0008] To achieve the above-mentioned purpose, the present invention provides a benign tumor development trend assessment system, including an image output device and a servo computer device. The image output device is used to output a first image and a second image extracted at the same position of a benign tumor. The servo computer device includes an image receiving module, an image pre-processing module, a target extraction module, a feature extraction module and a trend analysis module. The image receiving module receives the first image and the second image. The image pre-processing module performs image pre-processing on the first image and the second image respectively to obtain a first local image and a second local image respectively. The target extraction module automatically detects the first local image and the second local image and circles the location of the benign tumor respectively to obtain a first region of interest (ROI) image and a second region of interest image respectively. The feature extraction module automatically identifies the first region of interest image and the second region of interest image respectively to obtain at least one first feature and at least one second feature respectively. The trend analysis module analyzes at least one first feature and at least one second feature to obtain a tumor development trend analysis result for determining the tumor development trend.

[0009] In one embodiment, the image pre-processing module includes an image adjustment unit, an image correction unit, and a local image extraction unit. The image adjustment unit adjusts the brightness and clarity of the first image and the second image respectively and removes the background to obtain the first adjusted image and the second adjusted image. The image correction unit corrects the position of the second adjusted image with the first adjusted image to obtain the first corrected image and the second corrected image. The local image extraction unit extracts local images from the first corrected image and the second corrected image respectively to obtain the first local image and the second local image, wherein the first local image and the second local image cover images of benign tumors and tissues near benign tumors.

[0010] In one embodiment, the first image and the second image are images before treatment or images after treatment. When the first image and the second image are images before treatment, the tumor development trend analysis result is a prediction that the benign tumor will not respond to radiation treatment, or a prediction that the benign tumor will respond to radiation treatment; when the first image and the second image are images after treatment, the tumor development trend analysis result is a judgment that the benign tumor will not respond to radiation treatment, or a judgment that the benign tumor will respond to radiation treatment.

[0011] In one embodiment, when the tumor development trend analysis result is that the benign tumor responds to radiation treatment, it means that the benign tumor responds to radiation treatment but has pseudoprogression, or the benign tumor responds to radiation treatment without pseudoprogression.

[0012] In one embodiment, the server computer device further includes a judgment module, which generates a judgment result according to the tumor development trend analysis result.

[0013] In one embodiment, when the first image and the second image are images before treatment, and the tumor development trend analysis result predicts that the benign tumor will not respond to radiation treatment, the judgment result is to perform other treatments; when the first image and the second image are images before treatment, and the tumor development trend analysis result predicts that the benign tumor will respond to radiation treatment, the judgment result is to recommend radiation treatment; when the first image and the second image are images after treatment, and the tumor development trend analysis result determines that the benign tumor will not respond to radiation treatment, the judgment result is to recommend other treatments; and when the first image and the second image are images after treatment, and the tumor development trend analysis result determines that the benign tumor will respond to radiation treatment, the judgment result is to recommend continued tracking.

[0014] In one embodiment, the trend analysis module stores a plurality of trend paths, and the trend analysis module analyzes these trend paths with the aid of at least one first feature and at least one second feature, wherein these trend paths are obtained by analyzing a plurality of reference images by the trend analysis module.

[0015] In one embodiment, the server computer device further includes a judgment result output module to output the judgment result.

[0016] In one embodiment, the server computer device further includes a processor, which executes an image receiving module, an image pre-processing module, a target object extraction module, a feature extraction module, a trend analysis module, a judgment module, and a judgment result output module.

[0017] In one embodiment, the benign tumor development trend assessment system further comprises a user computer device for receiving the determination result output by the server computer device.

[0018] In one embodiment, the first image and the second image have different contrasts, at least one first feature comprises a texture feature or a grayscale feature, and at least one second feature comprises a texture feature or a grayscale feature.

[0019] In one embodiment, the benign tumor is an acoustic neuroma, a meningioma, a pituitary tumor, a neurocytoma, or a craniopharyngioma.

[0020] To achieve the above-mentioned purpose, the present invention further provides a servo computer device, which is applied to a benign tumor development trend assessment system. The benign tumor development trend assessment system includes an image output device and a servo computer device. The servo computer device includes an image receiving module, an image pre-processing module, a target extraction module, a feature extraction module and a trend analysis module. The image receiving module receives a first image and a second image extracted at the same position of a benign tumor. The image pre-processing module performs image pre-processing on the first image and the second image respectively to obtain a first partial image and a second partial image respectively. The target extraction module automatically detects the first partial image and the second partial image and circles the position of the benign tumor respectively to obtain a first region of interest image and a second region of interest image respectively. The feature extraction module automatically identifies the first region of interest image and the second region of interest image respectively to obtain at least one first feature and at least one second feature respectively. The trend analysis module analyzes at least one first feature and at least one second feature to obtain a tumor development trend analysis result for judging the tumor development trend.

[0021] In one embodiment, the server computer device further includes a judgment module, which generates a judgment result according to the tumor development trend analysis result.

[0022] In one embodiment, the image pre-processing module includes an image adjustment unit, an image correction unit, and a local image extraction unit. The image adjustment unit adjusts the brightness and clarity of the first image and the second image respectively and removes the background to obtain the first adjusted image and the second adjusted image. The image correction unit corrects the position of the second adjusted image with the first adjusted image to obtain the first corrected image and the second corrected image. The local image extraction unit extracts local images from the first corrected image and the second corrected image respectively to obtain the first local image and the second local image, wherein the first local image and the second local image cover images of benign tumors and tissues near benign tumors.

[0023] In one embodiment, the server computer device further includes a processor, and the processor executes the image receiving module, the image pre-processing module, the target object extraction module, the feature extraction module, the trend analysis module and the judgment module.

[0024] To achieve the above objectives, the present invention further provides a computer-readable storage medium, which is applied to the aforementioned benign tumor development trend assessment system. The storage medium stores the aforementioned modules, which can be executed by a processor of a computer device.

[0025] In summary, the tumor development trend assessment system of the present invention can automatically assess the patient's future tumor development trend based on tumor images, and can provide different suggestions to medical staff based on the assessment results, so as to provide preferred treatment methods for different patients, thereby improving the treatment effect of benign tumors. In addition, it can also be used to assess whether the patient is likely to have pseudo-progression of the tumor, so as to inform the patient in advance, which is more helpful to reduce the patient's doubts and the pressure of medical staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1A It is a schematic diagram of a preferred embodiment of the benign tumor development trend assessment system of the present invention.

[0027] Figure 1B for Figure 1A Schematic diagram of the image pre-processing module shown.

[0028] Figure 2 FIG. 1 is a schematic diagram of a server computer device of the benign tumor development trend assessment system of the present invention.

[0029] Figure 3 It is a schematic diagram of the implementation process of the benign tumor development trend assessment system of the present invention.

[0030] FIG. 4A to FIG. 4E Schematic diagram of a tumor image processed by the benign tumor development trend assessment system of the present invention. Figure 4A A schematic diagram showing a first image and a second image; Figure 4BA schematic diagram showing a first adjustment image and a second adjustment image; Figure 4C A schematic diagram showing a first calibrated image and a second calibrated image; Figure 4D A schematic diagram showing a first partial image and a second partial image; Figure 4E A schematic diagram showing a first region of interest image and a second region of interest image.

[0031] FIG. 5A to FIG. 5C It is a schematic diagram of the benign tumor development trend assessment system of the present invention using tumor characteristics to obtain tumor development trend analysis results, wherein the tumor development trend analysis results are that the tumor is unresponsive to radiation treatment or that the tumor is responsive to radiation treatment. Figure 5A 3D distribution maps showing tumor features from multiple reference images; Figure 5B Is Figure 5A The prediction model score graph of the tumor feature distribution transformation; Figure 5C It is a receiver operating characteristic curve graph used to assess whether a tumor is non-responsive to radiation therapy or responsive to radiation therapy.

[0032] FIG. 6A to FIG. 6C The diagram is a schematic diagram of the benign tumor development trend assessment system of the present invention using tumor characteristics to obtain tumor development trend analysis results, wherein the tumor development trend analysis results are that the tumor responds to radiation treatment but has pseudoprogression, or the tumor responds to radiation treatment and has no pseudoprogression. Fig. 6A 3D distribution maps showing tumor features from multiple reference images; Figure 6B Is Fig. 6A The prediction model score graph of the tumor feature distribution transformation; Figure 6C Receiver operating characteristic curves for evaluating tumor response to radiation therapy with pseudoprogression or tumor response to radiation therapy without pseudoprogression. DETAILED DESCRIPTION

[0033] The following will describe preferred embodiments of the benign tumor development trend assessment system, the server computer device, and the computer-readable storage medium according to the present invention with reference to the relevant drawings, wherein the same elements will be described with the same reference numerals.

[0034] The tumor development trend assessment system of the present invention can automatically assess the patient's future tumor development trend based on tumor images. It can provide different suggestions to medical staff based on the assessment results, so as to provide preferred treatment methods for different patients, thereby improving the treatment effect of benign tumors. In addition, it can also be used to assess whether the patient is likely to have pseudo-progression of the tumor, so as to inform the patient in advance, which is more helpful to reduce the patient's doubts and the pressure of medical staff.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those of ordinary skill in the art to which the present invention belongs. Although any methods and materials similar or equivalent to those described herein may be used in the test experiments of the present invention, preferred materials and methods are described herein. When describing and claiming the present invention, the following terms will be used. It should be understood that the purpose of the terms used herein is only to describe specific embodiments, rather than limitations of the present invention.

[0036] The term "benign tumor" refers to a tumor that is confined to the site of occurrence and does not metastasize or spread, such as but not limited to acoustic neuroma, meningioma, pituitary tumor, neuroblastoma, or craniopharyngioma. In this article, the term "benign tumor" may also be referred to as "tumor".

[0037] The term "tumor progression" refers to the continued growth and enlargement of a tumor (continued enlargement).

[0038] The term "pseudoprogression" refers to the fact that after treatment or manipulation, the tumor will first swell, resulting in transient tumor growth, after which the tumor will gradually shrink and become smaller (get bigger and then shrink again), which is different from tumor progression (continued enlargement).

[0039] As used herein, the term "radiation therapy" refers to radiosurgery using radiation, such as but not limited to Gamma Knife radiosurgery, Cyber ​​Knife radiosurgery, Photon Knife radiosurgery, or other radiation therapy methods known to those skilled in the art for treating benign tumors.

[0040] The term "U-Net neural network" also known as "3D U-Net neural network" or "3D single-path U-Net neural network" refers to a neural network derived from a convolutional neural network (CNN), particularly used for the analysis of medical images. For examples of the use of U-Net neural networks for analyzing medical images, please refer to Wang G et al., Automatic Segmentation of Vestibular Schwannoma from T2-Weighted MRI by Deep Spatial Attention with Hardness-Weighted Loss.arXiv preprint arXiv:190603906.2019 and Ronneberger O et al., Convolutional networks for biomedical imagesegmentation.International Conference on Medical image computing and computer-assisted intervention:Springer; 2015.p.234-41. The U-Net neural network consists of 23 layers of convolution and 4 pooling. In the U-Net neural network, in the same convolutional layer, only kernels of the same size are used for medical image segmentation, such as but not limited to 3X 3X 3.

[0041] The term "3D dual-path U-Net neural network" refers to a neural network that is an improvement of the "U-Net neural network". In this embodiment, it is composed of 10 layers of convolution and pooled twice. In the 3D dual-path U-Net neural network, in the same convolution layer, two kernels of different sizes are used to segment medical images, such as but not limited to 3X 3X 1 and 1X 1X 3. The planar part of the image (along the xy axis of the image) can be analyzed by 3X 3X 1, and the three-dimensional part of the image (along the z axis of the image) can be analyzed by 1X 1X 3. Therefore, compared with the "U-Net neural network", the analysis of medical images can be more accurate.

[0042] The terms "first" and "second" are used herein for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features.

[0043] Please refer to Figure 1A , Figure 1AFIG. 1 is a schematic diagram of a preferred embodiment of a benign tumor development trend assessment system of the present invention. In this embodiment, a benign tumor development trend assessment system 100 includes an image output device 1 and a server computer device 2 .

[0044] Please also refer to Figure 1A and Figure 4A , FIG. 4A to FIG. 4E Schematic diagram of a tumor image processed by the benign tumor development trend assessment system of the present invention. Figure 4A Schematic diagram showing a first image and a second image. The image output device 1 is used to output a first image I11 and a second image I21 extracted at the same position of a benign tumor. For example, the benign tumor is, for example but not limited to, an acoustic neuroma, a meningioma, a pituitary tumor, a neuroblastoma, or a craniopharyngioma. The first image I11 and the second image I21 are Magnetic Resonance Imaging (MRI) images, and the contrast of the first image I11 and the second image I21 is different. In other words, the first image I11 and the second image I21 are images obtained by performing imaging with different parameters using a MRI instrument at the same position of a benign tumor, such as, but not limited to, T1-weighted (T1-weighted, T1W), T2-weighted (T2-weighted, T2W), or T1-weighted gadolinium contrast enhanced (T1W+C) images. For example, the first image I11 and the second image I21 can be T1W images, T2W images, or T1W+C images, but they are different. That is, when the first image I11 is a T1W image, the second image I21 can be a T1W+C image or a T2W image; when the first image I11 is a T2W image, the second image I21 can be a T1W image or a T1W+C image; when the first image I11 is a T1W+C image, the second image I21 can be a T1W image or a T2W image. Preferably, the first image I11 is a T1W+C image, and the second image I21 is a T2W image.

[0045] like Figure 1A As shown, in this embodiment, the server computer device 2 includes an image receiving module 21 , an image pre-processing module 22 , a target object extraction module 23 , a feature extraction module 24 and a trend analysis module 25 .

[0046] Please also refer to Figure 1A and FIG. 4A to FIG. 4E , used to explain the role of each module. Figure 4B A schematic diagram showing a first adjustment image and a second adjustment image; Figure 4C A schematic diagram showing a first calibrated image and a second calibrated image; Figure 4DA schematic diagram showing a first partial image and a second partial image; Figure 4E A schematic diagram showing a first region of interest image and a second region of interest image.

[0047] like Figure 1A and Figure 4A As shown, the image receiving module 21 is used to receive the first image I11 and the second image I21 outputted by the image output device 1 .

[0048] like Figure 1A , Figure 1B , FIG. 4A to FIG. 4D As shown, Figure 1B for Figure 1A The image pre-processing module 22 performs image pre-processing on the first image I11 and the second image I21 to obtain a first partial image I14 and a second partial image I24. Figure 1A and Figure 1B , the image pre-processing module 22 may include an image adjustment unit 221, an image correction unit 222, and a local image extraction unit 223. The image adjustment unit 221 adjusts the brightness and clarity of the first image I11 and the second image I21 and removes the background to obtain the first adjusted image I12 and the second adjusted image I22. The image correction unit 222 corrects the position of the second adjusted image I22 with the first adjusted image I12 to obtain the first corrected image I13 and the second corrected image I23. The local image extraction unit 223 extracts local images from the first corrected image I13 and the second corrected image I23 to obtain the first local image I14 and the second local image I24. That is to say, after the image pre-processing module 22 adjusts the brightness and clarity of the first image I11 and the second image I21, it removes the background (such as positioning points, masks), and then uses the position of one image to correct the position of the other image. After the brightness, clarity, and position of both are corrected, the local image of the benign tumor (including the tissue near the benign tumor) is extracted to obtain the first local image I14 and the second local image I24.

[0049] like Figure 1A , FIG. 4D to FIG. 4EAs shown, the target object extraction module 23 automatically detects the first local image I14 and the second local image I24 and circles the position of the benign tumor, so as to obtain the first region of interest (ROI) image I15 and the second region of interest image I25 respectively. For example, the target object extraction module 23 can be a convolutional neural network (CNN) that automatically detects the first local image I14 and the second local image I24 and circles the position of the benign tumor. Preferably, the target object extraction module 23 can be a convolutional neural network that can automatically detect and circle the position of the benign tumor in the image, such as but not limited to a U-Net neural network (3D single-path U-Net neural network) or a 3D dual-path U-Net neural network; preferably, the target object extraction module 23 automatically detects the first local image I14 and the second local image I24 and circles the position of the benign tumor, respectively, through a 3D dual-path U-Net neural network.

[0050] The feature extraction module 24 automatically recognizes the first region of interest image I15 and the second region of interest image I25 to obtain at least one first feature and at least one second feature. For example, the at least one first feature is, but not limited to, a texture feature, a grayscale feature, or a texture feature plus a grayscale feature, and the at least one second feature is, but not limited to, a texture feature, a grayscale feature, or a texture feature plus a grayscale feature. The texture features and grayscale features include but are not limited to the mean of local binary patterns (LBP), long run low gray-level emphasis (LRLGLE), skewness, standard deviation, and root mean square of the first region of interest image and the second region of interest image, wherein the mean of local binary patterns is obtained by analyzing the first region of interest image or the second region of interest image using a local binary pattern filter; the long run low gray-level emphasis is obtained by analyzing the first region of interest image or the second region of interest image using a gray-level run-length matrix (GLRLM) filter; the skewness, standard deviation, and root mean square are obtained by analyzing the first region of interest image or the second region of interest image using a histogram filter. For example, the feature extraction module 24 may be a convolutional neural network (CNN) that uses different filters to automatically identify the first region of interest image I15 and the second region of interest image I25, respectively, to obtain at least one first feature and at least one second feature, respectively. Preferably, the feature extraction module 24 may be a convolutional neural network that can automatically identify tumor features in images that uses different filters to analyze the first region of interest image I15 and the second region of interest image I25, such as but not limited to a 3D single-path U-Net neural network or a 3D dual-path U-Net neural network; preferably, the feature extraction module 24 uses a 3D dual-path U-Net neural network to analyze the first region of interest image I15 and the second region of interest image I25, respectively, using different filters.

[0051] The trend analysis module 25 analyzes at least one first feature and at least one second feature to obtain a tumor development trend analysis result for determining the tumor development trend. The first image and the second image may be images before treatment or images after treatment. When the first image and the second image are images before treatment, the tumor development trend analysis result may be a prediction that a benign tumor is not responsive to radiation treatment, or a prediction that a benign tumor is responsive to radiation treatment; when the first image and the second image are images after treatment, the tumor development trend analysis result may be a judgment that a benign tumor is not responsive to radiation treatment, or a judgment that a benign tumor is responsive to radiation treatment. When the tumor development trend analysis result is that a benign tumor is responsive to radiation treatment, it may be that a benign tumor is responsive to radiation treatment but has pseudoprogression, or that a benign tumor is responsive to radiation treatment and has no pseudoprogression. Here, benign tumors are not responsive to radiation treatment, or benign tumors are responsive to radiation treatment, which means that they are used to predict / determine whether the benign tumor changes after being subjected to radiation treatment. If there is a change (including enlargement or reduction), it means that the benign tumor responds to the radiation treatment, and if there is no change, it means that the benign tumor does not respond to the radiation treatment. When the tumor development trend analysis result is that the benign tumor responds to the radiation treatment (including tumor enlargement or tumor reduction), it can be that the benign tumor responds to the radiation treatment but has a false progression (the tumor first becomes larger and then becomes smaller), or the benign tumor responds to the radiation treatment and has no false progression (the tumor becomes smaller). For example, the trend analysis module 25 can be, for example but not limited to, by analyzing at least one first feature and at least one second feature using a support vector machine (SVM), manually, or gap analysis to obtain the tumor development trend analysis result. Preferably, the trend analysis module 25 analyzes at least one first feature and at least one second feature using a support vector machine. In this embodiment, the trend analysis module 25 may store a plurality of trend paths, and the trend analysis module 25 uses at least one first feature and at least one second feature to analyze these trend paths. These trend paths are obtained by the trend analysis module 25 analyzing a plurality of reference images. In detail, the benign tumor development trend assessment system 100 may have a plurality of reference images, and these reference images may be stored in a database or in the trend analysis module 25. The trend analysis module 25 may obtain a plurality of trend paths after analyzing these reference images by, for example, but not limited to, a support vector machine, and store these trend paths in the trend analysis module 25. When the trend analysis module 25 uses, for example, but not limited to, a support vector machine to analyze at least one first feature and at least one second feature, it uses at least one first feature and at least one second feature to analyze these trend paths, so as to analyze which trend path the at least one first feature and the at least one second feature are more in line with. The detailed process will be described later and will not be repeated here.

[0052] Please refer to Figure 1A and Figure 4A In this embodiment, the servo computer device 2 may further include a judgment module 26 for generating a judgment result based on the tumor development trend analysis result. In another embodiment, the servo computer device 2 may further include a judgment result output module 27 for outputting the judgment result. In detail, the first image I11 and the second image I21 may be images before treatment or after treatment. When the first image I11 and the second image I21 are images before treatment, and the tumor development trend analysis result predicts that the benign tumor will not respond to radiation treatment, the judgment result is a recommendation for other treatments; when the first image I11 and the second image I21 are images before treatment, and the tumor development trend analysis result predicts that the benign tumor will respond to radiation treatment (including benign tumors responding to radiation treatment but with pseudoprogression, or benign tumors responding to radiation treatment without pseudoprogression When the first image I11 and the second image I21 are images after treatment and the tumor development trend analysis result is that the benign tumor is not responsive to the radiation treatment, the judgment result is that other treatments are recommended; when the first image I11 and the second image I21 are images after treatment and the tumor development trend analysis result is that the benign tumor is responsive to the radiation treatment (including that the benign tumor is responsive to the radiation treatment but has false progression, or that the benign tumor is responsive to the radiation treatment and has no false progression), the judgment result is that continuous follow-up is recommended.

[0053] Please refer to Figure 1A In this embodiment, the server computer device 2 may further include a processor S1, which executes the image receiving module 21, the image pre-processing module 22, the target object extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26 and the judgment result output module 27 as described above to perform the processing flow as described above.

[0054] Please refer to Figure 1A In this embodiment, the benign tumor development trend assessment system 100 may further include a user computer device 3 for receiving the determination result output by the server computer device 2 .

[0055] Please refer to Figure 2 And refer to Figure 1A , Figure 2 Schematic diagram of a servo computer device of a benign tumor development trend assessment system of the present invention. The servo computer device 2 is applied to a benign tumor development trend assessment system 100. The benign tumor development trend assessment system 100 includes an image output device 1 and a servo computer device 2. The servo computer device 2 may include the modules described above, and the processing procedures and functions executed by each module have been described in detail above, and will not be repeated here.

[0056] like Figure 2 As shown, the server computer device 2 may further include a computer-readable storage medium S2, a communication element S3, a display element S4, an input element S5, and a housing (not shown). The server computer device 2 may be, for example, a server, a mobile phone, a tablet, a laptop computer, a desktop computer, or other computer devices. The server, mobile phone, tablet computer, laptop computer, and desktop computer include a housing to accommodate the processor S1, the computer-readable storage medium S2, and the communication element S3; the display element S4 and the input element S5 of the mobile phone, tablet computer, and laptop computer are mounted on the housing, and the server and desktop computer are connected to the display element S4 and the input element S5 outside the host.

[0057] The processor S1 is coupled to the computer-readable storage medium S2, the communication element S3, the display element S4, and the input element S5, and is configured to execute the image receiving module 21, the image pre-processing module 22, the target extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26, and the judgment result output module 27 as described above. The processor S1 is, for example, a processor S1 that can execute a module (such as a module, a program code, or an instruction). The server computer device 2 may include one or more processors S1, and the processor S1 may include one or more cores. The computer-readable storage medium S2 includes a random access memory or a non-volatile computer-readable storage medium, etc. The non-volatile computer-readable storage medium is, for example, a hard disk, a solid state drive (SSD), a flash memory, etc., which stores the modules (such as modules, program codes, or instructions) executable by the processor S1. The processor S1 can load the modules (such as modules, program codes, or instructions) from the non-volatile computer-readable storage medium to the random access memory and execute them. The communication element S3 is, for example, a network card, a network chip, a modem, etc., which can provide a network connection. The display element S4 includes a display card, a display chip, a display, etc. The input element S5 is, for example, a keyboard, a mouse, or a touch screen.

[0058] In the above embodiment, the image receiving module 21, the image pre-processing module 22, the target object extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26 and the judgment result output module 27 are stored in the computer-readable storage medium S2 in the form of software and can be executed by the processor S1 of the computer device. However, the image receiving module 21, the image pre-processing module 22, the target object extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26 and the judgment result output module 27 can also be stored in the random access memory of the processor S1 in the form of software (not shown); or, the image receiving module 21, the image pre-processing module 22, the target object extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26 The judgment result output module 27 may be in the form of hardware (for example, but not limited to, a special application integrated circuit, ASIC), coupled to the processor S1 (not shown) to perform its functions; or, the image receiving module 21, the image pre-processing module 22, the target object extraction module 23, the feature extraction module 24, the trend analysis module 25, the judgment module 26 and the judgment result output module 27 may be in the form of firmware, such as software embedded in a special application integrated circuit (not shown), and the present invention is not limited thereto.

[0059] In addition, the user computer device 3 may also include a processor, a computer-readable storage medium, a communication element, a display element, an input element, and a housing (not shown). The user computer device 3 may be, for example, a mobile phone, a tablet, a laptop computer, a desktop computer, or other computer devices. The mobile phone, tablet computer, laptop computer, and desktop computer include a housing to accommodate the processor, a computer-readable storage medium, and a communication element; the display element and input element of the mobile phone, tablet computer, and laptop computer are mounted on the housing, and the desktop computer is connected to the display element and input element outside the host. The user computer device 3 is connected to the server computer device 2 through, for example, but not limited to, a network to receive the judgment result output by the server computer device 2.

[0060] Please refer to Figure 3 And refer to Figure 1A , Figure 1B , FIG. 4A to FIG. 4E , used to illustrate the operation of the benign tumor development trend assessment system 100 of the present invention. Figure 3The schematic diagram of the implementation process of the benign tumor development trend assessment system of the present invention is shown in the figure. As shown in the figure, first, the first image I11 and the second image I21 extracted from the same position of the benign tumor are output to the server computer device 2 through the image output device 1. Then, the server computer device 2 receives the first image I11 and the second image I21 output by the image output device 1 through the image receiving module 21. The server computer device 2 then performs image pre-processing on the first image I11 and the second image I21 through the image pre-processing module 22 to obtain a first partial image I14 and a second partial image I24, respectively. The image pre-processing includes adjusting the brightness and clarity of the first image I11 and the second image I21 and removing the background through the image adjustment unit 221 to obtain a first adjusted image I12 and a second adjusted image I22; correcting the position of the second adjusted image I22 with the first adjusted image I12 through the image correction unit 222 to obtain a first corrected image I13 and a second corrected image I23; and performing partial image extraction on the first corrected image I13 and the second corrected image I23 through the partial image extraction unit 223 to obtain a first partial image I14 and a second partial image I24. Next, the servo computer device 2 automatically detects the first local image I14 and the second local image I24 through the target object extraction module 23 and selects the location of the benign tumor to obtain the first region of interest image I15 and the second region of interest image I25 respectively. Next, the servo computer device 2 automatically identifies the first region of interest image I15 and the second region of interest image I25 through the feature extraction module 24 to obtain at least one first feature and at least one second feature respectively. Next, the servo computer device 2 analyzes the at least one first feature and the at least one second feature through the trend analysis module 25 to obtain the tumor development trend analysis result. Then, the servo computer device 2 generates a judgment result based on the tumor development trend analysis result through the judgment module 26, and outputs the judgment result to the user computer device 3 through the judgment result output module 27. Finally, the user computer device 3 receives the judgment result output by the servo computer device 2. In short, the image output device 1 outputs the first image I11 and the second image I21 extracted at the same position of the benign tumor to the server computer device 2. The server computer device 2 automatically processes the first image I11 and the second image I21 through various modules to obtain a judgment result, and outputs the judgment result to the user computer device 3. Here, the image output device 1 is, for example but not limited to, a Universal Serial Bus (USB) port, an optical disk, a disk, a computer device, or a magnetic resonance imaging device that can store and output images. The examples of the server computer device 2 and the user computer device 3 have been described in detail above and will not be repeated here.

[0061] Experimental example

[0062] Reference Image Database

[0063] Reference images were obtained from 336 patients with acoustic neuroma who were treated at Taipei Veterans General Hospital in Taiwan from 1993 to 2019 and who did not undergo craniotomy to remove the tumor and were followed up for at least 24 months after radiosurgery (median follow-up, 65.1 months). All patients who underwent this study signed written informed consent approved by the hospital's Institutional Review Board (IRB). Reference images were MRI images obtained using a 1.5-T MR scanner (Signa Horizon LX2, GE Medical System) before, after, and during follow-up, including T1W+C images, T2W images, and T1W images. Here, radiation treatment refers to GKRS. Tumor volume was then longitudinally measured (in cm) on MRI images before, after, and during follow-up by an experienced neuroradiologist. 3 According to the response of VS to radiation therapy (GKRS), the patients were further divided into three groups, including tumor response without pseudoprogression (130 cases), tumor response with pseudoprogression (135 cases) and tumor non-response (71 cases). Table 1 lists the clinical characteristics of the three groups of patients.

[0064] Table 1: Clinical data of 336 VS patients

[0065]

[0066] As shown in Table 1, clinical data of 336 VS patients are listed in Table 1. There were no significant differences in patient age, gender, GKRS prescribed dose, and GKRS maximum dose among the three groups. The total tumor volume of the tumor non-response group was significantly smaller than that of the tumor response without pseudoprogression group (p<0.001). Regarding the changes in tumor volume after GKRS treatment, the tumor volume of the response without pseudoprogression group gradually decreased from -13.1±17.4% to -66.0±20.6% between six months and the final follow-up. The group with response but pseudoprogression was different, showing a significant increase in tumor volume (37.2±32.6%) in the six-month follow-up images, and then gradually decreased to -48.3±22.4% until the final follow-up. For the tumor non-response group, the tumor volume increased from 29.4±35.1% at the six-month follow-up to 48.9±93.7% at the final follow-up.

[0067] Image pre-processing

[0068] MRI images of 336 patients were pre-processed to reduce the differences in imaging parameters and improve the reliability of radiomics analysis. First, image clarity (resolution) adjustment was performed to resample all pixel sizes to 0.50 x 0.50 x 3.00 mm. 3 , and there is no gap between each continuous slice of MR imaging. Then, the six-parameter rigid body transformation and mutual information algorithm are used to positionally calibrate the T2W and T1W images of the same patient taken at the same time with the T1W+ C images. Subsequently, the local image of the benign tumor (including the tissue near the benign tumor) is extracted and the location of the benign tumor is circled to obtain the image of the region of interest.

[0069] Establishing trend path

[0070] Next, the images of the region of interest were analyzed by imaging atlas to calculate their grayscale features and texture features, and these features were statistically compared to analyze the features with significant differences among the three groups of patients for establishing trend paths.

[0071] Characteristics that differ significantly between the groups with benign tumors that respond to radiation therapy or the groups with benign tumors that do not respond to radiation therapy:

[0072] There were 3 characteristics that showed significant differences (p<3.65x10 -5 ), recorded in Table 2 below. They are the grayscale features obtained from the T2W image using the histogram filter, including the standard deviation and the root mean square, and the texture features obtained from the T1W+C image using the local binary pattern filter. The texture features are the local binary pattern average values. The above three features are used to indicate that the tumor state is cystic or loose tissue.

[0073] Table 2

[0074]

[0075] *LLH is a wavelet filter, where L is a low-pass filter and H is a high-pass filter. The order in which the three are arranged indicates the filters used for the X, Y, and Z axes of the image, respectively. That is, low-pass filters are used for the X and Y axes, and a high-pass filter is used for the Z axis.

[0076] Characteristics that differed significantly between the group with benign tumors that responded to radiation therapy and did not have pseudoprogression or the group with benign tumors that responded to radiation therapy but had pseudoprogression:

[0077] There were significant differences in three characteristics between the group of benign tumors that responded to radiation therapy without pseudoprogression (hereinafter referred to as "tumor response without pseudoprogression" or "response without pseudoprogression") and the group of benign tumors that responded to radiation therapy but had pseudoprogression (hereinafter referred to as "tumor response with pseudoprogression" or "response with pseudoprogression") (p < 3.65x10 -5 ), recorded in Table 3 below. They are respectively the texture features obtained from the T1W+C image using the grayscale length matrix filter, the texture feature is the long-range low grayscale advantage, and the grayscale feature obtained using the histogram filter, the grayscale feature is the skewness; and the grayscale feature obtained from the T2W image using the histogram filter, the grayscale feature is the standard deviation. It is worth noting that the standard deviation feature obtained from the T2W image can be used to distinguish whether the tumor responds to radiation treatment or not. When the tumor responds to radiation treatment, it can be further used to distinguish whether the tumor responds and has no pseudoprogression, or responds but has pseudoprogression.

[0078] Table 3

[0079]

[0080] The above features obtained from the MRI images of the above 336 VS patients were analyzed separately, and the features in Table 2 were used for the first evaluation, and the features in Table 3 were used for the second evaluation, wherein the first evaluation was used to establish a trend path that can distinguish benign tumors that respond to radiation treatment or do not respond, and the second evaluation was used to establish a trend path that can distinguish benign tumors that respond to radiation treatment but have pseudoprogression, or respond without pseudoprogression.

[0081] Used to predict / determine whether benign tumors will respond or not to radiation treatment

[0082] Please refer to FIG. 5A to FIG. 5C , used to illustrate the trend path established by the first assessment, FIG. 5A to FIG. 5C The figure is a schematic diagram of the benign tumor development trend assessment system of the present invention using tumor characteristics to obtain tumor development trend analysis results, wherein the tumor development trend analysis results are that the tumor is not responsive to radiation treatment, or that the tumor is responsive to radiation treatment. Figure 5A As shown, Figure 5AThe 3D distribution diagram of tumor characteristics of multiple reference images is displayed. The non-responsive tumor group is hollow dots, and the responsive tumor group is solid dots. The 3D distribution of the local binary mode mean, standard deviation, and root mean square characteristics of the two groups can be roughly distinguished. A gray surface is formed between the two groups, and different trend paths are established. When a new tumor image obtains these three features according to the aforementioned method, analysis shows that its features fall on the side of the gray surface close to the solid dots, or on the side of the gray surface close to the hollow dots. This means that the tumor in the new tumor image is unresponsive or responsive to radiation treatment. Figure 5B As shown, Figure 5B Is Figure 5A The prediction model score diagram of the tumor feature distribution conversion. The prediction model scores of the two groups can be distinguished by the dotted line. When a new tumor image obtains these three features according to the above method, by converting its feature 3D distribution map into a score and analyzing whether the score falls above or below the dotted line, it can be known whether the tumor in the new tumor image is unresponsive or responsive to radiation treatment. Figure 5C As shown, Figure 5C It is a receiver operating characteristic curve for evaluating whether a tumor is unresponsive to radiation treatment or responsive to radiation treatment. When the above three features are used to predict / judge whether a tumor is unresponsive to radiation treatment or responsive, the average area under the curve (AUC) = 0.877 (sensitivity = 86.4%, specificity = 90.5%, accuracy = 88.4%). The above three feature values ​​can be obtained by, for example, but not limited to, manual, support vector machine, gap analysis, etc. Figure 5C result.

[0083] When the prediction / judgment result is a response, further prediction / judgment is a response to radiation treatment but with pseudoprogression, or a response without pseudoprogression.

[0084] Please refer to FIG. 6A to FIG. 6C , used to illustrate the trend path established by the second assessment, FIG. 6A to FIG. 6C The figure is a schematic diagram of the benign tumor development trend assessment system of the present invention using tumor characteristics to obtain tumor development trend analysis results, wherein the tumor development trend analysis results are that the tumor responds to radiation treatment but has pseudoprogression, or the tumor responds to radiation treatment and has no pseudoprogression. Fig. 6AAs shown, Figure 6A shows the 3D distribution diagram of tumor characteristics of multiple reference images. The group with tumor response but pseudoprogression is hollow dots, and the group with tumor response and no pseudoprogression is solid dots. The 3D distribution of long-range low grayscale advantage, standard deviation, and skewness characteristics of the two groups can be roughly distinguished. A gray surface is formed between the two groups, and different trend paths are established. When a new tumor image obtains these three characteristics according to the aforementioned method, it can be learned through analysis that its characteristics fall on the side of the gray surface close to the solid dots, or the side of the gray surface close to the hollow dots. It can be known that the tumor in the new tumor image is responsive to radiation treatment without pseudoprogression, or responsive but with pseudoprogression. Figure 6B As shown, Figure 6B Is Fig. 6A The prediction model score diagram of the tumor feature distribution conversion. The prediction model scores of the two groups can be distinguished by the dotted line. When a new tumor image obtains these three features according to the above method, by converting its feature 3D distribution map into a score and analyzing whether the score falls above or below the dotted line, it can be known that the tumor in the new tumor image is responsive to radiation treatment without false progression, or responsive but with false progression. Figure 6C As shown, Figure 6C It is a receiver operating characteristic curve for evaluating whether the tumor responds to radiation treatment but has pseudoprogression, or responds to radiation treatment without pseudoprogression. When the above three features are used to predict / judge whether the tumor responds to radiation treatment without pseudoprogression, or responds but has pseudoprogression, its AUC = 0.816 (sensitivity = 79.5%, specificity = 84.6%, accuracy = 82.1%). The above three feature values ​​can be obtained by, for example, but not limited to, manual, support vector machine, gap analysis, etc. FIG. 6A to FIG. 6C result.

[0085] In summary, the tumor development trend assessment system of the present invention can automatically assess the patient's future tumor development trend based on tumor images, and can provide different suggestions to medical staff based on the assessment results, so as to provide preferred treatment methods for different patients, thereby improving the treatment effect of benign tumors. In addition, it can also be used to assess whether the patient is likely to have pseudo-progression of the tumor, so as to inform the patient in advance, which is more helpful to reduce the patient's doubts and the pressure of medical staff.

[0086] The above description is for illustrative purposes only and is not intended to be limiting. Any equivalent modifications or changes made thereto without departing from the spirit and scope of the present invention shall be included in the scope of the appended claims.

Claims

1. A benign tumor development trend assessment system, It is characterized in that The benign tumor development trend assessment system comprises: An image output device, configured to output a first image and a second image extracted at the same position of a benign tumor, wherein the first image and the second image are images obtained by performing angiography using different parameters of the image output device, and the first image and the second image are respectively selected from T1-weighted images, T2-weighted images, or T1-weighted contrast-enhanced images; and A server computer device comprising: An image receiving module, receiving the first image and the second image; An image pre-processing module performs image pre-processing on the first image and the second image to obtain a first partial image and a second partial image respectively; a target object extraction module, which automatically detects the first local image and the second local image and circles the location of the benign tumor to obtain a first region of interest (ROI) image and a second region of interest image respectively; a feature extraction module, automatically identifying the first region of interest image and the second region of interest image, respectively, to obtain at least one first feature and at least one second feature, wherein the at least one first feature is a texture feature, or a grayscale feature, or a texture feature and a grayscale feature, and the at least one second feature is a texture feature, or a grayscale feature, or a texture feature and a grayscale feature; A trend analysis module analyzes the at least one first feature and the at least one second feature to obtain a tumor development trend analysis result for determining the tumor development trend; and A judgment module generates a judgment result according to the tumor development trend analysis result, wherein the first image and the second image are images before treatment or images after treatment. When the first image and the second image are images before treatment, the tumor development trend analysis result is a prediction that the benign tumor will not respond to radiation treatment, or a prediction that the benign tumor will respond to radiation treatment; when the first image and the second image are images after treatment, the tumor development trend analysis result is a judgment that the benign tumor will not respond to radiation treatment, or a judgment that the benign tumor will respond to radiation treatment; wherein when the tumor development trend analysis result is that the benign tumor responds to radiation treatment, it means that the benign tumor responds to radiation treatment but has pseudoprogression, or the benign tumor responds to radiation treatment without pseudoprogression; wherein when When the first image and the second image are images before treatment and the tumor development trend analysis result predicts that the benign tumor will not respond to radiation treatment, the judgment result is to perform other treatments; when the first image and the second image are images before treatment and the tumor development trend analysis result predicts that the benign tumor will respond to radiation treatment, the judgment result is to recommend radiation treatment; when the first image and the second image are images after treatment and the tumor development trend analysis result determines that the benign tumor will not respond to radiation treatment, the judgment result is to recommend other treatments; and when the first image and the second image are images after treatment and the tumor development trend analysis result determines that the benign tumor will respond to radiation treatment, the judgment result is to recommend continued tracking.

2. The system according to claim 1, wherein the image pre-processing module include: An image adjustment unit, which adjusts the brightness and clarity of the first image and the second image and removes the background to obtain a first adjusted image and a second adjusted image; an image correction unit, which corrects the position of the second adjusted image using the first adjusted image to obtain a first corrected image and a second corrected image; and a local image extraction unit, performing local image extraction on the first corrected image and the second corrected image respectively to obtain the first local image and the second local image, The first local image and the second local image include images of the benign tumor and tissues near the benign tumor.

3. The system according to claim 1, wherein the trend analysis module stores a plurality of trend paths, and the trend analysis module performs analysis with the help of the at least one first feature and the at least one second feature and the trend path, wherein the trend path is obtained by analyzing a plurality of reference images by the trend analysis module.

4. The system according to claim 1, wherein the servo computer device further comprises a judgment result output module and a processor, the judgment result output module outputs the judgment result, and the processor executes the image receiving module, the image pre-processing module, the target object extraction module, the feature extraction module, the trend analysis module, the judgment module and the judgment result output module.

5. The system according to claim 4, further comprising a user computer device for receiving the determination result output by the server computer device.

6. The system according to claim 1, wherein the first image and the second image have different contrasts, the at least one first feature comprises a texture feature or a grayscale feature, and the at least one second feature comprises a texture feature or a grayscale feature.

7. The system of claim 1, wherein the benign tumor is an acoustic neuroma, a meningioma, a pituitary tumor, a neuroblastoma, or a craniopharyngioma.

8. A servo computer device, applied to a benign tumor development trend assessment system, the benign tumor development trend assessment system comprising an image output device and the servo computer device, It is characterized in that The server computer device comprises: An image receiving module receives a first image and a second image extracted at the same position of a benign tumor, wherein the first image and the second image are images obtained by performing angiography using different parameters of the image output device, and the first image and the second image are respectively selected from T1-weighted images, T2-weighted images, or T1-weighted contrast-enhanced images; An image pre-processing module performs image pre-processing on the first image and the second image to obtain a first partial image and a second partial image respectively; A target object extraction module automatically detects the first local image and the second local image and circles the location of the benign tumor to obtain a first region of interest image and a second region of interest image respectively; a feature extraction module, automatically identifying the first region of interest image and the second region of interest image, respectively, to obtain at least one first feature and at least one second feature, wherein the at least one first feature is a texture feature, or a grayscale feature, or a texture feature and a grayscale feature, and the at least one second feature is a texture feature, or a grayscale feature, or a texture feature and a grayscale feature; A trend analysis module, analyzing the at least one first feature and the at least one second feature to obtain a tumor development trend analysis result for determining the tumor development trend; A judgment module, generating a judgment result according to the tumor development trend analysis result; and The processor executes the image receiving module, the image pre-processing module, the target object extraction module, the feature extraction module, the trend analysis module and the judgment module.

9. The server computer device according to claim 8, wherein the image pre-processing module include: An image adjustment unit, which adjusts the brightness and clarity of the first image and the second image and removes the background to obtain a first adjusted image and a second adjusted image; an image correction unit, which corrects the position of the second adjusted image using the first adjusted image to obtain a first corrected image and a second corrected image; and a local image extraction unit, performing local image extraction on the first corrected image and the second corrected image respectively to obtain the first local image and the second local image, The first local image and the second local image include images of the benign tumor and tissues near the benign tumor.

10. A computer-readable storage medium, used in a benign tumor development trend assessment system, It is characterized in that The storage medium stores the module according to any one of claims 1 to 7, and the module can be executed by a processor of a computer device.

Images