Radiotherapy target region automatic segmentation method and system based on image processing

Abstract

The invention provides a radiotherapy target region automatic segmentation method and system based on image processing, and the method comprises the steps: obtaining a CT image and an X-ray image of an examination part, calculating the entropies corresponding to three color channels of the X-ray image, and correcting the X-ray image based on the entropies; calculating to obtain a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and calculating to obtain a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image; performing reverse calculation based on the high-frequency fusion coefficient and the low-frequency fusion coefficient by using a multi-scale geometric analysis tool to obtain a fusion image; obtaining a slice with a target region in the plurality of slices, obtaining a radiotherapy dose corresponding to the slice with the target region, extracting a second feature map of the slice with the target region based on the radiotherapy dose, fusing the second feature map with the slice with the target region, segmenting the fused features, and obtaining a second feature map of the slice with the target region; and the target region segmentation precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an automatic segmentation method and system for radiotherapy target areas based on image processing. Background Art

[0002] Image segmentation is an important research topic in medical image processing and analysis. However, manual segmentation by doctors is not only time-consuming and labor-intensive, but also easily affected by subjective experience, with large errors and low automation. In order to solve these problems, medical image processing methods based on computer-aided diagnosis have always been a hot topic of research. In recent years, deep learning methods have achieved great success in the field of computer vision, and therefore have been introduced by researchers into medical image processing and analysis. Deep learning methods have powerful feature learning capabilities and have made good progress in disease diagnosis, prediction, lesion detection and segmentation.

[0003] Image segmentation technology can provide very valuable information. Radiotherapy target segmentation is an important step in image analysis. It can accurately locate and quantify the shape and size of the tumor and provide the best treatment plan for patients. However, the existing image segmentation methods generally have low segmentation accuracy and cannot automatically and accurately segment the target area. Therefore, in view of the shortcomings of the existing technology, we propose an automatic radiotherapy target segmentation method based on image processing. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an automatic segmentation method and system for radiotherapy target area based on image processing, which are used to solve the technical problems in the prior art.

[0005] In a first aspect, the invention provides the following technical solution: a method for automatic segmentation of radiotherapy target area based on image processing, the method comprising:

[0006] Acquire a CT image and an X-ray image of the inspection part, calculate the entropy corresponding to each of the three color channels of the X-ray image, correct the X-ray image based on the entropy, and process the corrected X-ray image using a histogram normalization technique to obtain a grayscale image;

[0007] Decomposing the CT image by using a multi-scale geometric analysis tool to obtain a first low-frequency sub-band image and a first high-frequency sub-band image, and decomposing the grayscale image to obtain a second low-frequency sub-band image and a second high-frequency sub-band image, calculating a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and calculating a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image;

[0008] Using a multi-scale geometric analysis tool to reversely calculate the high-frequency fusion coefficient and the low-frequency fusion coefficient to obtain a fused image, and slicing the fused image to form a plurality of slices;

[0009] Obtain slices with target areas from the plurality of slices, obtain radiotherapy doses corresponding to the slices with target areas, extract second feature maps of the slices with target areas based on the radiotherapy doses, fuse the second feature maps with the slices with target areas, and segment the fused features.

[0010] Compared with the prior art, the beneficial effects of the present application are as follows: by calculating the entropy corresponding to each of the three color channels of the X-ray image, and correcting the setting of the X-ray image based on the entropy, the X-ray image can be effectively enhanced, which is helpful for its fusion with the CT image. Through the fused image, a better visual effect can be obtained, and the accuracy of the automatically segmented radiotherapy target area can be significantly improved. In addition, through the fused image, the doctor can obtain more comprehensive diagnostic information, so as to more accurately locate the tumor, evaluate the invasion range of the tumor, and formulate a treatment plan. By extracting the second feature map of the slice with the target area based on the therapeutic dose, and fusing the second feature map with the slice with the target area, the accuracy of target area segmentation can be improved.

[0011] Furthermore, the step of calculating the low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image and calculating the high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image includes:

[0012] Acquire a first saliency map of the first low-frequency sub-band image and a second saliency map of the second low-frequency sub-band image, and calculate the significance of the first saliency map and the second saliency map respectively by using a prognostic model;

[0013] Based on the significance, the significance of the first significance map and the second significance map is divided into a first area and a second area, low-frequency fusion coefficients of the first area and the second area are calculated respectively, and high-frequency fusion coefficients of the first high-frequency sub-band image and the second high-frequency sub-band image are calculated using an absolute value formula.

[0014] Furthermore, the calculation formula of the low-frequency fusion coefficient of the first region includes:

[0015] A1=W CT1 ×G CT1 +W X1 ×G X1

[0016] W CT =H CT / (H CT +HX )

[0017] W X =H C / (H CT +H X )

[0018] Where A1 represents the low-frequency fusion coefficient of the first region, W CT Represents the weight coefficient of the CT image, W X Represents the weight coefficient of the X-ray image, G CT1 represents the first low-frequency sub-band image of the corresponding CT image of the first region, G X1 represents the second low-frequency sub-band image of the corresponding X-ray image of the first region, H CT represents the saliency of the CT image, H X Indicates the saliency of X-ray images;

[0019] The calculation formula of the low-frequency fusion coefficient of the second area includes:

[0020] A2=W CT2 ×G CT2 +W X2 ×G X2

[0021] W bCT =(1-y)×H CT1 / (H CT1 +H X1 )

[0022] W bPET =(1-y)×H CT1 / (H CT1 +H X1 )

[0023] Where A2 represents the low-frequency fusion coefficient of the second region, G CT2 represents the first low-frequency sub-band image of the corresponding CT image of the second region, G X2 represents the second low-frequency sub-band image of the corresponding X-ray fluoroscopic image of the second region, H CT1 represents the saliency of the CT image, H X1 represents the saliency of the X-ray image, and y represents the global threshold;

[0024] The absolute value formula includes:

[0025]

[0026] Among them, A3 represents the high-frequency fusion coefficient, G CT3 Represents the high-frequency sub-band image of the CT image, G X3It represents the high-frequency sub-band image of the X-ray image, a represents the decomposition layer number, and b represents the filtering direction.

[0027] Furthermore, the step of obtaining a radiotherapy dose corresponding to the slice with the target area and extracting a second feature map of the slice with the target area based on the radiotherapy dose includes:

[0028] Calculating a corresponding radiotherapy dose based on the slice with the target region using an exponential decay formula, and radiating the radiotherapy dose to the slice with the target region to obtain a dose map;

[0029] The dose map is input into a feature extraction model to obtain a plurality of first feature maps of different sizes, and the plurality of first feature maps of different sizes are spliced ​​into a second feature map using a splicing formula.

[0030] Furthermore, the exponential decay formula includes:

[0031] F (x) =F0e -ux

[0032] In the formula, F (x) represents the beam intensity at position x, F0 represents the initial intensity of the beam, i.e., the beam intensity at x=0, u represents the linear attenuation coefficient, and e represents the base of the natural logarithm;

[0033] The splicing formula includes:

[0034] f(P)=(1-m)(1-n)f(D1)+m(1-n)f(D2)+(1-m)hf(D3)+mnf(D4)

[0035]

[0036] In the formula, f(P) represents the pixel value of the new point to be interpolated, f(D1), f(D2), f(D3) and f(D4) represent the four vertex pixel values ​​of the first feature map, m and n represent the interpolation weights, X and Y represent the horizontal and vertical coordinates of the target point, respectively. represents the horizontal coordinate of vertex D1, represents the horizontal coordinate of vertex D2, represents the ordinate of vertex D1, Indicates the ordinate of vertex D3.

[0037] Furthermore, the step of fusing the second feature map with the slice with the target area and segmenting the fused features includes:

[0038] Using a weighted formula, the second feature map is fused with the slice with the target area to obtain a fused feature, and the fused feature is input into a segmentation model to obtain a segmentation feature;

[0039] The weighted formula includes:

[0040] B=[B1*c1;B2*c2]

[0041] In the formula, c1 represents the second feature map, c2 represents the slice with the target area, B1 represents the weight of the second feature map, B2 represents the weight of the slice with the target area, and B represents the fused feature after weighted fusion.

[0042] Furthermore, the step of calculating the entropy corresponding to each of the three color channels of the X-ray fluoroscopic image and correcting the X-ray fluoroscopic image based on the entropy includes:

[0043] The entropy of the red channel, the green channel and the blue channel of the X-ray image are calculated respectively by using an entropy calculation formula, and the entropy of the red channel, the green channel and the blue channel are standardized to obtain a red judgment value, a green judgment value and the blue judgment value;

[0044] The red judgment value or the blue judgment value is selected as a judgment basis value, and the judgment basis value is compared with an interval threshold value, and corresponding weights are assigned to the three color channels of the X-ray image based on the comparison result.

[0045] In a second aspect, the invention provides the following technical solution: an automatic segmentation system for radiotherapy target area based on image processing, the system comprising:

[0046] A processing module, used for acquiring a CT image and an X-ray image of the inspection part, calculating the entropy corresponding to each of the three color channels of the X-ray image, correcting the X-ray image based on the entropy, and processing the corrected X-ray image using a histogram normalization technique to obtain a grayscale image;

[0047] a calculation module, configured to decompose the CT image using a multi-scale geometric analysis tool to obtain a first low-frequency sub-band image and a first high-frequency sub-band image, and to decompose the grayscale image to obtain a second low-frequency sub-band image and a second high-frequency sub-band image, and to calculate a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and to calculate a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image;

[0048] A slicing module, used for obtaining a fused image by reverse calculation based on the high-frequency fusion coefficient and the low-frequency fusion coefficient using a multi-scale geometric analysis tool, and slicing the fused image to form a plurality of slices;

[0049] A segmentation module is used to obtain slices with target areas from the plurality of slices, obtain the radiotherapy dose corresponding to the slices with target areas, extract a second feature map of the slices with target areas based on the radiotherapy dose, fuse the second feature map with the slices with target areas, and segment the fused features.

[0050] In a third aspect, the invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the automatic segmentation method of radiotherapy target area based on image processing as described above is implemented.

[0051] In a fourth aspect, the invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned method for automatic segmentation of radiotherapy target areas based on image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flowchart of a method for automatic segmentation of a radiotherapy target area based on image processing provided in a first embodiment of the present invention;

[0054] Figure 2 A flow chart of the network structure of the segmentation model and feature extraction model provided in the first embodiment of the present invention;

[0055] Figure 3 A structural block diagram of a radiotherapy target area automatic segmentation system based on image processing provided in a second embodiment of the present invention;

[0056] Figure 4 A schematic diagram of the hardware structure of a computer provided in the third embodiment of the present invention.

[0057] The embodiments of the present invention will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout are the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0059] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0060] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0061] Embodiment 1

[0062] In the first embodiment of the present invention, see Figure 1 , a radiotherapy target area automatic segmentation method based on image processing, comprising the following steps S01 to S04:

[0063] S01, obtaining a CT image and an X-ray image of the inspection part, calculating the entropy corresponding to each of the three color channels of the X-ray image, correcting the X-ray image based on the entropy, and processing the corrected X-ray image using a histogram normalization technique to obtain a grayscale image;

[0064] Specifically, the step of calculating the entropy corresponding to each of the three color channels of the X-ray fluoroscopic image and correcting the X-ray fluoroscopic image based on the entropy includes:

[0065] S11, using an entropy calculation formula to respectively calculate the entropies of the red channel, the green channel, and the blue channel of the X-ray image, and performing standardization processing on the entropies of the red channel, the green channel, and the blue channel to obtain a red judgment value, a green judgment value, and the blue judgment value;

[0066] S12, selecting the red judgment value or the blue judgment value as a judgment basis value, and comparing the judgment basis value with an interval threshold, and assigning corresponding weights to the three color channels of the X-ray image based on the comparison result.

[0067] More specifically, the entropy calculation formula includes:

[0068]

[0069] Among them, S represents entropy, Z represents the maximum value of the grayscale value of the pixel of the X-ray perspective image, that is, [0, Z] represents the range of the grayscale value of the pixel of the X-ray perspective image, and p i Indicates the proportion of pixels with grayscale value i to the total number of pixels, i∈[0,Z];

[0070] In the specific implementation, the entropy of the three color channels (RGB) is first calculated by the entropy calculation formula to obtain the entropy of the red channel (Sr), the entropy of the green channel (Sg) and the entropy of the blue channel (Sb), and then Sr, Sg and Sb are standardized to obtain the red judgment value, the green judgment value and the blue judgment value, and the sum of the three is always 0, and then the red judgment value or the blue judgment value is selected as the judgment basis value, and the interval threshold is set at the same time, and then based on the comparison between the judgment basis value and the set interval threshold, the X-ray image is divided into three brightness levels, and different weights are given based on different brightness levels to improve the overall contrast and clarity of the X-ray image, and finally the histogram specification technology is used to process the corrected X-ray image to obtain a grayscale image. It can be understood that by giving different weights to different brightness levels, the images (X-ray images) of the three brightness levels are given corresponding enhancement weights to improve the enhancement effect.

[0071] In this embodiment, the blue judgment value is selected as the judgment basis value, and the interval threshold is [-0.13, 0.31]. The judgment of the three brightness levels is as follows:

[0072] If the judgment basis value (blue judgment value) is greater than the maximum value of the interval threshold, the X-ray image is at the brightest level, so in the grayscale process, the contribution of the red and green channels is ignored, while the contribution of the blue channel is completely retained. It can be understood that the red and green channels contain almost no tumor information, so the weight expression of the three color channels is: W1 = [0, 0, 1];

[0073] If the judgment basis value (blue judgment value) belongs to the interval threshold, the X-ray image is of medium brightness, so in the grayscale process, the information of the red channel is not considered, and the information of the blue channel and the green channel needs to be considered, so the weight expression of the three color channels is: W2 = [0, 0.31 + Sr ′ +Sg ′ ,0.69+Sb ′ ], of which 0.69+Sb ′ <1, and 0.31+Sr ′ +Sg ′ >0;

[0074] If the judgment basis value (blue judgment value) is less than the minimum value of the interval threshold, the X-ray image is at the lowest brightness level, and the blue channel is not considered in the grayscale process. The information of the red channel and the green channel needs to be considered, so the weight expression of the three color channels is: W2 = [0.5 + Sr ′ ,0.5+Sb ′ +Sg ′ ,0]; among them, Sr ′ , Sg ′ and Sb ′ They are respectively represented as red judgment value, green judgment value and blue judgment value, and W1, W2 and W3 represent the weights of three color channels of three levels of X-ray images.

[0075] Specifically, the expression for standardization is:

[0076] L ′ =3*{L / J-(1 / 3)};

[0077] Among them, L ′ is the red judgment value (Sr ′ )、Green judgment value (Sg ′ ) or blue judgment value (Sb ′ ), L is the entropy of the red channel (Sr), the entropy of the green channel (Sg) and the entropy of the blue channel (Sb), and J is the sum of the entropy of the red channel (Sr), the entropy of the green channel (Sg) and the entropy of the blue channel (Sb); It can be understood that when it is necessary to solve the red judgment value, then L ′ is the red judgment value (Sr ′ ), and L is the entropy of the red channel (Sr).

[0078] CT images use X-rays and computer processing to produce detailed cross-sectional images of the internal structure of the human body. X-ray images, commonly known as X-ray images or radiographic images, are a medical imaging technology that uses X-rays to penetrate the human body or other objects and record their internal structures.

[0079] S02, using a multi-scale geometric analysis tool to decompose a first low-frequency sub-band image and a first high-frequency sub-band image of the CT image and a second low-frequency sub-band image and a second high-frequency sub-band image of the grayscale image, and calculating a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image and a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image;

[0080] Specifically, the steps of calculating the low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image and calculating the high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image include:

[0081] S21, obtaining a first saliency map of the first low-frequency sub-band image and a second saliency map of the second low-frequency sub-band image, and calculating the significance of the first saliency map and the second saliency map respectively by using a prognostic model;

[0082] S22, based on the significance, dividing the significance of the first significance map and the second significance map into a first region and a second region, respectively calculating low-frequency fusion coefficients of the first region and the second region, and calculating high-frequency fusion coefficients of the first high-frequency sub-band image and the second high-frequency sub-band image by using an absolute value formula.

[0083] In this embodiment, the multi-scale geometric analysis tool may be a non-subsampled contourlet transform, ie, a non-subsampled pyramid filter (NSP).

[0084] More specifically, the calculation formula for the low-frequency fusion coefficient of the first region includes:

[0085] A1=W CT1 ×G CT1 +W X1 ×G X1 ;

[0086] W CT =H CT / (H CT +H X );

[0087] W X =H X / (H CT +H X );

[0088] Where A1 represents the low-frequency fusion coefficient of the first region, W CT Represents the weight coefficient of the CT image, W X Represents the weight coefficient of the X-ray image, G CT1 represents the first low-frequency sub-band image of the corresponding CT image of the first region, G X1 represents the second low-frequency sub-band image of the corresponding X-ray fluoroscopic image of the first region, H CT represents the saliency of the CT image, H X Indicates the saliency of X-ray images;

[0089] The calculation formula of the low-frequency fusion coefficient of the second area includes:

[0090] A2=W CT2 ×G CT2 +W X2 ×G X2 ;

[0091] W bCT =(1-y)×H C′1 / (H CT1 +H X1 );

[0092] W bPET =(1-y)×H CT1 / (H CT1 +H X1 );

[0093] Where A2 represents the low-frequency fusion coefficient of the second region, G CT2 represents the first low-frequency sub-band image of the corresponding CT image of the second region, G X2 represents the second low-frequency sub-band image of the corresponding X-ray fluoroscopic image of the second region, H CT1 represents the saliency of the CT image, H X1 represents the saliency of the X-ray image, and y represents the global threshold;

[0094] The absolute value formula includes:

[0095]

[0096] Among them, A3 represents the high-frequency fusion coefficient, G CT3 Represents the high-frequency sub-band image of the CT image, G X3 It represents the high-frequency sub-band image of the X-ray image, a represents the decomposition layer number, and b represents the filtering direction.

[0097] S03, using a multi-scale geometric analysis tool to reversely calculate based on the high-frequency fusion coefficient and the low-frequency fusion coefficient to obtain a fused image, and slicing the fused image to form a plurality of slices.

[0098] It is worth noting that through the fused images, better visual effects can be obtained, which can significantly improve the accuracy of the automatically segmented radiotherapy target area. And the GTV manually outlined on the fused image is generally smaller than the volume based on the CT image alone. GTV is a term in the field of tumor radiotherapy, and its full name is "Gross Tumor Volume", that is, "total tumor volume". It refers to the total volume of the tumor visible in imaging examinations (such as CT, MRI or PET scans), including the primary tumor and its possible satellite lesions. The fused image is formed, which can improve the accuracy and efficiency of diagnosis in some cases. For example, when detecting malignant tumors (MA, which may refer to malignant tumors), CT images can provide detailed anatomical information, while X-ray fluoroscopy can provide information about organ function and dynamic changes. Through image fusion technology, doctors can obtain more comprehensive diagnostic information, so as to more accurately locate tumors, assess the invasion range of tumors, and formulate treatment plans.

[0099] S04, obtaining slices with target areas from among the plurality of slices, obtaining radiotherapy doses corresponding to the slices with target areas, extracting second feature maps of the slices with target areas based on the radiotherapy doses, fusing the second feature maps with the slices with target areas, and segmenting the fused features.

[0100] Specifically, the step of obtaining the radiotherapy dose corresponding to the slice with the target area and extracting the second feature map of the slice with the target area based on the radiotherapy dose includes:

[0101] Calculating a corresponding radiotherapy dose based on the slice with the target region using an exponential decay formula, and radiating the radiotherapy dose to the slice with the target region to obtain a dose map;

[0102] The dose map is input into a feature extraction model to obtain a plurality of first feature maps of different sizes, and the plurality of first feature maps of different sizes are spliced ​​into a second feature map using a splicing formula.

[0103] More specifically, the exponential decay formula includes:

[0104] F (x) =F0e -ux ;

[0105] In the formula, F (x) represents the beam intensity at position x, F0 represents the initial intensity of the beam, i.e., the beam intensity at x=0, u represents the linear attenuation coefficient, and e represents the base of the natural logarithm;

[0106] In this embodiment, the radiotherapy dose will show a decay trend outside the target area due to various factors. Based on the calculation result of the exponential decay formula, in this embodiment, 90% of the prescription dose is selected as the isodose line for auxiliary segmentation of CTV.

[0107] The splicing formula includes:

[0108] f(P)=(1-m)(1-n)f(D1)+m(1-n)f(D2)+(1-m)hf(D3)+mnf(D4);

[0109]

[0110] In the formula, f(P) represents the pixel value of the new point to be interpolated, f(D1), f(D2), f(D3) and f(D4) represent the four vertex pixel values ​​of the first feature map, m and n represent the interpolation weights, X and Y represent the horizontal and vertical coordinates of the target point, respectively. represents the horizontal coordinate of vertex D1, represents the horizontal coordinate of vertex D2, represents the ordinate of vertex D1, Indicates the ordinate of vertex D3.

[0111] More specifically, the step of fusing the second feature map with the slice with the target area and segmenting the fused features includes:

[0112] Using a weighted formula, the second feature map is fused with the slice with the target area to obtain a fused feature, and the fused feature is input into a segmentation model to obtain a segmentation feature;

[0113] The weighted formula includes:

[0114] B=[B1*c1;B2*c2];

[0115] In the formula, c1 represents the second feature map, c2 represents the slice with the target area, B1 represents the weight of the second feature map, B2 represents the weight of the slice with the target area, and B represents the fused feature after weighted fusion.

[0116] Optionally, the range of B1 is 1-1.4, and the range of B2 is 0.6-1. In this embodiment, the range of B1 is 1.2, and the range of B2 is 0.8.

[0117] In a specific implementation, in the process of obtaining slices with target regions among the slices, a detector and a corrector are used, and the detector is used to classify each slice to determine whether it contains CTV (target region). The detector includes a ResNet50 network, wherein the loss function used by the detector is a binary cross-entropy (BCE) loss function.

[0118] Among them, CTV is a term in the field of tumor radiotherapy, and its full name is “Clinical Target Volume”.

[0119] See also Figure 2 In this embodiment, the network structures of the segmentation model and the feature extraction model are consistent, wherein both models include an encoder and a decoder, wherein the encoder includes six encoding units connected in sequence, each encoding unit includes a convolution layer, a batch normalization layer, and a ReLU activation layer for feature extraction, and the downsampling result of the previous encoding unit is used as the input of the next encoding unit, and the downsampling multiples of the first four encoding units are 32, 16, 8, and 4 respectively;

[0120] Among them, the fifth and sixth coding units do not perform downsampling in order to avoid losing context information when the received input size is small; the downsampling and upsampling results of the fourth coding unit are subjected to dilated convolution operation with a dilation factor of 2, and then the results are sent to the fifth coding unit; the fifth coding unit performs a convolution operation after receiving the input, and then performs a dilated convolution with a dilation factor of 4, and finally sends it to the sixth coding unit. The sixth coding unit uses the same operation as the fifth coding unit, with a dilation factor of 8;

[0121] The decoder includes five decoding units connected in sequence, and the encoding / decoding unit is composed of the splicing of the encoding unit with the corresponding sequence number and the subsequent encoding unit.

[0122] It can be understood that the encoder of the segmentation model is responsible for extracting feature information of the fusion feature, and the encoder of the feature extraction model is responsible for extracting feature information of the dose map.

[0123] In this embodiment, the loss function used in the operation process of obtaining the segmentation features by the segmentation model is expressed as follows:

[0124]

[0125] Among them, L represents the loss function used in the operation process of the segmentation model to obtain the segmentation features, γ represents the hyperparameter used to balance the loss function, and X iis the feature vector of the i-th sample, Indicates the y i The feature center of the category, y i represents the i-th category, Represents the model's predicted probability for the i-th sample, and N represents the total number of samples. By setting this loss function, the CTV segmentation accuracy is improved.

[0126] In summary, by calculating the entropy corresponding to each of the three color channels of the X-ray image and correcting the setting of the X-ray image based on the entropy, the X-ray image can be effectively enhanced, which is helpful for its fusion with the CT image. Through the fused image, a better visual effect can be obtained, and the accuracy of the automatically segmented radiotherapy target area can be significantly improved. In addition, through the fused image, the doctor can obtain more comprehensive diagnostic information, so as to more accurately locate the tumor, evaluate the invasion range of the tumor and formulate a treatment plan. The target segmentation accuracy can be improved by extracting the second feature map of the slice with the target area based on the treatment dose, and fusing the second feature map with the slice with the target area. Similarly, the use of segmentation model and feature extraction model can also improve the target segmentation accuracy.

[0127] Embodiment 2

[0128] like Figure 3 As shown, in a second embodiment of the present invention, a radiotherapy target area automatic segmentation system based on image processing is provided, and the system comprises:

[0129] The processing module 10 is used to obtain a CT image and an X-ray image of the inspection part, calculate the entropy corresponding to each of the three color channels of the X-ray image, correct the X-ray image based on the entropy, and process the corrected X-ray image using a histogram normalization technique to obtain a grayscale image;

[0130] A calculation module 20 is used to decompose the first low-frequency sub-band image and the first high-frequency sub-band image of the CT image and decompose the second low-frequency sub-band image and the second high-frequency sub-band image of the grayscale image using a multi-scale geometric analysis tool, calculate the low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and calculate the high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image;

[0131] A slicing module 30, configured to obtain a fused image by reversely calculating the high-frequency fusion coefficient and the low-frequency fusion coefficient using a multi-scale geometric analysis tool, and to slice the fused image to form a plurality of slices;

[0132] The segmentation module 40 is used to obtain slices with target areas from the plurality of slices, obtain the radiotherapy dose corresponding to the slices with target areas, extract a second feature map of the slices with target areas based on the radiotherapy dose, fuse the second feature map with the slices with target areas, and segment the fused features.

[0133] The automatic segmentation system for radiotherapy target area based on image processing provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment.

[0134] Embodiment 4

[0135] like Figure 4 As shown, in the third embodiment of the present invention, the embodiment of the present invention provides the following technical solution, a computer, including a memory 202, a processor 201, and a computer program stored in the memory 202 and executable on the processor 201, and when the processor 201 executes the computer program, the automatic segmentation method of radiotherapy target area based on image processing as described above is implemented.

[0136] Specifically, the processor 201 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0137] Among them, the memory 202 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 202 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 202 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 202 may be inside or outside a data processing device. In a specific embodiment, the memory 202 is a non-volatile memory. In a specific embodiment, the memory 202 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (ErasableProgrammable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically ErasableProgrammable Read-Only Memory, EEPROM for short), an electrically alterable ROM (ElectricallyAlterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0138] The memory 202 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 201 .

[0139] The processor 201 reads and executes the computer program instructions stored in the memory 202 to implement the above-mentioned radiotherapy target area automatic segmentation method based on image processing.

[0140] In some embodiments, the computer may further include a communication interface 203 and a bus 200. Figure 4 As shown, the processor 201, the memory 202, and the communication interface 203 are connected via a bus 200 and communicate with each other.

[0141] The communication interface 203 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 203 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0142] The bus 200 includes hardware, software or both, and couples the components of the computer to each other. The bus 200 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 200 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 200 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0143] Embodiment 4

[0144] In the fourth embodiment of the present invention, in combination with the above-mentioned automatic segmentation method of radiotherapy target area based on image processing, the embodiment of the present invention provides the following technical solution, a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned automatic segmentation method of radiotherapy target area based on image processing is implemented.

[0145] Those skilled in the art will appreciate that the logic and / or steps described in the flowchart as data or in other ways herein, for example, can be considered as a sequenced data table of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses.

[0146] More specific examples of computer-readable media (non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0147] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0148] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A radiotherapy target area automatic segmentation method based on image processing, characterized in that: The method comprises: Acquire a CT image and an X-ray image of the inspection part, calculate the entropy corresponding to each of the three color channels of the X-ray image, correct the X-ray image based on the entropy, and process the corrected X-ray image using a histogram normalization technique to obtain a grayscale image; Decomposing the CT image by using a multi-scale geometric analysis tool to obtain a first low-frequency sub-band image and a first high-frequency sub-band image, and decomposing the grayscale image to obtain a second low-frequency sub-band image and a second high-frequency sub-band image, calculating a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and calculating a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image; Using a multi-scale geometric analysis tool to reversely calculate the high-frequency fusion coefficient and the low-frequency fusion coefficient to obtain a fused image, and slicing the fused image to form a plurality of slices; Obtain slices with target areas from the plurality of slices, obtain radiotherapy doses corresponding to the slices with target areas, extract second feature maps of the slices with target areas based on the radiotherapy doses, fuse the second feature maps with the slices with target areas, and segment the fused features.

2. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 1, characterized in that: The step of calculating the low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image and calculating the high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image comprises: Acquire a first saliency map of the first low-frequency sub-band image and a second saliency map of the second low-frequency sub-band image, and calculate the significance of the first saliency map and the second saliency map respectively by using a prognostic model; Based on the significance, the significance of the first significance map and the second significance map is divided into a first area and a second area, low-frequency fusion coefficients of the first area and the second area are calculated respectively, and high-frequency fusion coefficients of the first high-frequency sub-band image and the second high-frequency sub-band image are calculated using an absolute value formula.

3. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 2, characterized in that: The calculation formula of the low-frequency fusion coefficient of the first region is include: A1=W CT1 ×G CT1 +W X1 ×G X1 W CT =H CT / (H CT +H X ) W X =H C / (H CT +H X ) Where A1 represents the low-frequency fusion coefficient of the first region, W CT Represents the weight coefficient of the CT image, W X Represents the weight coefficient of the X-ray image, G CT1 represents the first low-frequency sub-band image of the corresponding CT image of the first region, G X1 represents the second low-frequency sub-band image of the corresponding X-ray fluoroscopic image of the first region, H CT represents the saliency of the CT image, H X Indicates the saliency of X-ray images; The calculation formula of the low-frequency fusion coefficient of the second area includes: <h2 style=";text-align:left;direction:ltr">A2=W<h2 style=";text-align:left;direction:ltr"> CT2 <h2 style=";text-align:left;direction:ltr"> ×G<h2 style=";text-align:left;direction:ltr"> CT2 <h2 style=";text-align:left;direction:ltr"> +W<h2 style=";text-align:left;direction:ltr"> X2 <h2 style=";text-align:left;direction:ltr"> ×G<h2 style=";text-align:left;direction:ltr"> X2 W bCT =(1-y)×H CT1 / (H CT1 +H X1 ) W bPET =(1-y)×H CT1 / (H CT1 +H X1 ) Where A2 represents the low-frequency fusion coefficient of the second region, G CT2 represents the first low-frequency sub-band image of the corresponding CT image of the second region, G X2 represents the second low-frequency sub-band image of the corresponding X-ray fluoroscopic image of the second region, H CT1 represents the saliency of the CT image, H X1 represents the saliency of the X-ray image, and y represents the global threshold; The absolute value formula includes: Among them, A3 represents the high-frequency fusion coefficient, G CT3 Represents the high-frequency sub-band image of the CT image, G X3 It represents the high-frequency sub-band image of the X-ray image, a represents the decomposition layer number, and b represents the filtering direction.

4. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 1, characterized in that: The step of obtaining the radiotherapy dose corresponding to the slice with the target area and extracting the second feature map of the slice with the target area based on the radiotherapy dose comprises: Using an exponential decay formula to calculate a corresponding radiotherapy dose based on the slice with the target area, and radiating the radiotherapy dose to the slice with the target area to obtain a dose map; The dose map is input into a feature extraction model to obtain a plurality of first feature maps of different sizes, and the plurality of first feature maps of different sizes are spliced ​​into a second feature map using a splicing formula.

5. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 4, characterized in that: The exponential decay formula includes: F (x) =F0e -ux In the formula, F (x) represents the beam intensity at position x, F0 represents the initial intensity of the beam, i.e., the beam intensity at x=0, u represents the linear attenuation coefficient, and e represents the base of the natural logarithm; The splicing formula includes: f(P)=(1-m)(1-n)f(D1)+m(1-n)f(D2)+(1-m)hf(D3)+mnf(D4) In the formula, f(P) represents the pixel value of the new point to be interpolated, f(D1), f(D2), f(D3) and f(D4) represent the four vertex pixel values ​​of the first feature map, m and n represent the interpolation weights, X and Y represent the horizontal and vertical coordinates of the target point, respectively. represents the horizontal coordinate of vertex D1, represents the horizontal coordinate of vertex D2, represents the ordinate of vertex D1, Indicates the ordinate of vertex D3.

6. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 1, characterized in that: The step of fusing the second feature map with the slice with the target area and segmenting the fused features comprises: Using a weighted formula, the second feature map is fused with the slice with the target area to obtain a fused feature, and the fused feature is input into a segmentation model to obtain a segmentation feature; The weighted formula includes: B=[B1*c1;B2*c2] In the formula, c1 represents the second feature map, c2 represents the slice with the target area, B1 represents the weight of the second feature map, B2 represents the weight of the slice with the target area, and B represents the fused feature after weighted fusion.

7. The method for automatic segmentation of radiotherapy target area based on image processing according to claim 1, characterized in that: The step of calculating the entropy corresponding to each of the three color channels of the X-ray fluoroscopic image and correcting the X-ray fluoroscopic image based on the entropy comprises: The entropy of the red channel, the green channel and the blue channel of the X-ray image are calculated respectively by using an entropy calculation formula, and the entropy of the red channel, the green channel and the blue channel are standardized to obtain a red judgment value, a green judgment value and the blue judgment value; The red judgment value or the blue judgment value is selected as a judgment basis value, and the judgment basis value is compared with an interval threshold value, and corresponding weights are assigned to the three color channels of the X-ray image based on the comparison result.

8. An automatic radiotherapy target area segmentation system based on image processing, characterized in that: The system comprises: A processing module, used for acquiring a CT image and an X-ray image of the inspection part, calculating the entropy corresponding to each of the three color channels of the X-ray image, correcting the X-ray image based on the entropy, and processing the corrected X-ray image using a histogram normalization technique to obtain a grayscale image; a calculation module, configured to decompose the CT image using a multi-scale geometric analysis tool to obtain a first low-frequency sub-band image and a first high-frequency sub-band image, and to decompose the grayscale image to obtain a second low-frequency sub-band image and a second high-frequency sub-band image, and to calculate a low-frequency fusion coefficient of the first low-frequency sub-band image and the second low-frequency sub-band image, and to calculate a high-frequency fusion coefficient of the first high-frequency sub-band image and the second high-frequency sub-band image; A slicing module, used for obtaining a fused image by reverse calculation based on the high-frequency fusion coefficient and the low-frequency fusion coefficient using a multi-scale geometric analysis tool, and slicing the fused image to form a plurality of slices; A segmentation module is used to obtain slices with target areas from the plurality of slices, obtain the radiotherapy dose corresponding to the slices with target areas, extract a second feature map of the slices with target areas based on the radiotherapy dose, fuse the second feature map with the slices with target areas, and segment the fused features.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for automatic segmentation of radiotherapy target area based on image processing as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for automatic segmentation of radiotherapy target area based on image processing as described in any one of claims 1 to 7 is implemented.

Images