Medical image preprocessing method, device and medium based on labeled images

Abstract

The present application discloses a medical image preprocessing method, device and medium based on annotated images. The annotated images are obtained by manually annotating the segmentation targets in the medical images. The preprocessing method includes the following steps: calculating a first mask based on the annotated image, wherein the foreground of the first mask has no intersection with the foreground of the annotated image; changing the distribution of pixel values ​​of the first region corresponding to the first mask in the medical image to weaken features similar to the segmentation target in the first region. Utilizing the solution of the present application, the first mask is obtained by calculating the annotated image, and the distribution of pixel values ​​of the first region is changed to weaken features similar to the segmentation target in the first region, thereby increasing the difficulty of identifying the segmentation target from the first region, thereby reducing the situation where the model identifies the under-annotated part as the segmentation target. When the preprocessed medical image is used for model training, the model is made to converge in the right direction.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a medical image preprocessing method, device, and medium based on annotated images. Background Art

[0002] With the advancement of science and technology and the innovation of artificial intelligence, deep learning methods have become widely used in the medical field to solve medical image segmentation problems. A large amount of training data is typically required to achieve excellent results and prevent overfitting. However, the annotation files for training samples are often manually annotated, which can easily lead to under-annotation. For example, in a lung CT image, the annotator may not have labeled the actual bronchi. This data is detrimental to model training and limits the model's resolution.

[0003] like Figure 1 As shown, deep learning is used to acquire a segmentation model. This involves training a neural network model using image data and label files, obtaining a suitable neural network model as a segmentation model. This segmentation model can be used to segment medical images in clinical practice. The image data includes medical images, and the label files are obtained by medical staff after manual annotation of the medical images. The label files are recorded as the true value Y. After the image data and label files are preprocessed and the model is trained, a predicted result X for the image data is output. A loss function is used to calculate the difference between the predicted result X and the true value Y output by the model training. The model parameters are modified based on the size of this difference, thus providing guidance for model optimization.

[0004] Specifically, model training can include the following steps: 1. data preparation, 2. model structure design, 3. loss function design, and 4. iterative training.

[0005] The data in step 1 is sample data, including medical images and manually annotated label files. Data preparation involves collecting this data and performing preprocessing. Preprocessing generally includes normalizing the image and expanding the sample size.

[0006] Step 2 is the structural design of the specific neural network model, such as the number of layers and the type of each layer.

[0007] Step 3 is the design of the loss function mentioned above, which is used to accurately calculate the difference between the predicted result X and the true value Y.

[0008] Step 4 is to input the medical image into the network model, calculate the loss function based on the network model's output X and the true value Y, and then adjust the optimization model parameters based on the loss function. The above process is repeated to iteratively train the model until the training stop condition is met.

[0009] It can be seen from this that if the sample data used for the above training is under-labeled, the area recognized by the model may not exist in the annotation, causing the calculated value of the loss function to deviate from the true value, giving the model the wrong guidance in the optimization direction.

[0010] Existing deep learning-based methods preprocess images to increase the amount of sample data. These methods perform unified preprocessing operations on the entire image, such as random flipping, rotation, and translation. While these methods can increase the number of training samples, they cannot address the problem of under-annotated training samples. Summary of the Invention

[0011] In order to solve the problem of insufficient annotation affecting the training of neural network models in the existing technology, this application proposes a medical image preprocessing method based on labeled images. The technical solution is as follows:

[0012] In one aspect, the present application provides a medical image preprocessing method based on annotated images, wherein the annotated images are obtained by manually annotating segmentation targets in the medical images. The preprocessing method comprises the following steps:

[0013] A first mask is obtained by calculating the annotated image, wherein the foreground of the first mask has no intersection with the foreground of the annotated image;

[0014] The distribution of pixel values ​​in a first region corresponding to a first mask in the medical image is changed to weaken features in the first region that are similar to the segmentation target.

[0015] Furthermore, the weakening methods include filtering, adding noise, and setting pixel values.

[0016] Furthermore, the step of calculating and obtaining the first mask according to the labeled image includes:

[0017] Performing morphological processing on the annotated image to obtain a second mask, wherein the foreground of the second mask includes the foreground of the annotated image;

[0018] Perform an inversion operation on the second mask to obtain the first mask.

[0019] Furthermore, the medical image preprocessing method based on the labeled image also includes:

[0020] The distribution of pixel values ​​in a second region corresponding to the second mask in the medical image is changed to enhance features in the second region that are similar to the segmentation target.

[0021] Furthermore, the medical image preprocessing method based on the labeled image also includes:

[0022] The distribution of pixel values ​​of a second region corresponding to the second mask in the medical image is changed to weaken features in the second region that are similar to the segmentation target, and the degree of weakening of the second region is less than that of the first region.

[0023] Furthermore, the annotated image is a bronchial annotated image, and the step of calculating and obtaining the first mask according to the annotated image includes:

[0024] According to the blood vessel mask, the small blood vessel mask is calculated;

[0025] The morphologically processed small blood vessel mask is subtracted from the morphologically processed annotated image to obtain the first mask.

[0026] Furthermore, after the step of calculating and obtaining the first mask according to the labeled image, the method further includes:

[0027] Performing an inversion operation on the first mask to obtain a second mask;

[0028] The distribution of pixel values ​​of a second region corresponding to the second mask in the medical image is changed so that the second region is easily identified as a segmentation target.

[0029] Furthermore, the step of calculating and obtaining a small blood vessel mask according to the blood vessel mask includes:

[0030] According to the diameter or radius of the blood vessel, the foreground of the blood vessel mask is screened to obtain a small blood vessel mask.

[0031] Furthermore, the step of calculating and obtaining a small blood vessel mask according to the blood vessel mask includes:

[0032] Based on the lung mask, a lung peripheral mask is obtained;

[0033] The lung peripheral mask and the blood vessel mask are summed to obtain the small blood vessel mask.

[0034] On the other hand, the present application provides an image processing device, which includes a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and runnable on the processor, wherein when the processor executes the program, it implements any of the above medical image preprocessing methods based on labeled images.

[0035] On the other hand, the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the steps of the above-mentioned medical image preprocessing method based on labeled images.

[0036] The beneficial effects of the present application are as follows: using the solution of the present application, a first mask is obtained by calculating the annotated image, the foreground of the first mask has no intersection with the foreground of the annotated image, and the foreground of the first mask corresponds to the first region of the medical image. By changing the distribution of pixel values ​​in the first region, the features similar to the segmentation target in the first region are weakened, thereby increasing the difficulty of identifying the segmentation target from the first region, thereby reducing the situation where the model identifies the under-annotated part as the segmentation target. When using the preprocessed medical image for model training, the model can converge in the right direction, improving the segmentation accuracy while accelerating the model convergence speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the framework of using deep learning to obtain a segmentation model in the prior art;

[0038] Figure 2 This is a flowchart of an embodiment of a medical image preprocessing method based on labeled images of the present application;

[0039] Figure 3 yes Figure 2 A sub-flowchart corresponding to step S10;

[0040] Figure 4 yes Figure 2 Another seed flow chart corresponding to step S10;

[0041] Figure 5 This is a flowchart of an embodiment of a medical image preprocessing method based on labeled images of the present application;

[0042] Figure 6A This is a detailed flowchart of an embodiment of a medical image preprocessing method based on labeled images of the present application;

[0043] Figures 6B-6G yes Figure 6A Schematic diagram of medical images and masks involved;

[0044] Figures 7A-7B yes Figure 6A Schematic diagram of comparison of pretreatment effects in ;

[0045] Figure 8A This is a detailed flowchart of another embodiment of the medical image preprocessing method based on labeled images of the present application;

[0046] Figures 8B-8N yes Figure 8A Schematic diagram of medical images and masks involved;

[0047] Figures 9A-9B yes Figure 8A Schematic diagram of comparison of pretreatment effects in ;

[0048] Figure 10 It is a structural diagram of the hardware operating environment of an embodiment of the medical image preprocessing method based on labeled images of the present application. DETAILED DESCRIPTION

[0049] To facilitate understanding of the present application, the present application is described in more detail below with reference to the accompanying drawings and specific embodiments. The accompanying drawings provide preferred embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described in this specification. Rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of the present application.

[0050] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this application. For example, the term "plurality" includes two or more.

[0051] On the one hand, the present application provides a medical image preprocessing method based on annotated images, which is used to achieve the effect that the under-annotated portions of the medical images preprocessed by the method are not easily identified as segmentation targets. Using the medical images preprocessed by the present application method as the training set and test set sample data for the training model, and training and testing the training model, will be more conducive to the training model converging in the right direction, while improving segmentation accuracy and accelerating the model convergence speed.

[0052] Specifically, refer to Figure 2 The labeled image is obtained by manually labeling the segmentation targets in the medical image. The preprocessing method includes the following steps:

[0053] S10: Obtain a first mask by calculation according to the annotated image, wherein the foreground of the first mask has no intersection with the foreground of the annotated image;

[0054] S20: changing the distribution of pixel values ​​in the first region corresponding to the first mask in the medical image to weaken features in the first region that are similar to the segmentation target.

[0055] Among them, medical images include medical imaging pictures collected through CT scans, MRI, ultrasound, etc., which are generally displayed as grayscale images without color information.

[0056] The annotated image is obtained by manually annotating the medical image and then segmenting the medical image based on the manual annotation results.

[0057] Manual annotation involves manually marking medical images with various annotations, such as red marking, circling, or dotting, to identify the segmentation target. For example, in lung CT images, the location of the bronchi is manually annotated. Bronchi typically have holes. Manual annotation methods include selecting a point within the bronchus, drawing a circle or box within the bronchus, or outlining the bronchus with a broken line or curve.

[0058] Since the manual annotations only indicate the location of the bronchi and often cannot accurately give the range of each bronchi, the medical image is segmented according to the manual annotation results to obtain an annotated image. Segmentation refers to a binary segmentation of the medical image, and the segmentation result can be represented by a binary image (i.e., an annotated image) of the same size as the medical image, where the pixel value of each pixel is used to indicate whether the corresponding pixel in the medical image belongs to the segmentation target. If it does, the pixel is called the foreground, and if it does not, the pixel is called the background. In the embodiments of the present application, the example of the foreground pixel value being 1 and the background pixel value being 0 is used for illustration. In fact, in other embodiments, the values ​​may be opposite, for example, the foreground pixel value is 0 and the background pixel value is 1, and the present application does not impose any restrictions on this.

[0059] Furthermore, when displaying, since the pixel value range of grayscale images is generally 0-255, rather than simply 0 and 1, in order to be able to intuitively see the labeled image in the image, the pixel value of 1 is generally reassigned to 255 when displayed. Figure 6B , which is the annotated image of the bronchi after the pixel values ​​are reassigned (taking the bronchi as an example, the annotated image can also be called a bronchial mask). Among them, for the purpose of clear display, the white part is the effect of reassigning the pixel point whose actual pixel value is 1 to 255. It should be noted that the actual value of the pixel value of the annotated image in this method is still 0 or 1. In this application, when displaying a binary image, 1 corresponds to white and 0 corresponds to black; in actual applications, it can also be reversed, that is, 1 corresponds to black and 0 corresponds to white.

[0060] Furthermore, the medical image preprocessing method based on annotated images of the present application obtains a first mask after performing certain calculations on the annotated image. This calculation may include morphological processing. The distribution of pixel values ​​in the first region determined by the first mask is then altered, thereby preventing the image within the first region from being recognized as a segmentation target.

[0061] Specifically, taking the above-mentioned bronchus as an example, Figure 6BAs shown, the white area in the annotated image corresponds to a portion of the bronchus, while the black area may include unlabeled bronchus. The method of the present application specifically changes the pixels of the medical image corresponding to the black area so that the bronchus contained therein is not identified as a bronchus.

[0062] Furthermore, the area of ​​pixels to be changed is generally smaller than the entire black area. That is, only the pixels of a portion of the medical image corresponding to the black area are changed. The portion of the medical image to be changed is defined as the first area in this application.

[0063] Furthermore, the weakening methods include filtering, adding noise, and setting pixel values.

[0064] For example, when the segmentation target is the bronchus, there is a large amount of air in the bronchus with a low HU value, while the HU value of the bronchial wall is high, which will appear as a hole in the CT image. Then, for the first region, filtering, adding noise, setting pixel values ​​(uniform value within the region, or random distribution) and other methods can be adopted to weaken or even destroy its hole characteristics. In this way, even if there are unmarked bronchi in the first region, they will not be recognized as bronchi by the model after processing.

[0065] Furthermore, the size of the first area should be neither too large nor too small. When the first area is too large, for example, it is directly equal to the overall black area, some pixels adjacent to the marked bronchi may be misoperated, for example, they may be treated as non-bronchial pixels and weakened, thereby reducing the segmentation accuracy and being detrimental to model convergence. When the first area is too small, due to the lack of comprehensive coverage, some unlabeled (under-labeled) bronchi are not weakened, resulting in the presence of a large number of under-labeled bronchi in the entire medical image that can be identified, which also affects the segmentation accuracy and is detrimental to model convergence.

[0066] Furthermore, this application uses a first mask to determine the size and boundaries of the first region. Taking the bronchi described above as an example, the foreground of the annotated image (bronchial mask) includes the manually annotated bronchi, while the foreground of the first mask includes the unannotated (underannotated) regions. Specifically, the size of the first mask should be, as previously described, neither too large nor too small.

[0067] Furthermore, in order to ensure that the marked segmentation targets are not affected, appropriate morphological processing can be performed on the marked image, such as dilating the foreground and then negating it to obtain a first mask with the largest range. Taking the bronchial mask as an example, Figure 6C is the image after the bronchial mask is expanded, Figure 6D for Figure 6C The image after negation. Figure 6DIt is a first mask with a maximum range. In the medical image corresponding to the white area of ​​the first mask, the corresponding pixel distribution value can be changed without affecting the marked segmentation target (the marked bronchi).

[0068] Furthermore, the range of the first mask may be further narrowed according to other medical prior knowledge, thereby obtaining a more accurate preprocessing effect.

[0069] Further, refer to Figure 3 and Figure 4 , two methods for calculating and obtaining the first mask based on the annotated image are given, thereby obtaining two first regions of different sizes.

[0070] like Figure 3 As shown, the step of calculating and obtaining the first mask according to the labeled image includes:

[0071] S101: performing morphological processing on the annotated image to obtain a second mask, where the foreground of the second mask includes the foreground of the annotated image;

[0072] S102: Perform an inversion operation on the second mask to obtain a first mask.

[0073] The foreground in the second mask corresponds to the segmented target area; the foreground in the first mask corresponds to the under-labeled non-segmented target area.

[0074] Specifically, the morphological processing here includes dilation. Dilation uses a specific algorithm to expand the edges of an image, either expanding the edges of an object or filling internal gaps. This results in the foreground area of ​​the second mask being slightly larger than that of the original annotated image. Correspondingly, the inversion operation here refers to inverting all pixel values ​​in the original image, for example, reassigning 0 to 1 and 1 to 0. This results in the foreground area of ​​the first mask corresponding to the area with the foreground of the second mask removed.

[0075] Further, refer to Figure 5 , medical image preprocessing methods based on labeled images also include:

[0076] S30: changing the distribution of pixel values ​​in the second region corresponding to the second mask in the medical image to enhance features in the second region that are similar to the segmentation target.

[0077] Specifically, the enhancement here includes enhancing edge characteristics of the image.

[0078] By enhancing the labeled segmentation targets, the features of the labeled parts in the same image can be made easier to identify, while the features of the unlabeled parts can be more easily ignored.

[0079] Further, refer to Figure 5 , medical image preprocessing methods based on labeled images also include:

[0080] S40: changing the distribution of pixel values ​​of a second region corresponding to the second mask in the medical image to weaken features in the second region that are similar to the segmentation target, wherein the degree of weakening of the second region is less than that of the first region.

[0081] Specifically, taking the above-mentioned bronchial segmentation as an example, if Gaussian filtering is used to weaken the features of the bronchi in the first and second regions, the size and / or standard deviation of the Gaussian kernel used in the first region must be larger than that in the second region, so that the unlabeled segmentation targets in the first region are less likely to be recognized by the model.

[0082] like Figure 4 As shown, the annotated image is a bronchial mask, and the steps of calculating and obtaining the first mask according to the annotated image include:

[0083] S111: Calculate and obtain a small blood vessel mask based on the blood vessel mask;

[0084] S112: Subtract the morphologically processed annotated image from the morphologically processed small blood vessel mask to obtain a first mask.

[0085] This is based on the prior knowledge that blood vessels generally accompany trachea. A larger-scale operator (also called structural element) is used to expand the blood vessels to include the accompanying bronchi, thereby obtaining a more accurate first mask.

[0086] Specifically, lung images typically include trachea and blood vessels. Bronchus are often located in close proximity to small blood vessels, facilitating the exchange of carbon dioxide in the alveoli and oxygen in the blood. Therefore, it can be assumed that the range of the bronchus can be further narrowed to the corresponding area of ​​the small blood vessel mask. Subtracting the bronchial mask from the small blood vessel mask yields a more accurate first mask.

[0087] Before the subtraction operation, to ensure more accurate segmentation, each mask can be morphologically processed, such as by dilating each mask. Specifically, during dilation, a relatively large-scale operator can be selected for the small vessel mask, while a relatively small-scale operator can be selected for the bronchial mask. This ensures that the dilated small vessel mask encompasses the bronchial mask, resulting in less error when the dilated bronchial mask is subtracted from the dilated small vessel mask.

[0088] Furthermore, this application provides two methods for obtaining small blood vessel masks, as follows:

[0089] Preferably, the step of calculating and obtaining the small blood vessel mask according to the blood vessel mask includes:

[0090] According to the diameter or radius of the blood vessel, the foreground of the blood vessel mask is screened to obtain a small blood vessel mask.

[0091] This method is based on the fact that the size of the small blood vessels in the lungs must be smaller than a certain threshold, so that the small blood vessel area can be directly screened.

[0092] Preferably, the step of calculating and obtaining the small blood vessel mask according to the blood vessel mask includes:

[0093] Based on the lung mask, a lung peripheral mask is obtained;

[0094] The lung peripheral mask and the blood vessel mask are summed to obtain the small blood vessel mask.

[0095] This method uses the fact that small pulmonary blood vessels are generally located in the peripheral region of the lungs. Therefore, the overlap between the peripheral lung region and the entire pulmonary blood vessels is considered the small blood vessel region, and thus an AND operation is performed. The AND operation involves adding the pixel values ​​of the two masks. For example, the ANDs of 0 and 1, 1 and 0, and 0 and 0 all become 0, while only the AND of 1 and 1 becomes 1. As a result, the foreground of the resulting small blood vessel mask only retains the portion where both the peripheral lung mask and the blood vessel mask are 1, i.e., the overlap between the two foregrounds.

[0096] Furthermore, after the step of calculating and obtaining the first mask according to the labeled image, the method further includes:

[0097] S113: performing an inversion operation on the first mask to obtain a second mask;

[0098] Furthermore, after obtaining the second mask, the method further includes:

[0099] The distribution of pixel values ​​of a second region corresponding to the second mask in the medical image is changed so that the second region is easily identified as a segmentation target.

[0100] Specifically for the image processing of the second area, refer to the aforementioned methods, including enhancement, weakening, etc., which will not be repeated here.

[0101] Further, refer to Figures 6A-9B ,This specification takes bronchial image segmentation as an example, and gives two different ,implementations.

[0102] like Figure 6A As shown, a specific process for preprocessing medical images for bronchial segmentation in lung CT images (hereinafter referred to as CT images) is provided, and the steps are as follows:

[0103] 1. For bronchial mask, that is, annotated image (such as Figure 6B As shown) is expanded to obtain the second mask (as shown Figure 6C ).

[0104] 2. Perform the inversion operation on the second mask to obtain the first mask (such as Figure 6D ).

[0105] 3. Perform image enhancement on the original CT image to make the marked segmentation target (bronchus) more obvious, for example, the edge of the hole is more obvious.

[0106] 4. Multiply the enhanced CT image with the second mask to obtain the enhanced second region image (such as Figure 6F Multiplication involves multiplying the pixels of corresponding coordinates in the two images. For example, if the foreground pixel value of the second mask is 1 and the background pixel value is 0, after multiplication with the enhanced CT image, the resulting enhanced CT image will only contain the area contained in the foreground of the second mask, with all other pixel values ​​being 0. For ease of visualization, the background is shown in white.

[0107] 5. Weaken the original CT image to make the unlabeled segmentation target (bronchus) more blurred, for example, the edge of the hole becomes more blurred or even disappears.

[0108] 6. Multiply the weakened CT image with the first mask to obtain a weakened first region image (e.g. Figure 6E The principle of multiplication is the same as step 4 and will not be repeated here.

[0109] 7. Add the enhanced second region image and the weakened first region image to obtain the final pre-processed CT image (e.g. Figure 6G Addition is the process of adding the pixels of the corresponding coordinates of the two images. Since the first mask and the second mask are inversely related, the addition of the two images is exactly the complete CT image.

[0110] refer to Figures 7A-7B , shows the effect diagram of the above pretreatment, where the arrows point to the unlabeled bronchus.

[0111] like Figure 7A As shown in the figure, before pretreatment, the holes of the unmarked bronchus have clear edges and distinct features. Figure 7B As shown in the figure, after pretreatment, the edges of the holes of the unmarked bronchus are blurred or even disappear.

[0112] like Figure 8A As shown, another specific process for preprocessing medical images for bronchial segmentation in lung CT images (hereinafter referred to as CT images) is provided, and the steps are as follows:

[0113] 1. Mask the lungs (eg Figure 8BThe closed mask of the lungs is obtained (as shown on the left). Figure 8B (As shown on the right). A closing operation involves dilation followed by erosion. It fills small holes within an object, connects adjacent objects, and smoothes boundaries. An opening operation involves erosion followed by dilation. It eliminates small objects, separates objects at thin points, and smoothes the boundaries of larger objects. Erosion, on the other hand, uses a specific algorithm to remove edges, removing "burrs" from the object's edges. This is the opposite of the dilation effect described above.

[0114] 2. Perform an erosion operation on the lung closed mask obtained in step 1 to obtain a lung erosion mask (such as Figure 8C shown).

[0115] 3. Perform an XOR operation on the lung closed mask obtained in step 1 and the lung erosion mask obtained in step 2 to obtain the lung peripheral mask (i.e. the peripheral lung area, such as Figure 8D XOR refers to performing XOR on pixels with the same coordinates in two images to obtain the difference between the two images and remove the same parts. For example, 0 XOR equals 1, 1 XOR equals 0, 1 XOR equals 1, 1 XOR equals 1, and 0 XOR equals 0.

[0116] 4. Add the entire lung vessel mask to the lung peripheral mask obtained in step 3 to obtain the small vessel mask (e.g. Figure 8E The medical principles on which this is based have been discussed above and will not be repeated here.

[0117] 5. Expand the small blood vessel mask to obtain the expanded small blood vessel mask (such as Figure 8F shown).

[0118] 6. Expand the bronchial mask (original annotated image) to obtain the expanded bronchial expansion mask (e.g. Figure 8H shown).

[0119] 7. Subtract the bronchial expansion mask from the small blood vessel expansion mask to obtain the first mask (e.g. Figure 8I ). Specifically, pixel values ​​that are negative after subtraction can be set to 0. Accordingly, in step 10, the first mask is inverted to obtain the second mask.

[0120] 8. Image weakening of original CT images (such as Figure 8J As shown in Figure 2), the unlabeled segmentation target (bronchus) becomes more blurred, for example, the edge of the hole becomes more blurred or even disappears (as shown in Figure 2). Figure 8K Specific weakening can be achieved by using Gaussian smoothing or adding noise.

[0121] 9. The weakened CT image (such as Figure 8K As shown) and the first mask (as shown Figure 8I )

[0122] Perform multiplication operation to obtain the weakened first region image (such as Figure 8L ). The principle of multiplication has been discussed above and will not be repeated here.

[0123] 10. Invert the first mask to obtain the second mask.

[0124] 11. Multiply the original CT image with the second mask to obtain the second region (e.g. Figure 8M shown).

[0125] 12. Add the second region image to the weakened first region image to obtain the final pre-processed CT image (e.g. Figure 8N ).

[0126] refer to Figures 9A-9B , shows the effect diagram of the above pretreatment, where the arrows point to the unlabeled bronchus.

[0127] like Figure 9A As shown in the figure, before pretreatment, the holes of the unmarked bronchus have clear edges and distinct features. Figure 9B As shown in the figure, after pretreatment, the edges of the holes of the unmarked bronchus are blurred or even disappear.

[0128] It should be noted that the multiplication operation in the above detailed process is performed after the medical image is morphologically processed, which is only one of the implementation methods. This method takes into account that in addition to the unified set value, certain other image processing (such as filtering) needs to utilize the surrounding pixels. In order to make the edge effect of the first area or the second area smoother, the overall image processing is performed first, such as filtering, and then multiplication is performed to obtain the first area or the second area. Other methods of multiplying first and then performing morphological processing can also implement the solution of this application, which is not limited here.

[0129] Furthermore, both the labeled image and the segmented image can be used as masks in image processing, representing parallel concepts. In the above embodiments, except for the bronchial mask which is manually labeled, the other lung masks, vascular masks, etc. can be segmented images directly recognized by the machine.

[0130] Furthermore, the application ratio of the preprocessing method described in the present application can be adjusted, that is, the preprocessing operation does not necessarily have to be applied to all samples, and a certain ratio of the training data can be retained to maintain the original characteristics.

[0131] Furthermore, the preprocessing method described in this application can adjust the preprocessing intensity for the foreground and background, such as the degree of filtering and smoothing, the density of noise, the parameters of the operator, etc., so as to obtain preprocessed images with different effects, and can further expand the sample data and increase the samples of the training set and test set required for model training. In the field of medical imaging, such sample images are a very scarce resource.

[0132] On the other hand, the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned medical image preprocessing method based on labeled images are implemented.

[0133] Specific reference Figure 10 In practical applications, Figure 10 This is a structural diagram of the hardware operating environment involved in the medical image preprocessing method based on labeled images in this application.

[0134] like Figure 10 As shown, the hardware operating environment may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0135] Those skilled in the art will understand that Figure 10 The hardware structure for running the medical image preprocessing method based on labeled images shown in the figure does not constitute a limitation on the device for running the medical image preprocessing method based on labeled images, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0136] like Figure 10As shown, memory 1005, a readable storage medium, may include an operating system, a network communication module, a user interface module, and a medical image preprocessing program based on annotated images. The operating system is a management and control program that supports the operation of the network communication module, the user interface module, the medical image preprocessing program based on annotated images, and other programs or software. The network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0137] exist Figure 10 In the hardware structure shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the medical image preprocessing program based on annotated images stored in the memory 1005 and execute the aforementioned medical image preprocessing method based on annotated images.

[0138] In summary, the present application has the following beneficial effects: by calculating the annotated image, a first mask is obtained, the foreground of the first mask has no intersection with the foreground of the annotated image, and the foreground of the first mask corresponds to the first region of the medical image. By changing the distribution of pixel values ​​in the first region, the features in the first region that are similar to the segmentation target are weakened, thereby increasing the difficulty of identifying the segmentation target from the first region, thereby reducing the situation where the model identifies the under-annotated part as the segmentation target. When using preprocessed medical images for model training, the model converges in the right direction, improving the segmentation accuracy while accelerating the model convergence speed.

[0139] Furthermore, in order to ensure that the marked segmentation targets are not affected, appropriate morphological processing is performed on the marked image, such as the dilation and then the inversion of the foreground to obtain a first mask with the maximum range. In the medical image area corresponding to the foreground of the first mask, corresponding pixel distribution values ​​can be changed without affecting the marked segmentation targets.

[0140] Furthermore, the range of the first mask is further narrowed based on other medical prior knowledge, thereby obtaining a more accurate preprocessing effect.

[0141] Furthermore, by enhancing the labeled segmentation targets, the features of the labeled parts in the same image can be made easier to identify, while the features of the unlabeled parts can be more easily ignored.

[0142] Furthermore, the features in the second region that are similar to the segmentation target may be weakened to a lesser extent than that in the first region, so that the unlabeled segmentation targets in the first region are less likely to be recognized by the model.

[0143] Furthermore, the application ratio of the preprocessing method can be adjusted, that is, the preprocessing operation does not necessarily have to be applied to all samples, and a certain ratio of the training data can be retained to maintain the original characteristics.

[0144] Furthermore, the preprocessing method described in this application can adjust the preprocessing intensity for the foreground and background, such as the degree of filtering and smoothing, the density of noise, the parameters of the operator, etc., so as to obtain preprocessed images with different effects, and can further expand the sample data and increase the very scarce resources such as the training set and test set samples required for model training.

Claims

1. A medical image preprocessing method based on labeled images, characterized in that: The annotated image is obtained by manually annotating the segmentation targets in the medical image. The preprocessing method includes the steps of: Obtaining a first mask by calculation according to the annotated image, wherein a foreground of the first mask has no intersection with a foreground of the annotated image; changing a distribution of pixel values ​​of a first region corresponding to the first mask in the medical image to weaken features in the first region that are similar to the segmentation target; The step of calculating and obtaining the first mask according to the labeled image includes: performing morphological processing on the annotated image to obtain a second mask, wherein the foreground of the second mask includes the foreground of the annotated image; Performing an inversion operation on the second mask to obtain the first mask.

2. The medical image preprocessing method based on labeled images according to claim 1, characterized in that: The weakening methods include filtering, adding noise, and setting pixel values.

3. The medical image preprocessing method based on labeled images according to claim 1, characterized in that: The medical image preprocessing method based on annotated images further includes: changing the distribution of pixel values ​​in a second region corresponding to the second mask in the medical image to enhance features in the second region that are similar to the segmentation target.

4. The medical image preprocessing method based on labeled images according to claim 1, characterized in that: The medical image preprocessing method based on annotated images also includes: changing the distribution of pixel values ​​of a second area corresponding to the second mask in the medical image to weaken features in the second area that are similar to the segmentation target, and the degree of weakening of the second area is less than the degree of weakening of the first area.

5. The medical image preprocessing method based on labeled images according to claim 1, characterized in that: The annotated image is a bronchial annotated image, and the step of calculating and obtaining a first mask according to the annotated image includes: According to the blood vessel mask, the small blood vessel mask is calculated; The first mask is obtained by subtracting the morphologically processed annotated image from the morphologically processed small blood vessel mask.

6. The medical image preprocessing method based on labeled images according to claim 5, characterized in that: After the step of calculating and obtaining the first mask according to the labeled image, the method further includes: performing an inversion operation on the first mask to obtain a second mask; The distribution of pixel values ​​of a second region corresponding to the second mask in the medical image is changed so that the second region is easily identified as a segmentation target.

7. The medical image preprocessing method based on labeled images according to claim 5, characterized in that: The step of calculating and obtaining the small blood vessel mask based on the blood vessel mask includes: screening the foreground of the blood vessel mask according to the diameter or radius of the blood vessel to obtain the small blood vessel mask.

8. The medical image preprocessing method based on labeled images according to claim 5, characterized in that: The step of calculating and obtaining a small blood vessel mask based on the blood vessel mask includes: Based on the lung mask, a lung peripheral mask is obtained; An AND operation is performed on the lung peripheral mask and the blood vessel mask to obtain the small blood vessel mask.

9. An image processing device, characterized in that The image processing device includes a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein when the processor executes the program, the medical image preprocessing method based on annotated images according to any one of claims 1 to 8 is implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the medical image preprocessing method based on annotated images are implemented as described in any one of claims 1 to 8.

Images