A method, device and storage medium for multi-organ CT image data augmentation

Abstract

This application relates to a multi-organ CT image data augmentation method, device, and storage medium. The method involves collecting original CT images and standardizing them to form a dataset. Standardized images are extracted from the dataset, and the similarity between organs is calculated. Organs with high similarity are selected as replacement organs and placed in a background image. A search space is defined within the background image. The replacement organs are moved within the search space, and the Intersection over Union (IoU) value is calculated. The replacement organs are then moved based on the IoU value to obtain augmented images. This application effectively addresses the high dependency and difficulty of scalability of manual methods, significantly improving sample diversity and increasing medical credibility, while enhancing the generalization and accuracy of images.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to a method, device, and storage medium for multi-organ CT image data augmentation. Background Art

[0002] The quality of medical images is affected by multiple factors, such as the advanced equipment, the operator's experience, and the equipment's proper operating condition. Therefore, the overall cost of obtaining a high-quality medical image often exceeds the understanding of non-professionals. In fact, many medical institutions cannot meet even one of these factors. While obtaining high-quality medical images is difficult, this does not mean that diagnostic accuracy is entirely limited by image quality. Existing technologies offer a variety of methods to address deficiencies in imaging quality. For example, experts, based on clinical experience, can manually adjust the size, shape, and surrounding tissue of organs to simulate anatomical variations among different patients. Another example is the ability to increase the diversity of image data through random transformations using traditional graphics enhancement methods, or to generate new image samples through deep learning.

[0003] However, existing technologies face the following challenges: (1) Manual methods based on expert experience are difficult to scale to diverse anatomical scenarios. (2) Traditional graphics enhancement methods have poor perception of anatomical relationships and spatial constraints. (3) Deep learning-driven image enhancement methods require large amounts of labeled data for training, while medical images themselves suffer from insufficient sample quality. Summary of the Invention

[0004] The present application provides a multi-organ CT image data augmentation method, device, and storage medium that can adapt to anatomical scenes and perceive spatial constraints without relying on high-quality medical images.

[0005] A multi-organ CT image data augmentation method comprises the following steps:

[0006] Step 1: collecting original CT images and preprocessing the original CT images to obtain processed CT images; rotating and flipping the processed CT images, and uniformly adjusting the processed CT images to anatomical directions to obtain standardized images; in each of the standardized images, the pixels of each organ have unique coordinates;

[0007] Step 2: Use LabelMe software to assign a unique code to each organ in the standardized image. The coded organ and the corresponding standardized image form an image pair, and all image pairs constitute a data set.

[0008] Step 3: Extract the standardized images pairwise to the same spatial coordinate system, calculate the IoU value of the same type of organs in the two standardized images, and use the IoU value to represent the similarity of the same type of organs;

[0009] The formula for calculating the IoU value is as follows:

[0010] ;

[0011] In the formula, and Respectively indicate whether the same organs in the two standardized images are in the spatial coordinate system pixel coordinates If it exists, it takes 1, if it does not exist, it takes 0; A and B Respectively represent the codes of the same organs in two standardized images, Represents the coordinates of a pixel point in a spatial coordinate system; the symbol · represents a union operation;

[0012] Step 4: Record the IoU value of each organ in each set of standardized images and set the IoU value threshold. Select and retain the standardized image groups with a value greater than the IoU threshold.

[0013] Step 5: Select a standardized image as the background image, and select the one with the highest IoU value among the standardized images in the background image as the replacement image; use the center point of the organ to be replaced in the background image as the initial position, and move the center point of the replacement organ in the replacement image to the initial position;

[0014] Step 6: Define a search space with the initial position as the origin. x , y , z The displacement step is set in three directions, and all displacement step combinations in the search space are traversed. The replacement organs are moved in sequence. After each movement, the IoU value between the replacement organ and the organ to be replaced is calculated until all the IoU values ​​corresponding to the displacement step combinations are calculated.

[0015] Step 7: Select the displacement step size combination corresponding to the maximum IoU value in step 6 to move the replacement organ, and the replacement organ and the background image form an augmented image.

[0016] Furthermore, the preprocessing of the original CT image in step 1 includes denoising, image equalization, and contrast enhancement.

[0017] Furthermore, in step six, the initial value of the displacement step is 5 pixels. After the second movement is completed, the IoU value of this movement is compared with the IoU value when the previous movement is completed. If the IoU value after the current movement is completed is greater than the IoU value when the previous movement is completed, the displacement step is reduced to 1 pixel; if the IoU value after the current movement is completed is less than or equal to the IoU value when the previous movement is completed, the movement continues with a displacement step of 5 pixels.

[0018] A multi-organ CT image data augmentation device includes a processor and a memory storing program instructions.

[0019] The processor is configured to execute the above-mentioned multi-organ CT image data augmentation method when running the program instructions.

[0020] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the multi-organ CT image data augmentation method as described above.

[0021] The beneficial effects of this application are:

[0022] 1. This application can generate multi-organ CT images with correct anatomical positions and reasonable sizes, can adapt to the differences between different individuals, and effectively solve the problems of strong dependence and difficulty in scalability of manual methods.

[0023] 2. This application performs matching and displacement at the organ scale, which can significantly improve sample diversity and increase medical credibility.

[0024] 3. This application significantly reduces the reliance on large-scale annotated datasets, and can generate multiple images from fewer samples, thereby improving the generalization ability and accuracy of images. The augmented images can be used for subsequent image segmentation and other processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a standardized image sample in this application;

[0026] Figure 2 This is a schematic diagram of similarity calculation in this application;

[0027] Figure 3 Schematic diagram of the search space in this application. DETAILED DESCRIPTION

[0028] Based on the background technology, the present application provides a method for multi-organ CT image data augmentation, comprising the following steps:

[0029] Step 1: Collect the original CT images and preprocess them to obtain processed CT images; rotate and flip the processed CT images, and uniformly adjust the processed CT images to the anatomical direction to obtain standardized images; in each of the standardized images, the pixels of each organ have unique coordinates.

[0030] As attached Figure 1 As shown in the figure, preprocessing of the original image aims to improve image quality and consistency. This includes spatial filter denoising, image equalization using logarithmic transformation and histogram processing, and image contrast enhancement to enhance detail in dark areas. The processed CT image also undergoes normalization, specifically by rotating and flipping it to align it with the anatomical orientation.

[0031] Step 2: Use LabelMe software to set a unique code for each organ in the standardized image. The coded organ and the standardized image corresponding to the organ form an image pair, and all image pairs constitute a data set.

[0032] As attached Figure 1 As shown in Figure 1, organs of the same type have the same code in multiple standardized images. LabelMe software is used to label the standardized images pixel by pixel. Standardized images are always located in a spatial coordinate system, so each pixel can be represented by a three-dimensional coordinate. After encoding, both the image and the organ within it can be identified by the code. All images and organs constitute a dataset, which serves as the input for augmented data.

[0033] Step 3: Extract the standardized images pairwise to the same spatial coordinate system, calculate the IoU value of the same type of organs in the two standardized images, and use the IoU value to represent the similarity of the same type of organs;

[0034] The formula for calculating the IoU value is as follows:

[0035] ;

[0036] In the formula, and Respectively indicate whether the same organs in the two standardized images are in the spatial coordinate system pixel coordinates If it exists, it takes 1, if it does not exist, it takes 0; A and B Respectively represent the codes of the same organs in two standardized images, Represents the coordinates of a pixel point in a spatial coordinate system; the symbol · represents a union operation.

[0037] Step 4: Record the IoU value of each organ in each set of standardized images and set the IoU value threshold. Select and retain the standardized image groups with an IoU value greater than the threshold.

[0038] As attached Figure 2 As shown, standardized images are extracted pairwise and compared with each organ in turn, so that each organ in each image has multiple similar organs to be compared. The purpose of this step is to generate more augmented data. When calculating the IoU value, the larger the IoU value, the greater the similarity between the two organs. Considering the differences in organ size and shape in different images, unreasonable combinations will be generated when matching images, such as a large liver and a small liver. For such unreasonable combinations, this application sets an IoU threshold to eliminate such unreasonable combinations. In this embodiment, the IoU value is set to 0.9.

[0039] Step 5: Select a standardized image as the background image, and select the one with the highest IoU value in the standardized image group in the background image as the replacement image; use the center point of the organ to be replaced in the background image as the initial position, and move the center point of the replacement organ in the replacement image to the initial position.

[0040] In step 4, the size and shape of the organs are matched. In order to accurately determine the position of similar organs, this application needs to translate the replacement organs. The coordinates of the center point in step 5 can be determined by the pixel coordinates of the organ to be replaced. The pixel points on the edge of the coordinate axis correspond to x coordinate, y coordinate, z The coordinates of the center point can be obtained by algebraically adding the coordinates. After determining the center point of the organ to be replaced, use this as the initial position. Then calculate the center point of the replacement organ, and start the translation after its center point coincides with the initial position.

[0041] Step 6: Define a search space with the initial position as the origin. x , y , z The displacement steps are set in three directions, and all displacement step combinations in the search space are traversed. The replacement organs are moved in turn, and the IoU value between the replacement organ and the organ to be replaced is calculated after each movement is completed until the IoU values ​​corresponding to all displacement step combinations are calculated.

[0042] As attached Figure 3 As shown, in the search space x , y , z The displacement steps are set in three directions. A point cloud (not shown) is used as a normalized image in 3D space. Black dots represent the replacement organ mask, the symbol x represents the set search space, and the triangle symbol represents the final displacement step. All displacement step combinations in the search space are shifted sequentially. After the shift is completed, the IoU value with the replacement organ is recorded until all displacement step combinations have been shifted.

[0043] The calculation principle of the IoU value in step 6 is the same as that in step 3. The difference is that the organ in step 3 has no background image. Step 3 extracts the organ for similarity judgment, while step 6 moves the replacement organ in the background image. The replacement organ has an initial position, so the position of the organ is different, and the calculated IoU value is also different.

[0044] Step 7: Select the displacement step size combination corresponding to the maximum IoU value in step 6 to move the replacement organ, and the replacement organ and the background image form an augmented image.

[0045] In step 6, the initial displacement step is 5 pixels. After the second movement is completed, the IoU value of the current movement is compared with the IoU value at the completion of the previous movement. If the IoU value after the current movement is greater than the IoU value at the completion of the previous movement, the displacement step is reduced to 1 pixel. If the IoU value after the current movement is less than or equal to the IoU value at the completion of the previous movement, the movement is continued with a displacement step of 5 pixels. This step is used to speed up the calculation process.

[0046] The disclosed embodiments also provide a multi-organ CT image data augmentation device, comprising a processor and memory. Optionally, the device may also include a communication interface and a bus. The processor, communication interface, and memory may communicate with each other via the bus. The communication interface may be used for information transmission. The processor may invoke logic instructions in the memory to execute the multi-organ CT image data augmentation method described in the aforementioned embodiments.

[0047] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0048] The memory, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor executes the program instructions / modules stored in the memory to perform functional applications and data processing, thereby implementing the multi-organ CT image data augmentation method described in the above embodiments.

[0049] The memory may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory may include high-speed random access memory and non-volatile memory.

[0050] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned multi-organ CT image data augmentation method.

[0051] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0052] The technical solutions of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, and other media that can store program code, or a transient storage medium.

[0053] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A multi-organ CT image data augmentation method, characterized in that: The following steps are involved: Step 1: Collect the original CT image and preprocess the original CT image to obtain the processed CT image; Rotating and flipping the processed CT images to uniformly adjust the processed CT images to anatomical orientations to obtain standardized images; in each of the standardized images, pixels of each organ have unique coordinates; Step 2: Use LabelMe software to assign a unique code to each organ in the standardized image. The coded organ and the corresponding standardized image form an image pair, and all image pairs constitute a data set. Step 3: Extract the standardized images pairwise to the same spatial coordinate system, calculate the IoU value of the same type of organs in the two standardized images, and use the IoU value to represent the similarity of the same type of organs; The formula for calculating the IoU value is as follows: ； In the formula, and Respectively indicate whether the same organs in the two standardized images are in the spatial coordinate system pixel coordinates If it exists, it takes 1, if it does not exist, it takes 0; A and B Respectively represent the codes of the same organs in two standardized images, Represents the coordinates of a pixel point in a spatial coordinate system; the symbol · represents a union operation; Step 4: Record the IoU value of each organ in each set of standardized images and set the IoU value threshold. Select and retain the standardized image groups with a value greater than the IoU threshold. Step 5: Select a standardized image as the background image, and select the one with the highest IoU value among the standardized images in the background image as the replacement image; use the center point of the organ to be replaced in the background image as the initial position, and move the center point of the replacement organ in the replacement image to the initial position; Step 6: Define a search space with the initial position as the origin. x , y , z The displacement step is set in three directions, and all displacement step combinations in the search space are traversed. The replacement organs are moved in sequence. After each movement, the IoU value between the replacement organ and the organ to be replaced is calculated until all the IoU values ​​corresponding to the displacement step combinations are calculated. Step 7: Select the displacement step size combination corresponding to the maximum IoU value in step 6 to move the replacement organ, and the replacement organ and the background image form an augmented image.

2. The multi-organ CT image data augmentation method according to claim 1, characterized in that: In step 1, the original CT image is preprocessed, including denoising, image equalization, and contrast enhancement.

3. The multi-organ CT image data augmentation method according to claim 1, characterized in that: In step 6, the initial value of the displacement step is 5 pixels. After the second movement is completed, the IoU value of this movement is compared with the IoU value when the previous movement is completed. If the IoU value after the current movement is completed is greater than the IoU value when the previous movement is completed, the displacement step is reduced to 1 pixel; if the IoU value after the current movement is completed is less than or equal to the IoU value when the previous movement is completed, the movement continues with a displacement step of 5 pixels.

4. A multi-organ CT image data augmentation device, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the multi-organ CT image data augmentation method according to any one of claims 2 or 3 when running the program instructions.

5. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the method according to any one of claims 2 or 3 is implemented.

Images