Image processing method and device, equipment, storage medium and program product

Abstract

The invention provides an image processing method and device, equipment, a storage medium and a program product, and relates to the field of medical image processing, and the method comprises the steps: obtaining an original medical image of a detected object; performing first segmentation processing on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result; performing classification processing on the first target object in the original medical image to obtain a classification result; based on the classification result, performing second segmentation processing on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result; and on the basis of the first marking result and the second marking result, predicting the original medical image of the detected object to obtain a prediction result, and through automatic processing, improving the prediction efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an image processing method, device, equipment, storage medium and program product. Background Art

[0002] Bone mineral density (BMD) is an important indicator for measuring bone strength and osteoporosis. BMD is usually measured by dual-energy X-ray absorptiometry (DXA), quantitative computed tomography (QCT) and other methods. However, in clinical practice, it mainly has the following problems:

[0003] (1) When patients have aortic calcification, severe bone spurs, sclerosis, obesity and other diseases, DXA will have large errors.

[0004] (2) Although BMD measurement by QCT is less susceptible to interference from other factors, it is more expensive and has a higher radiation dose than DXA. It also requires specialized scanning of regions (such as the lumbar spine) and manual marking, which is a more complicated procedure.

[0005] Therefore, it is necessary to propose an image processing method, device, equipment, storage medium and program product suitable for bone density prediction. Summary of the invention

[0006] This specification provides an image processing method, device, equipment, storage medium and program product, which processes the original medical image to accurately segment the target vertebra, target muscle and target fat, and predicts the bone density of the target vertebra and the adjacent muscle and fat, thereby effectively improving the efficiency and accuracy of the prediction while reducing external interference.

[0007] An image processing method provided in this application adopts the following technical solution, including:

[0008] Acquire original medical images of the subject;

[0009] Performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result;

[0010] Performing classification processing on the first target object in the original medical image to obtain a classification result;

[0011] Based on the classification result, performing a second segmentation process on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result;

[0012] Based on the first labeling result and the second labeling result, the original medical image of the subject is predicted to obtain a prediction result.

[0013] Optionally, the first target object includes a plurality of target vertebrae;

[0014] Optionally, performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result includes:

[0015] In the original medical image, searching for the image region where the target vertebra is located as a core segmentation region;

[0016] Determining the area to be measured of each target vertebra according to the center point position of the core segmentation area of ​​the target vertebra;

[0017] Perform instance segmentation on the target vertebral region to generate the first labeling result.

[0018] Optionally, the classifying the first target object in the original medical image to obtain a classification result includes:

[0019] Based on the first labeling result, extracting an image block of the target vertebra from the original medical image;

[0020] determining whether the target vertebra includes an implant according to the CT values ​​of the pixels of the image block of the target vertebra;

[0021] According to the texture information of the target vertebra in the original medical image, it is determined whether the target vertebra has a severe compression fracture.

[0022] Optionally, the second target object includes: target muscle and target fat;

[0023] Optionally, based on the classification result, performing a second segmentation process on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result includes:

[0024] When the target vertebra is in the first lesion state, the original medical image is segmented and marked by the muscle-fat segmentation model; wherein the target muscle is marked to obtain a muscle marking result; and the target fat is marked to obtain a fat marking result.

[0025] Optionally, predicting the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result includes:

[0026] In combination with the first labeling result, calculating the CT value of each of the target vertebrae to generate a vertebral CT value;

[0027] In combination with the muscle marking result, the CT value of the target muscle is calculated as the muscle CT value;

[0028] Calculate the CT value of the target fat in combination with the fat marking result as the fat CT value;

[0029] A regression calculation is performed on the vertebral CT value, the muscle CT value, and the fat CT value to obtain a bone density prediction value as the prediction result.

[0030] Optionally, combining the first labeling result to calculate the CT value of each target vertebra to generate a vertebral CT value includes:

[0031] performing direction correction on each of the target vertebrae;

[0032] Based on the correction result, the middle layer of each target vertebra is selected to calculate the CT value, and the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra are obtained respectively.

[0033] The present application provides an image processing device that adopts the following technical solution, including:

[0034] An acquisition module, used for acquiring an original medical image of the subject;

[0035] A first segmentation module, configured to perform a first segmentation process on the original medical image to obtain a first target object, and mark the first target object to obtain a first marking result;

[0036] A classification module, used for classifying the first target object in the original medical image to obtain a classification result;

[0037] A second segmentation module, configured to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and mark the second target object to obtain a second marking result;

[0038] A prediction module is used to predict the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result.

[0039] Optionally, the first target object includes a plurality of target vertebrae;

[0040] Optionally, the first segmentation module includes:

[0041] A core segmentation submodule, used for searching the image region where the target vertebra is located in the original medical image as a core segmentation region;

[0042] A submodule for determining a region to be measured, used for determining a region to be measured of each target vertebra according to the center point position of the core segmentation region of the target vertebra;

[0043] The first labeling submodule is used to perform instance segmentation on the target vertebral body's test area to generate the first labeling result.

[0044] Optionally, the classification module includes:

[0045] An image block extraction submodule, configured to extract an image block of the target vertebra from the original medical image based on the first labeling result;

[0046] An implant determination submodule, used to determine whether the target vertebra includes an implant according to the CT value of the pixel points of the image block of the target vertebra;

[0047] The fracture judgment submodule is used to judge whether the target vertebra has a severe compression fracture according to the texture information of the target vertebra in the original medical image.

[0048] Optionally, the second target object includes: target muscle and target fat;

[0049] Optionally, the second segmentation module includes:

[0050] When the target vertebra is in the first lesion state, the original medical image is segmented and marked by the muscle-fat segmentation model; wherein the target muscle is marked to obtain a muscle marking result; and the target fat is marked to obtain a fat marking result.

[0051] Optionally, the prediction module includes:

[0052] A first calculation submodule, configured to calculate the CT value of each of the target vertebrae in combination with the first marking result, and generate a vertebral CT value;

[0053] A second calculation submodule is used to calculate the CT value of the target muscle in combination with the muscle marking result as the muscle CT value;

[0054] A third calculation submodule is used to calculate the CT value of the target fat as the fat CT value in combination with the fat marking result;

[0055] The prediction submodule is used to perform regression calculation on the vertebral CT value, the muscle CT value and the fat CT value to obtain a bone density prediction value as the prediction result.

[0056] Optionally, the first computing submodule includes:

[0057] A direction correction unit, used for performing direction correction on each of the target vertebrae;

[0058] The vertebral CT value calculation unit is used to select the middle layer of each target vertebra to calculate the CT value based on the correction result, and obtain the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra respectively.

[0059] This specification also provides a computer device, wherein the computer device includes:

[0060] processor; and,

[0061] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.

[0062] This specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs / instructions, and when the one or more programs / instructions are executed by a processor, any of the above methods is implemented.

[0063] This specification also provides a computer program product, wherein the computer program product includes: a computer program / instructions, and when the computer program / instructions are executed by a processor, any of the above methods is implemented.

[0064] The present invention obtains an original medical image of an object to be examined; performs a first segmentation process on the original medical image to obtain a first target object, and marks the first target object to obtain a first marking result; performs a classification process on the first target object in the original medical image to obtain a classification result; based on the classification result, performs a second segmentation process on the original medical image to obtain a second target object, and marks the second target object to obtain a second marking result; based on the first marking result and the second marking result, the original medical image of the object to be examined is predicted to obtain a prediction result, and the prediction is performed in combination with multiple marking results, thereby improving the prediction accuracy and reducing the labor cost through automated processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of the principle of an image processing method provided in an embodiment of this specification;

[0066] Figure 2 A schematic diagram of the structure of an image processing method provided in an embodiment of this specification;

[0067] Figure 3A schematic diagram of the vertebral segmentation result of S2 of an image processing method provided in an embodiment of this specification;

[0068] Figure 4 A schematic diagram of a network structure of muscle fat segmentation in S4 of an image processing method provided in an embodiment of this specification;

[0069] Figure 5 A schematic diagram of muscle fat segmentation results in S4 of an image processing method provided in an embodiment of this specification;

[0070] Figure 6a A segmentation mask image of the sagittal plane before vertebral correction in an image processing method provided in an embodiment of this specification;

[0071] Figure 6b A segmentation mask image of the sagittal plane after vertebral correction in an image processing method provided in an embodiment of this specification;

[0072] Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of this specification;

[0073] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0074] Fig. 9 A schematic diagram of a computer-readable storage medium provided in accordance with an embodiment of the present specification. DETAILED DESCRIPTION

[0075] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0076] The exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a particular embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0077] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0078] Figure 1 A schematic diagram of the principle of an image processing method provided in an embodiment of this specification includes:

[0079] S1 obtains an original medical image of the subject;

[0080] S2: performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result;

[0081] S3 classifies the first target object in the original medical image to obtain a classification result;

[0082] S4: Based on the classification result, performing a second segmentation process on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result;

[0083] S5 predicts the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result.

[0084] Bone mineral density (BMD) is an important indicator for measuring bone quality. It is crucial in risk assessment of spinal fusion surgery, auxiliary diagnosis of osteoporosis and other scenarios.

[0085] The commonly used bone density measurement method in imaging is dual-energy X-ray absorptiometry (DXA), but this method may have measurement errors when the patient has aortic calcification, severe bone spurs, sclerosis, and obesity.

[0086] Quantitative computed tomography (QCT) directly assesses the quality of trabecular bone without involving cortical bone and other tissue structures, so it is more sensitive and accurate in measuring BMD and is less susceptible to interference from other factors. However, QCT is more expensive than DXA, has a higher radiation dose, and requires a dedicated scan of the lumbar spine area.

[0087] In existing technologies, compared with ordinary CT, QCT adds correction phantoms and analysis software. However, in actual operation, users still need to manually process data, such as:

[0088] In the absence of a traditional external phantom, when bone density values ​​are measured by QCT using internal tissue structures (muscle and subcutaneous fat) as a reference, although experimental data show that its reproduction coefficient of variation is only 1.9% and the correlation with QCT using a phantom is very high (r=0.96); this method requires manual selection of cross-sectional layers parallel to the endplate to calculate the CT value of the target area.

[0089] In Philips' phantom-free QCT BMD (PLBMD) system, although experimental data show that its reproduction coefficient of variation is only 4%, the system also requires manual marking of target vertebrae, muscles and fat areas, and has high requirements on the professionalism of users.

[0090] Using air and aortic blood as internal calibration reference materials, the reproducible coefficient of variation of PLBMD was only 0.5%, and there was no significant difference with the bone density value corrected by the phantom (R2≥0.95); however, air has been used to correct CT values ​​during CT imaging, and aortic calcification, a common cardiovascular disease, can easily lead to a decrease in the stability of aortic blood, thereby affecting the calibration results of bone density values; and the segmentation of the target area requires manual selection of some seed points, and is not fully automatic.

[0091] Therefore, in order to further improve the efficiency and accuracy of bone density detection, reduce human involvement, and save labor costs, the present invention provides an image processing method suitable for bone density detection, such as Figure 2 As shown, it specifically includes:

[0092] S1 obtains an original medical image of the subject;

[0093] S11 reads the original medical image of the subject;

[0094] The original medical image is a three-dimensional CT image containing a spine; the three-dimensional CT image includes a plurality of slice images in the cross-sectional direction of the vertebral body, wherein the types of the vertebral body include lumbar vertebrae and thoracic vertebrae.

[0095] In one embodiment of the present specification, the original hospital image can be obtained based on the user's upload, or the original medical image can be directly acquired after the image acquisition device (such as a CT machine) scans the body of the subject. The specific reading method is not limited here.

[0096] The user referred to in the present invention refers to the user of the device of the present invention.

[0097] S12 performs a quality inspection on the original medical image according to a preset quality inspection condition; and determines whether the quality inspection of the original medical image passes;

[0098] In order to improve the accuracy and reliability of image data, quality inspection of original medical images is required.

[0099] The preset quality inspection conditions include: a number of review sub-conditions. When the input image does not meet at least one of the review sub-conditions, it is determined that the original medical image has not passed the quality inspection; when the input image meets all the review sub-conditions, it is determined that the input image has passed the quality inspection.

[0100] Preferably, the preset quality inspection conditions include: modal review sub-conditions and size review sub-conditions.

[0101] Among them, the modality review sub-conditions include: the image modality is CT.

[0102] The size review sub-condition includes: the dimensions of each side of the image (length, width, height) are ≥ 20mm. That is, the length of the original medical image is ≥ 20mm, the width of the original medical image is ≥ 20mm, and the height of the original medical image is ≥ 20mm. Based on the size review sub-condition, the image has sufficient resolution and coverage.

[0103] When the original medical image does not meet at least one of the modality review sub-condition and the size review sub-condition, it is determined that the original medical image has not passed the quality inspection, the subsequent analysis is terminated, and a reminder is issued to inform the user. When the original medical image meets both the modality review sub-condition and the size review sub-condition, it is determined that the original medical image has passed the quality inspection and the subsequent analysis and processing can continue.

[0104] Each review sub-condition in the preset quality inspection condition can be adjusted and expanded according to actual needs to adapt to different medical image analysis needs. The review order of the review sub-conditions can be processed sequentially or in parallel according to actual needs, and no specific restrictions are made here.

[0105] In one embodiment of the present specification, the quality check of the input image is performed by SimpleITK module. As an open source medical image processing library, SimpleITK supports reading, processing and visualization of multiple medical image formats.

[0106] S13 performs a preprocessing operation on the original medical image that has passed the quality inspection to generate an original medical image.

[0107] The preprocessing operations include: one or more of redirection, resampling, pixel value normalization, and data enhancement.

[0108] After performing any of the above preprocessing operations, the original medical image may be adjusted and updated to obtain the latest original medical image; the next preprocessing operation is to process the latest original medical image obtained by the previous preprocessing operation.

[0109] Specifically, the redirection includes: redirecting the latest original medical image so that the direction of the latest original medical image is a unit matrix, and adjusting the direction of the image to a preset standard direction for convenience of subsequent processing and analysis.

[0110] Resampling includes: adjusting the resolution of the latest original medical image to a preset resolution of 3mm×3mm×3mm. Resampling is used to ensure that the medical image has the same spatial resolution when performing bone density testing.

[0111] Pixel value normalization includes: normalizing the matrix pixel values ​​of the latest input image; selecting a window width and window position of [800,50], and then normalizing the pixel values ​​to between [-1,1]. By normalizing pixel values, the data range is unified and the accuracy of subsequent processing is improved.

[0112] Data augmentation includes randomly flipping the coronal plane left and right, resizing, rotating, cropping, and adding Gaussian noise to the latest original medical images.

[0113] The order of execution of the above-mentioned specific preprocessing operations can be flexibly adjusted according to specific needs or experimental designs. Preferably, the preprocessing operations are performed in sequence according to the following logical order: redirection and resampling are performed first to adjust the image direction and resolution, and then pixel value normalization and data enhancement are performed to improve image quality and diversity.

[0114] After performing the above-mentioned specific preprocessing operations, the latest original medical image is used as the original medical image for subsequent bone density detection.

[0115] Through the preprocessing operation of this step, the image quality can be optimized and the accuracy and reliability of the detection can be improved. At the same time, the order and parameters of the preprocessing operation can be flexibly adjusted according to specific needs to facilitate the subsequent efficient and accurate bone density detection.

[0116] S2: performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result;

[0117] In one embodiment of the present specification, the first segmentation process is vertebral body segmentation.

[0118] The first target object includes a plurality of target vertebrae. The original medical image is subjected to a first segmentation process to search and segment the first target object and determine whether all the target vertebrae are included.

[0119] Preferably, the first target object includes: a first target vertebra, a second target vertebra and a third target vertebra, wherein the first target vertebra is thoracic vertebra 12 (T12), the second target vertebra is lumbar vertebra 1 (L1), and the third target vertebra is lumbar vertebra 2 (L2).

[0120] S21, in the original medical image, searching for the image region where the target vertebra is located as a core segmentation region;

[0121] In one embodiment of the present specification, a first segmentation process is performed on the original medical image based on a core segmentation network to determine a core segmentation region of each target vertebra in the original medical image;

[0122] S211, for each slice image in the original medical image, combining the slice image and its adjacent slice images to construct a first slice group;

[0123] Each slice image in the original medical image is cut into preset pixels. In one embodiment of the present specification, each slice image in the original medical image is cut into 192*192 images, and the pixel values ​​at positions less than 192*192 are filled with 0.

[0124] For any slice image, each of the N slice images (ie, 2N slice images) that are adjacent to the slice image above and below the slice image is used as the adjacent slice images of the slice image.

[0125] Slice images and adjacent slice images of the slice images are aggregated to construct a first slice group; wherein the first slice group includes 2N+1 slice images.

[0126] S212 processes the first slice group corresponding to the target vertebral body to obtain slice segmentation regions of a plurality of slice images.

[0127] From the plurality of first slice groups, slice images of the target vertebra are searched and extracted, and are used as the target slice group.

[0128] The target slice group is processed by the core segmentation network to generate segmentation regions of multiple slice images. In one embodiment of the present specification, 2N+1 slice images of the target slice group are input into the core segmentation network for processing to obtain core segmentation regions of multiple slice images.

[0129] In one embodiment of the present specification, the core segmentation network may be a convolutional neural network or other network structures, and the network structure is not limited herein.

[0130] S213 determines the core segmentation region of the target vertebral body according to the segmentation regions of the plurality of slice images.

[0131] According to the segmentation regions of the multiple target slice images, a plurality of 3D core segmentation regions are determined respectively;

[0132] The multiple 3D core segmentation regions are optimized to obtain the core segmentation region of the target vertebra.

[0133] In one embodiment of the present specification, the planar core segmentation regions of multiple slice images of the target vertebra are superimposed, and the connected domains in the superimposed core segmentation regions are found, each connected domain corresponds to a three-dimensional vertebral core, thereby obtaining multiple 3D core segmentation regions. The multiple 3D core segmentation regions are optimized, and the impurity regions whose volumes of the connected domains are less than or equal to the preset volume threshold are removed, thereby obtaining the core segmentation regions of the target vertebra. The specific value of the preset volume threshold is not limited here.

[0134] S22, determining the area to be measured of each target vertebra according to the center point position of the core segmentation area of ​​the target vertebra;

[0135] The cross section where two center points adjacent to the center point of the core segmentation area of ​​the target vertebra are located is used as a boundary to define the boundary box of the target vertebra and construct the target vertebra to be measured area.

[0136] S23 performs instance segmentation on the target vertebral body's test area to generate the first labeling result.

[0137] S231 pre-sets an instance segmentation network;

[0138] In one embodiment of the present specification, the instance segmentation network may be a convolutional neural network, and preferably, a VNet-based 3D segmentation network model may be used.

[0139] During training, a medical image dataset containing target vertebral annotations is used for network training. Appropriate training parameters are set, such as loss function (such as Dice coefficient loss), optimizer (such as Adam), learning rate, etc. The specific training process is not limited here.

[0140] S222 constructs a second slice group in combination with the target vertebral body's test area;

[0141] In one embodiment of the present specification, for any slice image in the to-be-tested region of the target vertebral body, each of the N slice images (ie, 2N+1 slice images) adjacent to the slice image above and below is taken as an adjacent region.

[0142] The slice image and the adjacent area of ​​the slice image are taken to form a second slice group.

[0143] S223: Input all slice images in the second slice group into the instance segmentation network for processing, perform instance segmentation processing on the to-be-tested area of ​​each target vertebra, and obtain a first labeling result of the target vertebra.

[0144] Specifically, multiple slice images in each of the regions to be tested are processed respectively to obtain an instance segmentation region of each slice image in each of the regions to be tested.

[0145] The planar instance segmentation regions of multiple slice images are superimposed to form a 3D instance segmentation region.

[0146] Find the connected domains in the superimposed 3D instance segmentation regions, each connected domain corresponds to a 3D instance segmentation region; remove the impurity regions whose volume of the connected domain is less than or equal to the preset volume threshold to obtain the optimized instance segmentation result; use one or more instance segmentation regions as the segmentation results, mark the target vertebra to obtain the vertebral marking result of each target vertebra, and summarize the vertebral marking results of all target vertebrae as the first marking result, as shown in the following example: Figure 3 In one embodiment of the present specification, the first labeling result includes: a vertebral region mask of T12-L2.

[0147] The present invention can perform vertebral segmentation based on the original medical image without the need for special scanning and manual marking of the lumbar region, thereby improving the convenience of bone density prediction.

[0148] S24 performs an integrity check on the labeling result in the original medical image; and determines whether the labeling result includes all the target vertebrae.

[0149] In one embodiment of the present specification, the marking results are visually checked to ensure that each target vertebra is correctly segmented. If the marking results are incomplete or contain errors, the subsequent analysis is terminated and a reminder is issued to inform the user in time. If the marking results pass the integrity check, the bone density prediction is continued.

[0150] S3 classifies the first target object in the original medical image to obtain a classification result;

[0151] In one embodiment of the present specification, the classification result is a lesion status classification, and the classification result includes: a first lesion status and a second lesion status.

[0152] Specifically, when there are no implants in the target vertebrae and no severe compression fractures occur in the target vertebrae, the classification result of the target vertebrae is determined to be the first lesion state.

[0153] When an implant exists in the target vertebra and / or a severe compression fracture occurs in the target vertebra, the classification result of the target vertebra is determined to be the second lesion state.

[0154] The present invention does not impose any specific restrictions on the order of implant determination and severe compression fracture determination; the user can adjust it according to actual needs. In order to improve the accuracy of severe compression fracture determination, preferably, the target vertebra is firstly determined for implants, and when the target vertebra does not include an implant, it is then determined whether the target vertebra has a severe compression fracture.

[0155] S31 extracting the image block of the target vertebra from the original medical image based on the first labeling result;

[0156] Each target vertebra corresponds to an image block; in one image block, only one complete target vertebra is displayed.

[0157] In this step, since the image block containing the target vertebra is relatively small, it effectively saves running time and video memory usage, while reducing the interference of other parts connected to the target vertebra on subsequent lesion status judgment.

[0158] S32, determining whether the target vertebral body includes an implant according to the CT values ​​of the pixel points of the image block of the target vertebral body;

[0159] In the surgical treatment of diseases such as lumbar fractures, lumbar spondylolisthesis, and lumbar disc herniation, lumbar implants such as metal screws and steel plates may be used. These implants are usually made of metal and have different X-ray attenuation characteristics from the surrounding tissues. Therefore, in three-dimensional CT images, based on the physical properties of the implants, the implants and their surrounding tissues may show abnormal CT values. For example, metal implants usually appear as bright areas in CT images, and their CT values ​​are much higher than those of the surrounding tissues. This bright area may interfere with the accurate assessment of the surrounding tissues. Therefore, when predicting bone density, it is necessary to exclude the presence of implants.

[0160] Specifically, the number of pixel points whose CT value of the image block of the target vertebra is greater than 2000 is counted;

[0161] It is determined whether the number of pixels of the image block of the target vertebra with a CT value greater than 2000 is greater than a preset threshold, and whether the target vertebra includes an implant is determined based on the determination result of the number of pixels.

[0162] When the number of pixel points whose CT value of the image block of the target vertebra is greater than 2000 is greater than a preset threshold, it is determined that the target vertebra includes an implant.

[0163] When the number of pixel points whose CT values ​​of the target vertebra in the original medical image are greater than 2000 is less than or equal to a preset threshold, it is determined that the target vertebra in the original medical image does not include an implant.

[0164] S33 determines whether the target vertebra has a severe compression fracture according to the texture information of the target vertebra in the original medical image.

[0165] When a lumbar vertebral compression fracture occurs, the entire spine may have kyphosis. In the compressed area, the bone density may be significantly higher than other normal bones. Therefore, if the subject has a severe compression fracture, the overall bone density test may have a significant error. Therefore, severe compression fractures need to be excluded.

[0166] In one embodiment of the present specification, S331 constructs a fracture classification training set;

[0167] Specifically, searching and acquiring sample medical images of multiple patients including target vertebral parts;

[0168] The sample medical image is annotated with real bone status information of the target vertebra, and the real bone status information is used to indicate whether the target vertebra has a preset bone status;

[0169] Classification labels are set for each patient according to the fracture type to construct a fracture classification training set.

[0170] Among them, the classification labels include but are not limited to: healthy vertebrae, severe compression fractures, etc. Of course, considering that it is difficult to forcibly classify different bone states with large intra-class differences into one category for learning, and the prediction error rate is high, therefore, each classification label can be learned during training, and different bone states with large intra-class differences can be classified into different categories for learning, which reduces the difficulty of training and makes the prediction of the trained image processing model more accurate.

[0171] S332 builds and trains a fracture classification model;

[0172] The fracture classification model includes several classifiers, each classifier corresponds to a classification label, and is used to determine the prediction probability belonging to each classification label. The network parameters include the classification parameters of each classifier.

[0173] The fracture classification training set is input into the fracture classification model for training, and the sample medical images are tested to obtain the predicted probability that the bone parts belong to each classification label. Based on the predicted probability that the bone parts belong to each classification label, the sample detection results of the sample medical images are obtained.

[0174] Based on the difference between the real bone state information and the sample detection results, and in the adjustment direction that increases the difference between the classification parameters of each classifier, the network parameters of the fracture classification model are adjusted.

[0175] This step clarifies the scope of application of the present method, that is, the present invention does not perform bone density prediction on subjects with implants and / or severe compression fractures.

[0176] For subjects with known diseases such as implants and / or severe compression fractures, more professional and accurate methods (DXA, QCT, etc.) can be used for diagnosis to effectively avoid misdiagnosis. Through this limitation, the data modeling scope is concentrated on the early screening population for osteoporosis, which is conducive to improving the accuracy of the CT value-BMD value prediction equation.

[0177] S4: Based on the classification result, performing a second segmentation process on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result;

[0178] The second target objects include: target muscle and target fat.

[0179] Among them, the target muscles include: iliopsoas muscle.

[0180] Target fat includes: subcutaneous fat in the target area. Target area includes: L1-L5 vertebrae.

[0181] The second marking result includes: a muscle marking result and a fat marking result, wherein the muscle marking result corresponds to the target muscle, and the fat marking result corresponds to the target fat.

[0182] S41 builds and trains a muscle-fat segmentation model.

[0183] In one embodiment of the present specification, CT images containing L1, L2, L3, L4, and L5 regions are collected from a medical image database or clinical data; relevant experts manually annotate the iliopsoas muscle and the subcutaneous fat region of the target area to ensure the accuracy and consistency of the annotations. The images are preprocessed by normalization, denoising, and other preprocessing operations to generate a training data set to improve the stability and efficiency of model training.

[0184] The muscle fat segmentation model can be a 3D U-Net, V-Net, etc. in deep learning. In one embodiment of this specification, Figure 4 As shown in the figure, the muscle fat segmentation model is a 3D U-shaped encoder-decoder network, which includes an encoder, a decoder, and several convolutional layers for extracting semantic features. A non-local module is embedded between the encoder and the decoder, so that there is a jump connection in the corresponding stage to enhance the global feature capture capability. Among them, the encoder downsamples the image, and the decoder upsamples the image.

[0185] The model is trained using the training data set to learn the characteristics and distribution patterns of subcutaneous fat in the iliopsoas muscle and target vertebra. During the training process, the model parameters and learning rate are continuously adjusted to optimize its performance.

[0186] The segmentation results are verified using the validation data set to evaluate their accuracy and reliability. The model is further adjusted and optimized based on the validation results.

[0187] S42: When the target vertebra is in the first lesion state, the original medical image is segmented and marked by the muscle-fat segmentation model; wherein the target muscle is marked to obtain a muscle marking result; and the target fat is marked to obtain a fat marking result.

[0188] When the target vertebra is in the first lesion state, the original medical image is input into the trained muscle-fat segmentation model. The muscle-fat segmentation model extracts features from the original medical image and uses the encoder and convolutional layer to extract semantic features from the original medical image. The non-local module captures global features between the encoder and the decoder. The decoder upsamples the extracted features, gradually restores the image resolution, and finally outputs the labeling results of the iliopsoas muscle and the labeling results of the subcutaneous fat in the target area. Figure 5 As shown, the red area is the marking result of the iliopsoas muscle, and the green area is the marking result of the subcutaneous fat in the target area. The marking result of the iliopsoas muscle is taken as the muscle marking result, and the marking result of the subcutaneous fat in the target area is taken as the fat marking result.

[0189] S5 predicts the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result.

[0190] After performing the above operations, each target vertebra (i.e., T12-L2 vertebra), as well as the subcutaneous fat and iliopsoas muscle in the target area, have been automatically and accurately segmented. The marking and segmentation of the area do not require manual intervention, which improves the prediction efficiency and saves labor costs.

[0191] S51, combining the first labeling result, calculating the CT value of each target vertebra to generate a vertebral CT value;

[0192] S511 performs direction correction on each of the target vertebrae;

[0193] It is determined whether the sagittal plane of each target vertebra is in the standard position, and the direction of each target vertebra is corrected.

[0194] In one embodiment of the present specification, the actual direction of the vertebral body is determined according to the boundary and shape information of the vertebral body, and a segmentation mask image of the sagittal plane of the vertebral body before correction is obtained, such as Figure 6a As shown. Using image processing techniques such as image rotation and affine transformation, the vertebral direction is corrected to a standard position perpendicular to the image plane, and the segmentation mask image of the sagittal plane after vertebral correction is obtained as the correction result, as shown in Figure 6bDuring the correction process, it is necessary to ensure that the overall shape and internal structure of the vertebral body are not distorted or deformed.

[0195] S512, based on the correction result, selecting the middle layer of each target vertebra to calculate the CT value, and respectively obtaining the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra;

[0196] Specifically, the middle layer of the T12 vertebra is selected to calculate the CT value to obtain the CT value of the T12 vertebra; the middle layer of the L1 vertebra is selected to calculate the CT value to obtain the CT value of the L1 vertebra; the middle layer of the L2 vertebra is selected to calculate the CT value to obtain the CT value of the L2 vertebra.

[0197] In the above steps, the vertebral direction is corrected and the middle cross section of each target vertebra is taken to calculate the CT value respectively, thereby enhancing the consistency of the values ​​of different target vertebrae.

[0198] S513 obtains the vertebral CT value according to the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra in accordance with a preset value acquisition strategy.

[0199] S52, combining the muscle marking result, calculating the CT value of the target muscle as the muscle CT value;

[0200] If the target muscle (i.e., iliopsoas muscle) is missing, the first preset CT value is used as the muscle CT value. Preferably, the first preset CT value is 50HU.

[0201] S53, combining the fat marking result, calculating the CT value of the target fat as the fat CT value;

[0202] If the target fat (ie, the subcutaneous fat in the target area) is missing, the second preset CT value is used as the fat CT value, wherein the second preset CT value is -105HU.

[0203] S54 performs regression calculation on the vertebral CT value, the muscle CT value and the fat CT value to obtain a bone density prediction value as the prediction result.

[0204] The vertebral CT value, muscle CT value and fat CT value were used as independent variables; the three-variable linear equation regression method was used to obtain the predicted value of bone density based on these three independent variables.

[0205] In one embodiment of the present specification, several historical samples are obtained, the historical samples include historical examination data of several subjects, and the historical examination data include: historical vertebral CT values, historical muscle CT values, historical fat CT values, and historical bone density values.

[0206] Establish a regression equation for bone density prediction, Y = a1X1 + a2X2 + a3X3 + b; where a1, a2, and a3 are regression coefficients, and b is the intercept. Substitute historical samples into the regression equation for bone density prediction, specifically, use the historical vertebral CT value as X1, the historical muscle CT value as X2, and the historical fat CT value as X3, and use the least squares method or other regression methods to fit and obtain the best a1, a2, a3, and b.

[0207] After determining the regression coefficients and intercepts, the vertebral CT values, muscle CT values, and fat CT values ​​can be substituted into the regression equation for bone density prediction to obtain the predicted value of bone density.

[0208] The present invention combines the target vertebral body and the adjacent tissue area (iliopsoas muscle and subcutaneous fat in the target area) as a reference to predict bone density, which effectively overcomes interference factors such as the beam hardening effect during image scanning, and avoids interference from the air area between the human body and the external phantom, thereby improving the accuracy of bone density prediction. In addition, the method of the present invention is automated, and for the same original medical image, the steps of bone density prediction remain consistent, and the prediction results are reproducible and robust.

[0209] In daily CT examinations, the technical solution of the present invention can also be used to predict bone density, so as to assist users in timely discovering the possibility of potential osteoporosis in the subject based on the prediction results, and remind the subject to undergo detailed examinations in time to reduce the occurrence of fractures. In addition, the method of the present invention saves the time and cost of additional QCT examinations, etc., and reduces the risk of radiation exposure.

[0210] Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of this specification, the device specifically includes:

[0211] An acquisition module 710 is used to acquire an original medical image of the subject;

[0212] A first segmentation module 720, configured to perform a first segmentation process on the original medical image to obtain a first target object, and mark the first target object to obtain a first marking result;

[0213] A classification module 730 is used to classify the first target object in the original medical image to obtain a classification result;

[0214] A second segmentation module 740 is used to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and mark the second target object to obtain a second marking result;

[0215] The prediction module 750 is used to predict the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result.

[0216] Optionally, the first target object includes a plurality of target vertebrae;

[0217] Optionally, the first segmentation module 720 includes:

[0218] A core segmentation submodule, used for searching the image region where the target vertebra is located in the original medical image as a core segmentation region;

[0219] A submodule for determining a region to be measured, used for determining a region to be measured of each target vertebra according to the center point position of the core segmentation region of the target vertebra;

[0220] The first labeling submodule is used to perform instance segmentation on the target vertebral body's test area to generate the first labeling result.

[0221] Optionally, the classification module 730 includes:

[0222] An image block extraction submodule, configured to extract an image block of the target vertebra from the original medical image based on the first labeling result;

[0223] An implant determination submodule, used to determine whether the target vertebra includes an implant according to the CT value of the pixel points of the image block of the target vertebra;

[0224] The fracture judgment submodule is used to judge whether the target vertebra has a severe compression fracture according to the texture information of the target vertebra in the original medical image.

[0225] Optionally, the second target object includes: target muscle and target fat;

[0226] Optionally, the second segmentation module 740 includes:

[0227] When the target vertebra is in the first lesion state, the original medical image is segmented and marked by the muscle-fat segmentation model; wherein the target muscle is marked to obtain a muscle marking result; and the target fat is marked to obtain a fat marking result.

[0228] Optionally, the prediction module 750 includes:

[0229] A first calculation submodule, configured to calculate the CT value of each of the target vertebrae in combination with the first marking result, and generate a vertebral CT value;

[0230] A second calculation submodule is used to calculate the CT value of the target muscle in combination with the muscle marking result as the muscle CT value;

[0231] A third calculation submodule is used to calculate the CT value of the target fat as the fat CT value in combination with the fat marking result;

[0232] The prediction submodule is used to perform regression calculation on the vertebral CT value, the muscle CT value and the fat CT value to obtain a bone density prediction value as the prediction result.

[0233] Optionally, the first computing submodule includes:

[0234] A direction correction unit, used for performing direction correction on each of the target vertebrae;

[0235] The vertebral CT value calculation unit is used to select the middle layer of each target vertebra to calculate the CT value based on the correction result, and obtain the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra respectively.

[0236] The functions of the device in the embodiment of the present invention have been described in the above method embodiment, so for details not provided in the description of this embodiment, please refer to the relevant description in the above embodiment, and no further description will be given here.

[0237] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer programs / instructions. These computer programs / instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0239] These computer programs / instructions may also be stored in a readable memory of a computer device capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the readable memory of the computer device produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0240] These computer programs / instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer device or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer device or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0241] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0242] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: include: Acquire original medical images of the subject; Performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result; Performing classification processing on the first target object in the original medical image to obtain a classification result; Based on the classification result, performing a second segmentation process on the original medical image to obtain a second target object, and marking the second target object to obtain a second marking result; Based on the first labeling result and the second labeling result, the original medical image of the subject is predicted to obtain a prediction result.

2. An image processing method as claimed in claim 1, characterized in that: The first target object includes a plurality of target vertebrae; The performing a first segmentation process on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result, includes: In the original medical image, searching for the image region where the target vertebra is located as a core segmentation region; Determining the area to be measured of each target vertebra according to the center point position of the core segmentation area of ​​the target vertebra; Perform instance segmentation on the target vertebral area to generate the first labeling result.

3. An image processing method as claimed in claim 2, characterized in that: The classifying the first target object in the original medical image to obtain a classification result includes: Based on the first labeling result, extracting an image block of the target vertebra from the original medical image; determining whether the target vertebra includes an implant according to the CT values ​​of the pixels of the image block of the target vertebra; According to the texture information of the target vertebra in the original medical image, it is determined whether the target vertebra has a severe compression fracture.

4. An image processing method as claimed in claim 2, characterized in that: The second target object includes: target muscle and target fat.

5. An image processing method as claimed in claim 4, characterized in that: The step of performing a second segmentation process on the original medical image based on the classification result to obtain a second target object, and marking the second target object to obtain a second marking result includes: When the target vertebra is in the first lesion state, the original medical image is segmented and marked by the muscle-fat segmentation model; wherein the target muscle is marked to obtain a muscle marking result; and the target fat is marked to obtain a fat marking result.

6. An image processing method as claimed in claim 5, characterized in that: The step of predicting the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result includes: In combination with the first labeling result, calculating the CT value of each of the target vertebrae to generate a vertebral CT value; In combination with the muscle marking result, the CT value of the target muscle is calculated as the muscle CT value; Calculate the CT value of the target fat in combination with the fat marking result as the fat CT value; A regression calculation is performed on the vertebral CT value, the muscle CT value, and the fat CT value to obtain a bone density prediction value as the prediction result.

7. The image processing method according to claim 1, characterized in that: The step of combining the first marking result, calculating the CT value of each target vertebra, and generating a vertebral CT value includes: performing direction correction on each of the target vertebrae; Based on the correction result, the middle layer of each target vertebra is selected to calculate the CT value, and the CT value of the T12 vertebra, the CT value of the L1 vertebra, and the CT value of the L2 vertebra are obtained respectively.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring an original medical image of the subject; A first segmentation module, configured to perform a first segmentation process on the original medical image to obtain a first target object, and mark the first target object to obtain a first marking result; A classification module, used for classifying the first target object in the original medical image to obtain a classification result; A second segmentation module, configured to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and mark the second target object to obtain a second marking result; A prediction module is used to predict the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result.

9. A computer device, characterized in that: The computer equipment includes: processor; and, A memory storing computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs / instructions, and when the one or more programs / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images