An image processing method, apparatus, device, storage medium, and program product

Abstract

The application provides an image processing method, device and equipment, a storage medium and a program product, relates to the field of medical image processing, and comprises the following steps: acquiring an original medical image of a subject; performing first segmentation processing on the original medical image to obtain a first target object, and marking the first target object to obtain a first marking result; performing classification processing on the first target object in the original medical image to obtain a classification result; performing second segmentation processing on the original medical image based on the classification result to obtain a second target object, and marking the second target object to obtain a second marking result; and performing prediction on the original medical image of the subject based on the first marking result and the second marking result to obtain a prediction result.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more particularly to an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Bone mineral density (BMD) is an important indicator for measuring bone strength and the degree of osteoporosis. BMD is typically measured using methods such as dual-energy X-ray absorptiometry (DXA) and quantitative computed tomography (QCT). However, in clinical practice, the following main problems exist:

[0003] (1) When patients have aortic calcification, severe bone spurs, sclerosis, obesity and other diseases, DXA has a large error.

[0004] (2) Although BMD is less susceptible to interference from other factors when measured by QCT, QCT is more expensive and has a higher radiation dose than DXA; it also requires specialized scanning of a region (such as the lumbar region) and manual marking, making the process more complicated.

[0005] Therefore, it is necessary to propose an image processing method, device, equipment, storage medium, and program product suitable for bone density prediction. Summary of the Invention

[0006] This specification provides an image processing method, apparatus, device, storage medium, and program product that processes raw medical images to accurately segment target vertebrae, target muscles, and target fat. By using the target vertebrae and adjacent muscles and fat, bone density is predicted, effectively improving the efficiency and accuracy of prediction while reducing external interference.

[0007] The image processing method provided in this application adopts the following technical solution, including:

[0008] Acquire the original medical images of the subject;

[0009] The original medical image is subjected to a first segmentation process to obtain a first target object, and the first target object is marked to obtain a first marking result;

[0010] The first target object in the original medical image is classified to obtain a classification result;

[0011] Based on the classification results, the original medical image is subjected to a second segmentation process to obtain a second target object, and the second target object is labeled to obtain a second labeling result;

[0012] Based on the first and second labeling results, the original medical image of the examined object is predicted to obtain the prediction result.

[0013] Optionally, the first target object includes a plurality of target cones;

[0014] Optionally, the step of performing a first segmentation process on the original medical image to obtain a first target object, and labeling the first target object to obtain a first labeling result, includes:

[0015] In the original medical image, the image region where the target vertebra is located is identified as the core segmentation region;

[0016] Based on the center point location of the core segmentation region of the target vertebra, the test area of ​​each target vertebra is determined;

[0017] The test region of the target vertebra is segmented to generate the first labeling result.

[0018] Optionally, the classification process for the first target object in the original medical image to obtain a classification result includes:

[0019] Based on the first labeling result, an image patch of the target vertebra is extracted from the original medical image;

[0020] Based on the CT values ​​of the pixels in the image block of the target vertebral body, determine whether the target vertebral body includes an implant;

[0021] Based on the texture information of the target vertebra in the original medical image, it is determined whether the target vertebra has a severe compression fracture.

[0022] Optionally, the second target object includes: target muscle and target fat;

[0023] Optionally, the step of performing a second segmentation process on the original medical image based on the classification result to obtain a second target object, and labeling the second target object to obtain a second labeling result, includes:

[0024] When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using the muscle-fat segmentation model; wherein, the target muscle is labeled to obtain muscle labeling results; and the target fat is labeled to obtain fat labeling results.

[0025] Optionally, the step of predicting the original medical image of the examined object based on the first labeling result and the second labeling result to obtain the prediction result includes:

[0026] Based on the first marking result, the CT value of each target vertebra is calculated to generate the vertebral CT value;

[0027] Based on the muscle marking results, the CT value of the target muscle is calculated and used as the muscle CT value;

[0028] Based on the fat marking results, the CT value of the target fat is calculated and used as the fat CT value;

[0029] Regression calculations are performed on the vertebral body CT values, muscle CT values, and fat CT values ​​to obtain bone mineral density prediction values, which are used as the prediction results.

[0030] Optionally, the step of combining the first marking results to calculate the CT values ​​of each of the target vertebrae and generate vertebral CT values ​​includes:

[0031] Orientation correction is performed on each of the target vertebrae;

[0032] Based on the correction results, the CT value of the intermediate layer of each target vertebra is calculated to obtain the CT values ​​of the T12 vertebra, L1 vertebra, and L2 vertebra respectively.

[0033] The image processing apparatus provided in this application adopts the following technical solution, including:

[0034] The acquisition module is used to acquire the original medical images of the examined object;

[0035] The first segmentation module is used to perform a first segmentation process on the original medical image to obtain a first target object, and to mark the first target object to obtain a first marking result;

[0036] The classification module is used to classify the first target object in the original medical image to obtain the classification result;

[0037] The second segmentation module is used to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and to label the second target object to obtain a second labeling result;

[0038] The prediction module is used to predict the original medical image of the examined object based on the first labeling result and the second labeling result, and obtain the prediction result.

[0039] Optionally, the first target object includes a plurality of target cones;

[0040] Optionally, the first segmentation module includes:

[0041] The core segmentation submodule is used to locate the image region where the target vertebra is located in the original medical image, and use it as the core segmentation region;

[0042] The test area determination submodule is used to determine the test area of ​​each target vertebra based on the center point position of the core segmentation region of the target vertebra.

[0043] The first labeling submodule is used to perform instance segmentation on the test area of ​​the target vertebra and generate the first labeling result.

[0044] Optionally, the classification module includes:

[0045] An image patch extraction submodule is used to extract image patches of the target vertebra from the original medical image based on the first labeling result;

[0046] The implant determination submodule is used to determine whether the target vertebral body contains an implant based on the CT values ​​of the pixels in the image block of the target vertebral body;

[0047] The fracture determination submodule is used to determine whether the target vertebra has a severe compression fracture based on the texture information of the target vertebra in the original medical image.

[0048] Optionally, the second target object includes: target muscle and target fat;

[0049] Optionally, the second segmentation module includes:

[0050] When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using the muscle-fat segmentation model; wherein, the target muscle is labeled to obtain muscle labeling results; and the target fat is labeled to obtain fat labeling results.

[0051] Optionally, the prediction module includes:

[0052] The first calculation submodule is used to combine the first marking result to calculate the CT value of each target vertebra and generate the vertebral CT value;

[0053] The second calculation submodule is used to calculate the CT value of the target muscle by combining the muscle marking results, and use it as the muscle CT value.

[0054] The third calculation submodule is used to combine the fat marking results to calculate the CT value of the target fat, which is then used as the fat CT value.

[0055] The prediction submodule is used to perform regression calculations on the vertebral body CT value, the muscle CT value, and the fat CT value to obtain a bone mineral density prediction value, which is used as the prediction result.

[0056] Optionally, the first computing submodule includes:

[0057] Orientation correction unit, used to correct the orientation of each of the target vertebrae;

[0058] The vertebral body CT value calculation unit is used to select the intermediate layer of each target vertebral body to calculate the CT value based on the correction result, and obtain the CT value of T12 vertebral body, L1 vertebral body and L2 vertebral body respectively.

[0059] This specification also provides a computer device, wherein the computer device includes:

[0060] Processor; and,

[0061] A memory that stores computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0062] This specification also provides a computer-readable storage medium that stores one or more programs / instructions that, when executed by a processor, implement any of the methods described above.

[0063] This specification also provides a computer program product, wherein the computer program product includes: a computer program / instruction, which, when executed by a processor, implements any of the methods described above.

[0064] This invention acquires the original medical image of the subject; performs a first segmentation process on the original medical image to obtain a first target object, and labels the first target object to obtain a first labeling result; performs a classification process on the first target object in the original medical image to obtain a classification result; performs a second segmentation process on the original medical image based on the classification result to obtain a second target object, and labels the second target object to obtain a second labeling result; and predicts the original medical image of the subject based on the first labeling result and the second labeling result to obtain a prediction result. By combining multiple labeling results for prediction, the prediction accuracy is improved, and the automation process reduces labor costs. Attached Figure Description

[0065] Figure 1 A schematic diagram illustrating the principle of an image processing method provided in the embodiments of this specification;

[0066] Figure 2 This is a schematic diagram of the structure of an image processing method provided in an embodiment of this specification;

[0067] Figure 3A schematic diagram of the cone segmentation result of S2 in an image processing method provided in an embodiment of this specification;

[0068] Figure 4 A schematic diagram of the network structure for muscle and fat segmentation in S4 of an image processing method provided in an embodiment of this specification.

[0069] Figure 5 A schematic diagram of the muscle and fat segmentation result in S4 of an image processing method provided in an embodiment of this specification.

[0070] Figure 6a A segmentation mask of the sagittal plane before vertebral correction in an image processing method provided in an embodiment of this specification;

[0071] Figure 6b A segmentation mask of the sagittal plane after vertebral correction for an image processing method provided in an embodiment of this specification;

[0072] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this specification;

[0073] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0074] Figure 9 This is a schematic diagram of a computer-readable storage medium provided for an embodiment of this specification. Detailed Implementation

[0075] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0076] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. While conforming to the inventive concept, the features, structures, characteristics, or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0077] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0078] Figure 1 This is a schematic diagram illustrating the principle of an image processing method provided in an embodiment of this specification, including:

[0079] S1 acquires the raw medical images of the subject being examined;

[0080] S2 performs a first segmentation process on the original medical image to obtain a first target object, and marks the first target object to obtain a first marking result;

[0081] S3 performs classification processing on the first target object in the original medical image to obtain the classification result;

[0082] S4 performs a second segmentation process on the original medical image based on the classification result to obtain a second target object, and marks the second target object to obtain a second marking result;

[0083] S5 predicts the original medical image of the examined object based on the first labeling result and the second labeling result, and obtains the prediction result.

[0084] Bone mineral density (BMD) is an important indicator for measuring bone quality and is crucial in scenarios such as risk assessment for spinal fusion surgery and auxiliary diagnosis of osteoporosis.

[0085] The commonly used method for measuring bone mineral density in imaging is dual-energy X-ray absorptiometry (DXA). However, this method can have measurement errors when patients have aortic calcification, severe bone spurs, sclerosis, or obesity.

[0086] Quantitative computed tomography (QCT) directly assesses the quality of trabecular bone without involving cortical bone or other tissue structures, making it more sensitive and accurate in measuring bone marrow density (BMD) and less susceptible to interference from other factors. However, QCT is more expensive than DXA, involves higher radiation exposure, and requires specialized scanning of the lumbar spine region.

[0087] In existing technologies, compared to conventional CT, QCT adds a corrective phantom and analysis software. However, in practice, manual data processing is still required, for example:

[0088] In the absence of a traditional external phantom, when using internal tissue structures (muscle and subcutaneous fat) as a reference to measure bone mineral density values ​​via QCT, although experimental data show that the coefficient of variation for reproducibility is only 1.9% and the correlation with QCT using a phantom is high (r = 0.96), this method requires manually selecting a cross-sectional layer parallel to the endplate to calculate the CT value of the target area.

[0089] Although experimental data show that the coefficient of variation in the Philips phantom-free QCT BMD (PLBMD) system is only 4%, the system still requires manual marking of target vertebrae, muscles and fat areas, which places high demands on the professionalism of the users.

[0090] Using air and aortic blood as internal calibration reference materials, the coefficient of variation for PLBMD was only 0.5%, and there was no significant difference from the bone mineral density value corrected by the phantom (R2≥0.95). However, air has already been used for CT value correction during the CT imaging stage, and aortic calcification, a common cardiovascular disease, can easily lead to a decrease in the stability of aortic blood, thus affecting the calibration results of bone mineral density values. Furthermore, the segmentation of the target region requires manual selection of some seed points and is not fully automatic.

[0091] Therefore, in order to further improve the efficiency and accuracy of bone density detection, reduce human intervention, and save labor costs, this invention provides an image processing method suitable for bone density detection, such as... Figure 2 As shown, it specifically includes:

[0092] S1 acquires the raw medical images of the subject being examined;

[0093] S11 reads the original medical images of the examined object;

[0094] The original medical images are three-dimensional CT images containing the spine; the three-dimensional CT images include multiple slice images in the cross-sectional direction of the vertebral bodies. Among them, the vertebral bodies include lumbar vertebrae and thoracic vertebrae.

[0095] In one embodiment of this specification, original hospital images can be obtained based on user uploads; alternatively, original medical images can be directly acquired after the image acquisition device (e.g., a CT scanner) scans the subject's body. Specific reading methods are not limited here.

[0096] The term "user" in this invention refers to the user of the device of this invention.

[0097] S12 performs a quality check on the original medical image according to preset quality inspection conditions; determines whether the quality check of the original medical image passes.

[0098] To improve the accuracy and reliability of image data, it is necessary to perform quality checks on the original medical images.

[0099] The preset quality inspection conditions include several review sub-conditions. If the input image does not meet at least one of the review sub-conditions, the original medical image is deemed to have failed the quality inspection; if the input image meets all the review sub-conditions, the input image is deemed to have passed the quality inspection.

[0100] As a preferred option, the preset quality inspection conditions include: modal inspection sub-conditions and dimensional inspection sub-conditions.

[0101] The modality review sub-condition includes: the image modality is CT.

[0102] The size review sub-conditions include: all dimensions (length, width, and height) of the image must be ≥20mm. That is, the original medical image must be ≥20mm long, ≥20mm wide, and ≥20mm high. These size review sub-conditions ensure that the image has sufficient resolution and coverage.

[0103] If the original medical image fails to meet at least one of the modality review sub-conditions and the size review sub-conditions, the original medical image is deemed to have failed the quality check, subsequent analysis is terminated, and a notification is issued to the user. If the original medical image meets both the modality review sub-conditions and the size review sub-conditions, the original medical image is deemed to have passed the quality check and can continue with subsequent analysis and processing.

[0104] The various review sub-conditions in the preset quality inspection conditions can be adjusted and expanded according to actual needs to adapt to different medical image analysis requirements. The review order of the review sub-conditions can be processed sequentially or in parallel according to actual needs, without specific restrictions here.

[0105] In one embodiment of this specification, the input image is quality checked using the SimpleITK module. SimpleITK, an open-source medical image processing library, supports the reading, processing, and visualization of various medical image formats.

[0106] S13 performs preprocessing operations on the original medical images that have passed the quality check to generate original medical images.

[0107] Preprocessing operations include one or more of the following: retargeting, resampling, pixel value normalization, and data augmentation.

[0108] After performing any of the above preprocessing operations, the original medical image may be adjusted and updated to obtain the latest original medical image; the next preprocessing operation is to process the latest original medical image obtained from the previous preprocessing operation.

[0109] Specifically, retargeting involves retargeting the latest raw medical image so that its orientation is the identity matrix. This is done by adjusting the image's orientation to a preset standard orientation to facilitate subsequent processing and analysis.

[0110] Resampling involves adjusting the resolution of the latest original medical images to a preset resolution of 3mm × 3mm × 3mm. This resampling ensures that the medical images have the same spatial resolution when performing bone density testing.

[0111] Pixel value normalization includes: performing matrix pixel value normalization on the latest input image; selecting a window width and window level of [800, 50], and then normalizing the pixel values ​​to the range of [-1, 1]. Pixel value normalization unifies the data range and improves the accuracy of subsequent processing.

[0112] Data augmentation includes: randomly performing operations such as coronal plane flipping, scaling, rotation, cropping, and adding Gaussian noise on the latest raw medical images.

[0113] The specific order of execution of the preprocessing operations described above can be flexibly adjusted according to specific needs or experimental design. Preferably, the preprocessing operations are performed in the following logical order: first, retargeting and resampling are performed to adjust the image orientation and resolution, followed by pixel value normalization and data augmentation to improve image quality and diversity.

[0114] After performing the specific preprocessing operations described above, the latest raw medical image is used as the raw medical image for subsequent bone density detection.

[0115] This preprocessing step optimizes image quality and improves the accuracy and reliability of detection. Furthermore, the order and parameters of the preprocessing steps can be flexibly adjusted according to specific needs to facilitate efficient and accurate bone density detection in subsequent steps.

[0116] S2 performs a first segmentation process on the original medical image to obtain a first target object, and marks the first target object to obtain a first marking result;

[0117] In one embodiment of this specification, the first segmentation process is cone segmentation.

[0118] The first target object includes several target vertebrae. The original medical image undergoes a first segmentation process to locate and segment the first target object, determining whether it contains all target vertebrae.

[0119] Preferably, the first target vertebra includes a first target vertebra, a second target vertebra, and a third target vertebra. The first target vertebra is the T12 vertebra (T12); the second target vertebra is the L1 vertebra (L1); and the third target vertebra is the L2 vertebra (L2).

[0120] S21. Locate the image region where the target vertebra is located in the original medical image, and use it as the core segmentation region;

[0121] In one embodiment of this specification, the original medical image is subjected to a first segmentation process based on a core segmentation network to determine the core segmentation region of each target vertebra in the original medical image;

[0122] S211 For each slice image in the original medical image, combine the slice image and its adjacent slice images to construct a first slice group;

[0123] Each slice of the original medical image is cropped to a preset number of pixels. In one embodiment of this specification, each slice of the original medical image is cropped to a 192*192 pixel image, and pixels smaller than 192*192 are filled with 0 pixels.

[0124] For any slice image, the N slice images that are vertically adjacent to it (i.e., 2N slice images) are taken as the adjacent slice images of that slice image.

[0125] Summarize the sliced ​​images and their adjacent sliced ​​images to construct the first slice group; wherein the first slice group contains 2N+1 sliced ​​images.

[0126] S212 processes the first slice group corresponding to the target vertebra to obtain slice segmentation regions of multiple slice images.

[0127] From several first slice groups, find and extract slice images of the target vertebral body, and use them as the target slice group.

[0128] The target slice group is processed by a core segmentation network to generate segmented regions for multiple slice images. In one embodiment of this specification, 2N+1 slice images of the target slice group are input into the core segmentation network for processing to obtain core segmentation regions for multiple slice images.

[0129] In one embodiment of this specification, the core segmentation network may be a convolutional neural network or other network structures, and the network structure is not limited herein.

[0130] S213 determines the core segmentation region of the target vertebra based on the segmentation regions of the multiple slice images.

[0131] Based on the segmentation regions of multiple target slice images, multiple 3D core segmentation regions are determined respectively;

[0132] Multiple 3D core segmentation regions are optimized to obtain the core segmentation region of the target vertebra.

[0133] In one embodiment of this specification, planar core segmentation regions of multiple slice images of the target vertebra are superimposed, and connected components are found within the superimposed core segmentation regions. Each connected component corresponds to a three-dimensional vertebral core, thereby obtaining multiple 3D core segmentation regions. These multiple 3D core segmentation regions are then optimized by removing impurity regions whose connected component volumes are less than or equal to a preset volume threshold, thus obtaining the core segmentation region of the target vertebra. The specific value of the preset volume threshold is not limited here.

[0134] S22 determines the test area of ​​each target vertebra based on the center point position of the core segmentation region of the target vertebra;

[0135] The cross-sections containing the two adjacent center points above and below the center point of the core segmentation region of the target vertebra are used as boundaries to define the bounding box of the target vertebra and construct the test region of the target vertebra.

[0136] S23 performs instance segmentation on the test area of ​​the target vertebra to generate the first labeling result.

[0137] S231 pre-configures the instance segmentation network;

[0138] In one embodiment of this specification, the instance segmentation network can be a convolutional neural network, and preferably, it can be a 3D segmentation network model based on VNet, etc.

[0139] During training, a medical image dataset containing target vertebral body annotations is used to train the network. Appropriate training parameters are set, such as the loss function (e.g., Dice coefficient loss), optimizer (e.g., Adam), learning rate, etc. The specific training process is not limited here.

[0140] S222 Combines the test area of ​​the target vertebral body to construct a second slice group;

[0141] In one embodiment of this specification, for any slice image in the test area of ​​the target vertebra, the N slice images (i.e., 2N+1 slice images) that are vertically adjacent to the slice image are taken as the adjacent area.

[0142] Take the slice image and its adjacent regions to form a second slice group.

[0143] S223 inputs all slice images in the second slice group into the instance segmentation network for processing, performs instance segmentation processing on the test region of each target vertebra, and obtains the first labeling result of the target vertebra.

[0144] Specifically, multiple slice images in each test region are processed separately to obtain the instance segmentation region of each slice image in each test region.

[0145] The planar instance segmentation regions of multiple slice images are superimposed to form a 3D instance segmentation region.

[0146] Find connected components in the superimposed 3D instance segmentation regions, with each connected component corresponding to a 3D instance segmentation region; remove impurity regions whose volume is less than or equal to a preset volume threshold to obtain the optimized instance segmentation result; use one or more instance segmentation regions as the segmentation result, label the target vertebrae to obtain the vertebral labeling result for each target vertebra, and summarize the vertebral labeling results of all target vertebrae as the first labeling result, as detailed below. Figure 3 As shown. In one embodiment of this specification, the first marking result includes: a vertebral region mask for T12-L2.

[0147] This invention can segment the vertebral body based on original medical images, eliminating the need for specialized scanning and manual marking of the lumbar spine region, thus improving the convenience of bone density prediction.

[0148] S24 performs an integrity check on the marking results in the original medical image; and determines whether the marking results contain all the target vertebrae.

[0149] In one embodiment of this specification, the marking results are visually checked to ensure that each target vertebra is correctly segmented. If the marking results are incomplete or contain errors, subsequent analysis is terminated, and a notification is issued to the user. If the marking results pass the integrity check, bone mineral density prediction continues.

[0150] S3 performs classification processing on the first target object in the original medical image to obtain the classification result;

[0151] In one embodiment of this specification, the classification result is a lesion state classification, which includes: a first lesion state and a second lesion state.

[0152] Specifically, when there are no implants in the target vertebrae and no severe compression fractures have occurred in the target vertebrae, the classification result of the target vertebrae is determined to be the first lesion state.

[0153] When an implant is present in the target vertebral body, and / or when a severe compression fracture occurs in the target vertebral body, the target vertebral body is classified as a second lesion state.

[0154] This invention does not impose specific restrictions on the order of implant assessment and severe compression fracture assessment; users can adjust this order according to their actual needs. To improve the accuracy of severe compression fracture assessment, preferably, implant assessment is performed on the target vertebra first, and if the target vertebra does not contain an implant, then the presence of a severe compression fracture is assessed.

[0155] S31 Based on the first labeling result, extract the image block of the target vertebra from the original medical image;

[0156] Each target vertebra corresponds to an image block; only one complete target vertebra is displayed in an image block.

[0157] In this step, because the image block containing the target vertebra is small in size, it effectively saves running time and video memory usage, while reducing interference from other parts connected to the target vertebra on the subsequent lesion status assessment.

[0158] S32 determines whether the target vertebral body includes an implant based on the CT values ​​of the pixels in the image block of the target vertebral body;

[0159] In surgical treatment of conditions such as lumbar fractures, lumbar spondylolisthesis, and lumbar disc herniation, lumbar implants, such as metal screws and plates, may be used. These implants are usually made of metal and have different X-ray attenuation characteristics compared to surrounding tissues. Therefore, in 3D CT images, based on the physical characteristics of the implant, the implant and its surrounding tissues may show abnormal CT values. For example, metal implants typically appear as bright areas on CT images, with CT values ​​much higher than those of surrounding tissues. This bright area may interfere with the accurate assessment of surrounding tissues; therefore, the presence of implants needs to be excluded when performing bone mineral density prediction.

[0160] Specifically, count the number of pixels in the image block of the target vertebra with a CT value greater than 2000;

[0161] The system determines whether the number of pixels with a CT value greater than 2000 in the image block of the target vertebral body exceeds a preset threshold, and determines whether the target vertebral body includes an implant based on the result of the pixel count determination.

[0162] Specifically, if the number of pixels with a CT value greater than 2000 in the image block of the target vertebral body exceeds a preset threshold, the target vertebral body is considered to include an implant.

[0163] If the number of pixels with a CT value greater than 2000 in the original medical image is less than or equal to a preset threshold, then the target vertebra in the original medical image is considered not to contain the implant.

[0164] S33 determines whether the target vertebra has a severe compression fracture based on the texture information of the target vertebra in the original medical image.

[0165] Compression fractures of the lumbar vertebrae can lead to kyphosis of the entire spine. The bone density at the compression site may be significantly higher than that of normal bone. Therefore, if a subject has a severe compression fracture, the overall bone density measurement may show significant errors. Thus, it is necessary to rule out severe compression fractures.

[0166] In one embodiment of this specification, S331 constructs a fracture classification training set;

[0167] Specifically, find and acquire sample medical images of multiple patients containing the target vertebral body region;

[0168] The sample medical images are labeled with the actual bone state information of the target vertebra, which is used to indicate whether the target vertebra has the preset bone state.

[0169] Each patient was assigned a classification label based on fracture type, and a fracture classification training set was constructed.

[0170] The classification labels include, but are not limited to, healthy vertebrae and severe compression fractures. However, considering the high difficulty and error rate of forcibly grouping different bone conditions with significant intra-class differences into a single category for learning, it is advisable to learn each classification label during training. This reduces the training difficulty and makes the trained image processing model more accurate.

[0171] S332 constructs and trains a fracture classification model;

[0172] The fracture classification model includes several classifiers, each corresponding to a classification label, used to determine the predicted probability of belonging to each classification label. The network parameters include the classification parameters of each classifier.

[0173] The fracture classification training set is input into the fracture classification model for training. Sample medical images are detected to obtain the predicted probability of bone parts belonging to each classification label. Based on the predicted probability of bone parts belonging to each classification label, the sample detection results of the sample medical images are obtained.

[0174] Based on the difference between real bone condition information and sample detection results, and following the adjustment direction that increases the difference between the classification parameters of each classifier, the network parameters of the fracture classification model are adjusted.

[0175] This step clarifies the scope of application of this method, namely, that the present invention does not perform bone mineral density prediction on subjects with implants and / or those who have suffered severe compression fractures.

[0176] For subjects with established conditions such as implants and / or severe compression fractures, more specialized and precise methods (DXA, QCT, etc.) can be used for diagnosis to effectively avoid misdiagnosis. This limitation focuses the data modeling scope on the early screening population for osteoporosis, which is beneficial to improving the accuracy of the CT value-BMD value prediction equation.

[0177] S4 performs a second segmentation process on the original medical image based on the classification result to obtain a second target object, and marks the second target object to obtain a second marking result;

[0178] The second target groups include: target muscle and target fat.

[0179] The target muscles include the iliopsoas muscle.

[0180] The target fat includes subcutaneous fat in the target area. The target area includes the L1-L5 vertebrae.

[0181] The second set of labeling results includes muscle labeling results and fat labeling results. The muscle labeling results correspond to the target muscle, and the fat labeling results correspond to the target fat.

[0182] S41 builds and trains a muscle-fat segmentation model.

[0183] In one embodiment of this specification, CT images containing regions L1, L2, L3, L4, and L5 are collected from medical image databases or clinical data. The iliopsoas muscle and subcutaneous fat regions of the target area are manually annotated by relevant experts to ensure accuracy and consistency. The images undergo preprocessing operations such as normalization and denoising to generate a training dataset, thereby improving the stability and efficiency of model training.

[0184] The muscle and fat segmentation model can be a 3D U-Net, V-Net, or similar deep learning model. In one embodiment of this specification, such as... Figure 4 As shown, the muscle-fat segmentation model is a 3D U-shaped encoder-decoder network, including an encoder, a decoder, and several convolutional layers for extracting semantic features. Non-local modules are embedded between the encoder and decoder, allowing skip connections at corresponding stages to enhance global feature capture capabilities. Specifically, the encoder downsamples the image, and the decoder upsamples the image.

[0185] The model was trained using a training dataset to learn the characteristics and distribution patterns of subcutaneous fat in the iliopsoas muscle and the target vertebral body. During training, the model's parameters and learning rate were continuously adjusted to optimize its performance.

[0186] The segmentation results are validated using a validation dataset to evaluate their accuracy and reliability. Based on the validation results, the model is further adjusted and optimized.

[0187] S42 When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using the muscle-fat segmentation model; wherein, the target muscle is labeled to obtain muscle labeling results; and the target fat is labeled to obtain fat labeling results.

[0188] When the target vertebra is in the first lesion state, the original medical image is input into a pre-trained muscle-fat segmentation model. The muscle-fat segmentation model extracts features from the original medical image, using an encoder and convolutional layers to extract semantic features; a non-local module captures global features between the encoder and decoder; the decoder upsamples the extracted features, progressively restoring the image resolution, and finally outputs the labeled results of the iliopsoas muscle and the subcutaneous fat of the target region; for example... Figure 5 As shown, the red area represents the labeling results of the iliopsoas muscle; the green area represents the labeling results of the subcutaneous fat in the target area. The labeling results of the iliopsoas muscle are used as muscle labeling results, and the labeling results of the subcutaneous fat in the target area are used as fat labeling results.

[0189] S5 predicts the original medical image of the examined object based on the first labeling result and the second labeling result, and obtains the prediction result.

[0190] After the above operations are performed, each target vertebra (i.e., T12-L2 vertebrae), as well as the subcutaneous fat and iliopsoas muscle of the target area, have been automatically and accurately segmented. The marking and segmentation of the area do not require manual intervention, which improves prediction efficiency and saves labor costs.

[0191] S51 combines the first marking result to calculate the CT value of each target vertebra and generate the vertebral CT value;

[0192] S511 performs orientation correction on each of the target vertebrae;

[0193] Determine whether the sagittal plane of each target vertebra is in the standard position, and perform directional correction for each target vertebra.

[0194] In one embodiment of this specification, the actual orientation of the vertebral body is determined based on its boundary and shape information, resulting in a segmented mask image of the sagittal plane before vertebral body correction, as shown below. Figure 6a As shown. Using image processing techniques such as image rotation and affine transformation, the vertebral body orientation is corrected to a standard position perpendicular to the image plane, resulting in a segmentation mask image of the sagittal plane after vertebral body correction, which serves as the correction result, as shown. Figure 6bAs shown. During the correction process, it is necessary to ensure that the overall shape and internal structure of the vertebral body do not twist or deform.

[0195] Based on the correction results, S512 selects the middle layer of each target vertebra to calculate the CT value, and obtains the CT value of T12 vertebra, L1 vertebra and L2 vertebra respectively;

[0196] Specifically, the CT value of the T12 vertebra is calculated by selecting the middle layer of the T1 vertebra; the CT value of the L1 vertebra is calculated by selecting the middle layer of the L2 vertebra; and the CT value of the L2 vertebra is calculated by selecting the middle layer of the L2 vertebra.

[0197] In the above steps, by correcting the vertebral body orientation, the CT value is calculated separately for the middle cross section of each target vertebra, which enhances the consistency of the values ​​for different target vertebrae.

[0198] S513 obtains the CT value of the vertebral body according to the preset value acquisition strategy based on the CT value of the T12 vertebral body, the CT value of the L1 vertebral body, and the CT value of the L2 vertebral body.

[0199] S52 combines the muscle marking results to calculate the CT value of the target muscle, which is then used as the muscle CT value;

[0200] If the target muscle (i.e., the iliopsoas muscle) is missing, the first preset CT value is used as the muscle CT value. Preferably, the first preset CT value is 50 HU.

[0201] S53 Combines the fat marking results to calculate the CT value of the target fat, which is used as the fat CT value;

[0202] If there is a lack of target fat (i.e., subcutaneous fat in the target area), the second preset CT value will be used as the fat CT value. The second preset CT value is -105 HU.

[0203] S54 performs regression calculations on the vertebral body CT value, the muscle CT value, and the fat CT value to obtain a bone mineral density prediction value, which is used as the prediction result.

[0204] The CT values ​​of the vertebral body, muscle, and fat were used as independent variables; a ternary linear regression method was used to obtain the predicted value of bone mineral density based on these three independent variables.

[0205] In one embodiment of this specification, several historical samples are acquired. The historical samples include historical examination data of several subjects. The historical examination data includes: historical vertebral CT values, historical muscle CT values, historical fat CT values, and historical bone mineral density values.

[0206] A regression equation for bone mineral density prediction is established: Y = a1X1 + a2X2 + a3X3 + b; where a1, a2, and a3 are regression coefficients, and b is the intercept. Historical samples are substituted into the regression equation for bone mineral density prediction. Specifically, historical vertebral CT values ​​are used as X1, historical muscle CT values ​​as X2, and historical fat CT values ​​as X3. Least squares or other regression methods are used to fit the equation to obtain the optimal values ​​for a1, a2, a3, and b.

[0207] After determining the regression coefficients and intercepts, the vertebral CT values, muscle CT values, and fat CT values ​​can be substituted into the regression equation for bone mineral density prediction to obtain the predicted bone mineral density value.

[0208] This invention combines the target vertebral body and adjacent tissue regions (iliopsoas muscle and subcutaneous fat in the target area) as references for bone mineral density prediction. It effectively overcomes interference factors such as beam hardening during image scanning and avoids interference from air regions between the human body and the external phantom, thus improving the accuracy of bone mineral density prediction. Furthermore, the method is automated; for the same original medical image, the steps for bone mineral density prediction remain consistent, and the prediction results are reproducible and robust.

[0209] In routine CT scans, the technical solution of this invention can also be used to predict bone density. Based on the prediction results, users can promptly identify potential osteoporosis in the examinee and remind them to undergo further detailed examinations to reduce the risk of fractures. Furthermore, the method of this invention saves time and costs associated with additional examinations such as QCT, and reduces the risk of radiation exposure.

[0210] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this specification. The apparatus specifically includes:

[0211] The acquisition module 710 is used to acquire the original medical images of the subject being examined.

[0212] The first segmentation module 720 is used to perform a first segmentation process on the original medical image to obtain a first target object, and to mark the first target object to obtain a first marking result;

[0213] The classification module 730 is used to classify the first target object in the original medical image to obtain a classification result;

[0214] The second segmentation module 740 is used to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and to label the second target object to obtain a second labeling result;

[0215] The prediction module 750 is used to predict the original medical image of the examined object based on the first labeling result and the second labeling result, and obtain the prediction result.

[0216] Optionally, the first target object includes a plurality of target cones;

[0217] Optionally, the first segmentation module 720 includes:

[0218] The core segmentation submodule is used to locate the image region where the target vertebra is located in the original medical image, and use it as the core segmentation region;

[0219] The test area determination submodule is used to determine the test area of ​​each target vertebra based on the center point position of the core segmentation region of the target vertebra.

[0220] The first labeling submodule is used to perform instance segmentation on the test area of ​​the target vertebra and generate the first labeling result.

[0221] Optionally, the classification module 730 includes:

[0222] An image patch extraction submodule is used to extract image patches of the target vertebra from the original medical image based on the first labeling result;

[0223] The implant determination submodule is used to determine whether the target vertebral body contains an implant based on the CT values ​​of the pixels in the image block of the target vertebral body;

[0224] The fracture determination submodule is used to determine whether the target vertebra has a severe compression fracture based on the texture information of the target vertebra in the original medical image.

[0225] Optionally, the second target object includes: target muscle and target fat;

[0226] Optionally, the second segmentation module 740 includes:

[0227] When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using the muscle-fat segmentation model; wherein, the target muscle is labeled to obtain muscle labeling results; and the target fat is labeled to obtain fat labeling results.

[0228] Optionally, the prediction module 750 includes:

[0229] The first calculation submodule is used to combine the first marking result to calculate the CT value of each target vertebra and generate the vertebral CT value;

[0230] The second calculation submodule is used to calculate the CT value of the target muscle by combining the muscle marking results, and use it as the muscle CT value.

[0231] The third calculation submodule is used to combine the fat marking results to calculate the CT value of the target fat, which is then used as the fat CT value.

[0232] The prediction submodule is used to perform regression calculations on the vertebral body CT value, the muscle CT value, and the fat CT value to obtain a bone mineral density prediction value, which is used as the prediction result.

[0233] Optionally, the first computing submodule includes:

[0234] Orientation correction unit, used to correct the orientation of each of the target vertebrae;

[0235] The vertebral body CT value calculation unit is used to select the intermediate layer of each target vertebral body to calculate the CT value based on the correction result, and obtain the CT value of T12 vertebral body, L1 vertebral body and L2 vertebral body respectively.

[0236] The functions of the apparatus in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0237] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by a computer program / instruction. These computer programs / instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0239] These computer programs / instructions may also be stored in a readable storage medium of a computer device capable of directing a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the readable storage medium of the computer device produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0240] These computer programs / instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer equipment or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer equipment or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0241] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0242] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An image processing method, characterized in that, include: Acquire the original medical images of the subject; The original medical image is subjected to a first segmentation process to obtain a first target object, and the first target object is marked to obtain a first marking result; the first target object includes a plurality of target vertebrae; The first target object in the original medical image is classified to obtain a classification result; the classification result is a lesion state classification, wherein when there are no implants in the target vertebrae and no severe compression fractures have occurred in the target vertebrae, the classification result of the target vertebrae is identified as the first lesion state; when there are implants in the target vertebrae and / or a severe compression fracture has occurred in the target vertebrae, the classification result of the target vertebrae is identified as the second lesion state; Based on the classification results, the original medical image is subjected to a second segmentation process to obtain a second target object, and the second target object is labeled to obtain a second labeling result; the second target object includes: target muscle and target fat; Based on the first and second labeling results, the original medical image of the examined object is predicted to obtain a prediction result; specifically: Orientation correction is performed on each target vertebra; based on the correction results, the CT value is calculated on the middle cross section of each target vertebra, and the CT values ​​of T12 vertebra, L1 vertebra and L2 vertebra are obtained respectively; wherein, according to the preset value selection strategy, the CT value of the vertebra is obtained based on the CT value of T12 vertebra, the CT value of L1 vertebra and the CT value of L2 vertebra; Based on the muscle marking results, the CT value of the target muscle is calculated and used as the muscle CT value; if the target muscle is missing, the first preset CT value is used as the muscle CT value. Based on the fat marking results, the CT value of the target fat is calculated and used as the fat CT value; if there is a lack of target fat, the second preset CT value is used as the fat CT value. Regression calculations were performed on the vertebral body CT value, the muscle CT value, and the fat CT value. Using the vertebral body CT value, muscle CT value, and fat CT value as independent variables, a ternary linear equation regression method was used to obtain the predicted value of bone mineral density, which was then used as the prediction result.

2. The image processing method as described in claim 1, characterized in that, The first segmentation process of the original medical image to obtain a first target object, and the marking of the first target object to obtain a first marking result, includes: In the original medical image, the image region where the target vertebra is located is identified as the core segmentation region; Based on the center point location of the core segmentation region of the target vertebra, the test area of ​​each target vertebra is determined; The test region of the target vertebra is segmented to generate the first labeling result.

3. The image processing method as described in claim 1, characterized in that, The classification process of the first target object in the original medical image to obtain the classification result includes: Based on the first labeling result, an image patch of the target vertebra is extracted from the original medical image; Based on the CT values ​​of the pixels in the image block of the target vertebral body, determine whether the target vertebral body includes an implant; Based on the texture information of the target vertebra in the original medical image, it is determined whether the target vertebra has a severe compression fracture.

4. The image processing method as described in claim 1, characterized in that, The step of performing a second segmentation process on the original medical image based on the classification result to obtain a second target object, and then labeling the second target object to obtain a second labeling result, includes: When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using a muscle-fat segmentation model; wherein, the target muscle is labeled to obtain muscle labeling results; and the target fat is labeled to obtain fat labeling results. The muscle-fat segmentation model is a 3D U-shaped encoder-decoder network.

5. The image processing method as described in claim 4, characterized in that, The target muscle includes the iliopsoas muscle; the target fat includes subcutaneous fat in the target area. When the target vertebra is in the first lesion state, the original medical image is segmented and labeled using the muscle-fat segmentation model, including: The muscle-fat segmentation model extracts features from the original medical image, using an encoder and convolutional layers to extract semantic features. A nonlocal module captures global features between the encoder and decoder. The decoder upsamples the extracted features to gradually restore the image resolution and finally outputs the labeling results of the iliopsoas muscle and the subcutaneous fat of the target area. The labeling results of the iliopsoas muscle are used as the muscle labeling results, and the labeling results of the subcutaneous fat of the target area are used as the fat labeling results.

6. The image processing method as described in claim 1, characterized in that, The directional correction for each target vertebra includes: determining whether the sagittal plane of each target vertebra is in a standard position, and then performing directional correction for each target vertebra; specifically: Based on the boundary and shape information of the vertebral body, the actual orientation of the vertebral body is determined, and a segmentation mask of the sagittal plane before vertebral body correction is obtained. Using image processing technology, the orientation of the vertebral body is corrected to a standard position perpendicular to the image plane, and a segmentation mask of the sagittal plane after vertebral body correction is obtained as the correction result.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the original medical images of the examined object; The first segmentation module is used to perform a first segmentation process on the original medical image to obtain a first target object, and to mark the first target object to obtain a first marking result; the first target object includes a plurality of target vertebrae; A classification module is used to classify the first target object in the original medical image to obtain a classification result; the classification result is a lesion state classification, wherein when there are no implants in the target vertebrae and no severe compression fractures have occurred in the target vertebrae, the classification result of the target vertebrae is identified as a first lesion state; when there are implants in the target vertebrae and / or a severe compression fracture has occurred in the target vertebrae, the classification result of the target vertebrae is identified as a second lesion state; The second segmentation module is used to perform a second segmentation process on the original medical image based on the classification result to obtain a second target object, and to label the second target object to obtain a second labeling result; the second target object includes: target muscle and target fat; The prediction module is used to predict the original medical image of the examined object based on the first labeling result and the second labeling result, and obtain the prediction result; specifically: Orientation correction is performed on each target vertebra; based on the correction results, the CT value is calculated on the middle cross section of each target vertebra, and the CT values ​​of T12 vertebra, L1 vertebra and L2 vertebra are obtained respectively; wherein, according to the preset value selection strategy, the CT value of the vertebra is obtained based on the CT value of T12 vertebra, the CT value of L1 vertebra and the CT value of L2 vertebra; Based on the muscle marking results, the CT value of the target muscle is calculated and used as the muscle CT value; if the target muscle is missing, the first preset CT value is used as the muscle CT value. Based on the fat marking results, the CT value of the target fat is calculated and used as the fat CT value; if there is a lack of target fat, the second preset CT value is used as the fat CT value. Regression calculations were performed on the vertebral body CT value, the muscle CT value, and the fat CT value. Using the vertebral body CT value, muscle CT value, and fat CT value as independent variables, a ternary linear equation regression method was used to obtain the predicted value of bone mineral density, which was then used as the prediction result.

8. A computer device, characterized in that, The computer device includes: Processor; and, A memory storing computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs / instructions, which, when executed by a processor, implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, implements the method as described in any one of claims 1-6.

Images