Image processing method, training method of image processing model, and related devices

Abstract

The present application discloses an image processing method, a training method for an image processing model, and related devices. The method includes: obtaining a target medical image including a target bone part; performing detection on the target medical image to obtain prediction probabilities of the target bone part belonging to each preset category, where at least one preset category is a subclass of a preset bone state type and at least one preset category is a subclass of a reference bone state type; and obtaining a detection result of the target medical image based on the prediction probabilities of the target bone part belonging to each preset category, where the detection result of the target medical image is used to indicate whether a preset bone state exists in the target bone part in the target medical image. By the above method, the present application can improve the accuracy of image detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and particularly to an image processing method, a training method for an image processing model, and related devices. Background Art

[0002] With the growth of age and the loss of bone mass, the bone structure of the human body often undergoes osteoporotic compression fractures, resulting in kyphosis. In severe cases, it may even cause pain, making it impossible for people to sit, stand, or walk for a long time. The vertebra is an important bone structure of the human torso. The vertebral body is a thick oval structure at the front of the vertebra, which plays a major load-bearing role and is a high-incidence site for fractures.

[0003] In clinical practice, the bone structure of the human body can be observed from conventional CT scans. However, only when severe fractures induce other symptoms or cause pain will doctors carefully detect the fracture condition of the bone site at the request of the patient. If CT scan images can be fully utilized to automatically, accurately, and efficiently diagnose the fracture condition of the bone site, so as to achieve early prevention, early screening, and early treatment, the deterioration of the patient's bone site condition can be effectively avoided. Summary of the Invention

[0004] The present application provides at least an image processing method, a training method for an image processing model, and related devices.

[0005] In a first aspect of the present application, an image processing method is provided. The method includes: obtaining a target medical image including a target bone site; detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category, where at least one preset category is a subclass of a preset bone state type, and at least one preset category is a subclass of a reference bone state type; and obtaining a detection result of the target medical image based on the prediction probabilities of the target bone site belonging to each preset category, where the detection result of the target medical image is used to indicate whether a preset bone state exists in the target bone site in the target medical image.

[0006] Therefore, the prediction probabilities of the target bone site belonging to each preset category are obtained by detecting the target medical image including the target bone site. Therefore, it is possible to judge the bone state corresponding to the preset category, realize the judgment of multiple-category bone states, and reduce the occurrence of missed detection or false detection of the preset bone state, that is, improve the accuracy of detecting the target medical image.

[0007] Among them, the image processing method further includes: obtaining a sample medical image containing a bone part; wherein, the sample medical image is labeled with the true bone state information of the bone part, and the true bone state information indicates whether a preset bone state exists in the bone part; using an image processing model to detect the sample medical image to obtain the prediction probability that the bone part belongs to each preset category; based on the prediction probability that the bone part belongs to each preset category, obtaining the sample detection result of the sample medical image; and adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result.

[0008] Therefore, the image processing model can be used to detect the target medical image to obtain the detection result. There are huge intra-class differences in both fractured bone parts and non-fractured bone parts. If the bone detection model is trained to directly predict the preset bone state and the reference bone state, that is, the bone detection model needs to forcefully classify different bone state situations with large intra-class differences into one category for learning, which is quite difficult, and the trained bone detection model is prone to prediction errors, resulting in missed or false detections of the preset bone state situation. In this solution, the image processing model predicts each preset category that is a subclass of the preset bone state and the reference bone state. Therefore, during training, learning is carried out for each preset category, and different bone state situations with large intra-class differences are classified into different categories for learning, reducing the training difficulty and making the prediction of the trained image processing model more accurate. Subsequently, based on the relatively accurate probabilities of each preset category obtained by prediction, it is determined whether there is a preset bone state situation, which can achieve accurate identification of the preset bone state situation and reduce missed or false detections of the preset bone state situation.

[0009] Among them, the image processing model includes several classifiers, each classifier corresponding to a preset category and used to determine the prediction probability of belonging to the preset category. The network parameters include the classification parameters of each classifier; adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result includes: based on the difference between the true bone state information and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, adjusting the network parameters of the image processing model.

[0010] Therefore, the adjustment of the classification parameters of each classifier is jointly restricted by the difference between the true bone state information and the sample detection result labeled in the sample medical image and the difference between the classification parameters of each classifier.

[0011] Among them, based on the difference between the true bone state information and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, the network parameters of the image processing model are adjusted, including: obtaining a detection loss based on the difference between the true bone state information and the sample detection result; determining the target adjustment range of each network parameter based on the detection loss, and within the target adjustment range, determining the parameter values of each network parameter that minimize the constraint loss function, and adjusting each network parameter according to the determined parameter values, where the constraint loss function is negatively correlated with the difference between the classification parameters of each classifier.

[0012] Therefore, within the target adjustment range of each network parameter determined based on the detection loss, the parameter values of the classification parameters of each classifier that minimize the constraint loss function are determined, so that the distinguishability between each classifier after adjusting the network parameters is increased, that is, the difference between each classifier is increased, thereby ensuring that the potential categories learned by the image processing model are meaningful.

[0013] Among them, obtaining the target medical image including the target bone part includes: obtaining the original medical image; determining the position information of the target bone part in the original medical image; and extracting the target medical image including the target bone part from the original medical image based on the position information.

[0014] Therefore, the target medical image is smaller in size than the original medical image, improving the subsequent operation efficiency, reducing the subsequent running time and video memory occupancy, and also being beneficial to improving the judgment of the bone state of the target bone part.

[0015] Among them, the direction of the target bone part in the extracted target medical image is corrected, and / or the target bone part is located at a preset position in the target medical image.

[0016] Therefore, by setting that the direction of the extracted target bone part is corrected and / or the target bone part is located at a preset position in the target medical image, the directions of the target bone parts in each target medical image are the same and / or the positions of the target bone parts in the corresponding target medical images are the same, reducing the interference of the orientation information of the target bone part on the judgment of the fracture condition of the target bone part, thereby improving the accuracy of the judgment of the bone state of the target bone part.

[0017] Among them, the perpendicular bisector of the corrected target bone part is parallel to the preset coordinate axis of the original medical image; the preset position is the central position.

[0018] Therefore, the direction of the target bone part can be corrected based on the perpendicular bisector of the target bone part; the preset position can be set as the central position to facilitate the adjustment of the position of the target bone part.

[0019] Among them, determining the position information of the target bone site in the original medical image includes: performing instance segmentation on the original medical image to obtain a first segmentation result of the target bone and a second segmentation result of the reference bone in the original medical image, where the target bone site is located on the target bone; determining the position information of the target bone site from the target bone in the original medical image based on the first segmentation result and the second segmentation result; and / or, before determining the position information of the target bone site in the original medical image, the image processing method further includes: detecting whether there are abnormalities in the image parameters of the original medical image, where the image parameters include at least one of image modality, image slice interval, and image size; in response to the image parameters being normal, performing the determination of the position information of the target bone site in the original medical image and subsequent steps.

[0020] Therefore, since the relative positions of the target bone and the reference bone are relatively fixed, using the second segmentation result of the reference bone is beneficial to separating the target bone site from the target bone. In addition, by detecting abnormalities in the image parameters of the original medical image, it is ensured that the image parameters of the original medical image are normal, thereby guaranteeing that the position information of the target bone site in the original medical image can be determined in a timely and accurate manner subsequently.

[0021] Among them, before detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category, the image processing method further includes: preprocessing the target medical image, where the preprocessing includes at least one of image relocation, image resampling, and pixel normalization; and / or, detecting whether the target bone site in the target medical image includes an implant; in the case where the target bone site does not include an implant, detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category and subsequent steps.

[0022] Therefore, by preprocessing the target medical image, the target medical image is standardized and unified, facilitating subsequent detection of the preset bone state for the target medical image. In addition, by detecting whether the target bone site in the target medical image includes an implant, the interference of the implant on the preset bone state detection can be effectively excluded, enabling more attention to be paid to those bone sites to be treated and discovered during the preset bone state detection, and obtaining detection results that are more in line with clinical cognition.

[0023] Among them, based on the prediction probabilities of the target bone part belonging to each preset category, the detection result of the target medical image is obtained, including: selecting the target preset category from each preset category based on the prediction probabilities of the target bone part belonging to each preset category; wherein, the target preset category belongs to a subclass of the preset bone state type and the prediction probability of the target preset category is the largest among the preset categories belonging to the subclass of the preset bone state type; in response to the prediction probability of the target preset category being greater than or equal to the preset probability value, it is determined that the target bone part has the preset bone state.

[0024] Therefore, it is possible to directly determine whether the target bone part has the preset bone state according to the probability of the preset category belonging to the subclass of the fracture type.

[0025] Among them, the target bone part is a vertebral body; and / or, the target medical image is a three-dimensional medical image.

[0026] Therefore, it can be used to judge whether there is a preset bone state phenomenon in the vertebral body. In addition, the target medical image is a three-dimensional image, so that when subsequently determining whether the target bone part has the preset bone state based on the three-dimensional medical image, the rich geometric and texture information of the three-dimensional medical image can be utilized to improve the accuracy of judging the preset bone state of the target bone part.

[0027] The second aspect of this application provides a training method for an image processing model. The method includes: obtaining a sample medical image containing a bone part; wherein, the sample medical image is labeled with the true bone state information of the bone part, and the true bone state information indicates whether the bone part has the preset bone state; using the image processing model to detect the sample medical image to obtain the prediction probabilities of the bone part belonging to each preset category, wherein at least one preset category is a subclass of the preset bone state type and at least one preset category is a subclass of the reference bone state type; based on the prediction probabilities of the bone part belonging to each preset category, obtaining the sample detection result of the sample medical image, wherein the sample detection result of the sample medical image is used to represent whether the bone part in the sample medical image has the preset bone state; adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result.

[0028] Therefore, due to the large intra-class differences in both fractured bone regions and non-fractured bone regions, if an image processing model is trained to directly predict the preset bone state and the reference bone state, that is, the image processing model needs to force different bone state situations with large intra-class differences to be classified into one category for learning, which is quite difficult. Moreover, the trained image processing model is prone to prediction errors, resulting in missed or misdetected bone state situations. In this solution, the image processing model predicts each preset category that is a subclass of the preset bone state and the reference bone state. Therefore, during training, learning is carried out for each preset category, and different bone state situations with large intra-class differences are classified into different categories for learning, reducing the training difficulty and making the prediction of the trained image processing model more accurate. Subsequently, based on the relatively accurate probabilities of each preset category obtained by prediction, it is determined whether there is a preset bone state situation, enabling the accurate identification of the preset bone state situation and reducing the missed or misdetected of the preset bone state situation.

[0029] Among them, the image processing model includes several classifiers, each classifier corresponding to a preset category and used to determine the prediction probability belonging to the preset category. The network parameters include the classification parameters of each classifier; based on the difference between the true bone state information and the sample detection result, the network parameters of the image processing model are adjusted, including: based on the difference between the true bone state information and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, the network parameters of the image processing model are adjusted.

[0030] Therefore, the adjustment of the classification parameters of each classifier is jointly constrained by the difference between the true bone state information and the sample detection result in the sample medical image annotation and the difference between the classification parameters of each classifier.

[0031] The third aspect of this application provides an image processing device, which includes: an acquisition module for acquiring a target medical image including a target bone region; a detection module for detecting the target medical image to obtain the prediction probability that the target bone region belongs to each preset category, where at least one preset category is a subclass of the preset bone state type and at least one preset category is a subclass of the reference bone state type; a determination module for obtaining the detection result of the target medical image based on the prediction probability that the target bone region belongs to each preset category, where the detection result of the target medical image is used to indicate whether there is a preset bone state in the target bone region of the target medical image.

[0032] A fourth aspect of the present application provides a training device for an image processing model. The device includes: an acquisition module configured to acquire a sample medical image including a bone part, where the sample medical image is labeled with true bone state information of the bone part, and the true bone state information indicates whether a preset bone state exists in the bone part; a detection module configured to use the image processing model to detect the sample medical image to obtain prediction probabilities of the bone part belonging to each preset category, where at least one preset category is a subclass of a preset bone state type and at least one preset category is a subclass of a reference bone state type; a determination module configured to obtain a sample detection result of the sample medical image based on the prediction probabilities of the bone part belonging to each preset category, where the sample detection result of the sample medical image is used to indicate whether a preset bone state exists in the bone part of the sample medical image; and an adjustment module configured to adjust network parameters of the image processing model based on a difference between the true bone state information and the sample detection result.

[0033] A fifth aspect of the present application provides an electronic device. The electronic device includes a memory and a processor. The memory stores program instructions, and the processor is configured to execute the program instructions to implement the above-mentioned image processing method or the above-mentioned training method for the image processing model.

[0034] A sixth aspect of the present application provides a computer-readable storage medium. The computer-readable storage medium is configured to store program instructions that can be executed to implement the above-mentioned image processing method or the above-mentioned training method for the image processing model.

[0035] In the above solution, the prediction probabilities of the target bone part belonging to each preset category are obtained by detecting a target medical image including the target bone part. Therefore, it is possible to judge the bone state corresponding to the preset category, realize the judgment of the multi-category bone state, and reduce the occurrence of missed detection or false detection of the preset bone state, that is, improve the accuracy of detecting the target medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of an embodiment of the image processing method provided by the present application;

[0037] Figure 2 is a flowchart of an embodiment of detecting whether an implant is included in a target bone part provided by the present application;

[0038] Figure 3 is Figure 1 a flowchart of an embodiment of step S11 shown;

[0039] Figure 4 is Figure 3 a flowchart of an embodiment of step S112 shown;

[0040] Figure 5 It is a schematic flowchart of an embodiment for determining whether there is an abnormality in the image parameters of the original medical image provided by this application;

[0041] Figure 6 It is a schematic diagram of an embodiment of the vertebral body before correction provided by this application;

[0042] Figure 7 It is a schematic structural diagram of an embodiment of the corrected vertebral body provided by this application;

[0043] Figure 8 It is Figure 1 A schematic flowchart of an embodiment of step S13 shown;

[0044] Figure 9 It is a schematic flowchart of an embodiment of the training step of the image processing model provided by this application;

[0045] Figure 10 It is a schematic network structure diagram of an embodiment of the image processing model provided by this application;

[0046] Figure 11 It is a schematic flowchart of an embodiment for adjusting the network parameters of the image processing model provided by this application;

[0047] Figure 12 It is a schematic flowchart of an embodiment of the training method of the image processing model provided by this application;

[0048] Figure 13 It is a schematic structural diagram of an embodiment of the image processing device provided by this application;

[0049] Figure 14 It is a schematic structural diagram of an embodiment of the training device of the image processing model provided by this application;

[0050] Figure 15 It is a schematic structural diagram of an embodiment of the electronic device provided by this application;

[0051] Figure 16 It is a schematic structural diagram of an embodiment of the computer-readable storage medium provided by this application. Detailed implementation manners

[0052] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings of the specification.

[0053] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand this application.

[0054] In this text, the term "and / or" is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, in this text, the character " / " generally indicates that the associated objects before and after are in an "or" relationship. Furthermore, "multiple" in this text means two or more than two. Additionally, the term "at least one" in this text means any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0055] Please refer to Figure 1 , Figure 1 is a schematic flowchart of an embodiment of the image processing method provided by this application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the shown process sequence. As Figure 1 shown, this embodiment includes:

[0056] Step S11: Obtain a target medical image including a target bone part.

[0057] The method of this embodiment is used to detect whether a preset bone state exists in the target bone part in the target medical image. The target bone part described in this text includes but is not limited to vertebral bodies (such as vertebral bodies of the cervical spine, thoracic spine, lumbar spine, etc.), joints (such as elbow joints, shoulder joints, or hip joints, etc.), etc., and no specific limitation is made here. The preset bone state described in this text can be a fracture state or a non-fracture state.

[0058] In one embodiment, the target medical image is a three-dimensional medical image. The three-dimensional medical image has rich geometric and texture information, etc., enabling the subsequent determination of whether there is a fracture in the target bone part based on the rich geometric and texture information of the three-dimensional medical image, improving the accuracy of judging the fracture situation of the target bone part. Among them, the target medical image includes but is not limited to computed tomography images (CT images), magnetic resonance imaging (MRI images), positron emission tomography images (PET images), etc., and no specific limitation is made here. It can be understood that in other specific embodiments, the target medical image can also be a two-dimensional medical image, such as X-ray photographic images, etc.

[0059] Among them, it should be noted that in the following text, for the convenience of description, this application will take the target bone part as the vertebral body and the target medical image as the CT image as an example for description, but it can be understood that such a description does not limit the types of the target bone part and the target medical image.

[0060] In one embodiment, the target medical image including the target bone site can be specifically obtained from local storage or cloud storage. It can be understood that, in other embodiments, the target medical image including the target bone site can also be obtained by using a medical image acquisition device to capture the target bone site in real time.

[0061] In one embodiment, the obtained original medical image including the target bone site can be directly used as the target medical image. Since the original medical image is large, directly judging the bone state of the target bone site based on the original medical image will increase the computational workload, running time, and video memory occupancy; in addition, since the original medical image may include multiple bone sites, it is not conducive to accurately judging the bone state of the target bone site; therefore, in other embodiments, after obtaining the original medical image, the position of the target bone site in the original medical image will be determined, and then the target medical image including the target bone site will be extracted from the original medical image, and then the image block corresponding to the target bone site will be obtained. The image block corresponding to the target bone site is used as the target medical image including the target bone site, which reduces the size of the medical image compared to the original medical image, saves subsequent running time, computational amount, and video memory occupancy, and is also conducive to improving the accuracy of judging the preset bone state of the target bone site.

[0062] Step S12: Detect the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category.

[0063] In this embodiment, the target medical image is detected to obtain the prediction probabilities of the target bone site belonging to each preset category. Among them, at least one preset category is a subclass of the preset bone state type, and at least one preset category is a subclass of the reference bone state type.

[0064] Among them, the number of preset categories is not limited and can be specifically set according to actual usage needs. For example, set 3 preset categories corresponding to the subclasses of the preset bone state type, and set 3 preset categories corresponding to the subclasses of the reference bone state type. It should be noted that since detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category is an abstract unsupervised clustering, the specific names of the preset categories cannot be determined. Exemplarily, it can be unsupervised clustered into local bone fractures, compression fractures, healthy vertebrae, Schmorl's nodes, and metastases, etc.

[0065] In one embodiment, the target medical image can be directly detected by an image detection algorithm. Among them, the image detection algorithm is not limited and can be specifically set according to actual usage needs. It can be understood that, in other embodiments, an image processing model can also be used to obtain the target medical image and detect the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category, which will not be specifically limited here.

[0066] In one embodiment, multiple target medical images containing the target bone site can be input into the image processing model simultaneously. The image processing model can detect multiple target medical images simultaneously, that is, multi-target bone site detection is performed in parallel, saving the running time of detection and improving the detection efficiency. For example, taking the vertebral body as the target bone site, since the CT image of the human body is large, it can be segmented into several target medical images containing a single vertebral body. By inputting these several target medical images into the image processing model, the image processing model can detect several vertebral bodies simultaneously, and then integrate the detection results to obtain the bone status of the vertebral bodies in the entire CT image, reducing the time for detecting each target medical image and improving the detection efficiency.

[0067] In order to standardize and unify the target medical images, facilitate the detection of the target medical images, and improve the detection efficiency. Therefore, in one embodiment, before detecting the target medical image to obtain the prediction probability that the target bone site belongs to each preset category, preprocessing is performed on the target medical image. Among them, the preprocessing includes at least one of image repositioning, image resampling, and pixel normalization. Pixel normalization can be to adjust the pixel values to a preset range based on the selected window width and window level. For example, if the selected window width and window level are [1500, 450], the pixel values of the image are normalized to between [-1, 1]; image repositioning can be to adjust the direction of the image to be consistent with the identity matrix, and image repositioning can be achieved through operations such as rotation and interpolation.

[0068] Since when an implant has been inserted into the target bone site, it means that there was probably a lesion in the target bone site before and it has been treated. In order to effectively exclude the interference of the implant when detecting the target medical image, pay more attention to those target bone sites to be treated and discovered, and obtain a detection result that is more in line with clinical cognition. Therefore, in one embodiment, as Figure 2 shown, Figure 2 FIG. is a schematic flowchart of an embodiment for detecting whether the target bone site includes an implant provided by the present application. Before detecting the target medical image, it is necessary to detect whether the target bone site includes an implant, which specifically includes the following sub-steps:

[0069] Step S21: Detect whether the target bone site in the target medical image includes an implant.

[0070] In this embodiment, it is detected whether the target bone part in the target medical image includes an implant. When the target bone part in the target medical image does not include an implant, step S22 is executed; while when the target bone part in the target medical image includes an implant, it indicates that there was a lesion in the target bone part in the target medical image and it has been treated, and there is no need to judge the subsequent bone state, and the detection method stops executing at this time.

[0071] In one embodiment, when the target medical image is a CT image, it is possible to determine whether the target bone part in the target medical image includes an implant by determining the CT value of the pixel points in the target medical image. Specifically, it is statistically determined whether the number of pixel points with a CT value greater than 2000 in the target medical image is greater than a preset threshold. Among them, the preset threshold is not limited and can be specifically set according to actual usage needs. For example, the preset threshold is 500, 550, or 600, etc. For example, taking the target medical image as a CT image and the preset threshold as 500 as an example, the number of pixel points with a CT value greater than 2000 in the target medical image is counted as num HU>2000 (where HU is the unit of CT value); when the number of pixel points with a CT value greater than 2000 in the target medical image is greater than 500, that is, num HU>2000 >500, it is determined that the target bone part in the target medical image includes an implant; while when the number of pixel points with a CT value greater than 2000 in the target medical image is less than or equal to 500, that is, num HU>2000 ≤500, it is determined that the target bone part in the target medical image does not include an implant.

[0072] Step S22: In response to the target bone part not including an implant, perform detection on the target medical image to obtain the prediction probabilities of the target bone part belonging to each preset category and its subsequent steps.

[0073] In this embodiment, in response to the target bone part not including an implant, perform detection on the target medical image to obtain the prediction probabilities of the target bone part belonging to each preset category and its subsequent steps. That is to say, when the target bone part does not include an implant, the interference of the implant is effectively excluded, so that when the target bone part is detected subsequently, more attention can be paid to the target bone part, avoiding false positive misjudgments, and the obtained detection results are more in line with the judgment results of clinical cognition.

[0074] Step S13: Based on the prediction probabilities of the target bone part belonging to each preset category, obtain the detection result of the target medical image.

[0075] In this embodiment, based on the prediction probabilities of the target bone part belonging to each preset category, the detection result of the target medical image is obtained. Among them, the detection result of the target medical image is used to indicate whether the target bone part in the target medical image has a preset bone state. That is to say, according to the prediction probability values of the target bone part belonging to each preset category, it can be determined whether the target bone part in the target medical image has a preset bone state. Since the prediction probabilities of the target bone part belonging to each preset category are obtained by detecting the target medical image containing the target bone part, this application can judge the bone state corresponding to the preset category, that is, realize the judgment of the multi-category bone state, reduce the occurrence of missed detection or misdetection of the preset bone state, that is, improve the accuracy of detecting the target medical image, so as to realize the early detection and early treatment of the preset bone state of the target bone part, and further avoid the further deterioration of the condition.

[0076] In a specific embodiment, the image processing model obtains the detection result of the target medical image based on the prediction probabilities of the target bone part belonging to each preset category.

[0077] In one embodiment, the detection result of the target medical image can be determined according to the type to which the preset category corresponding to the largest prediction probability in the prediction probabilities belongs. It can be understood that in other embodiments, subclasses belonging to the same type can also be classified, and then the probability that the target bone part has a preset bone state can be determined according to the subclasses classified into the preset bone state type, and the probability that the target bone part does not have a preset bone state can be determined according to the subclasses classified into the reference bone state type. Then, the detection result of the target medical image is determined according to the probability that the target bone part has a preset bone state and the probability that the target bone part does not have a preset bone state.

[0078] In one embodiment, when it is determined that the target bone part in the target medical image has a preset bone state, the target bone part can be highlighted or the specific position of the target bone part can also be described in the disease diagnosis report in text form to remind the user which bone parts have preset bone state lesions, realize the early detection and early treatment of the preset bone state, and thus avoid the further deterioration of the condition.

[0079] In the above embodiment, the prediction probabilities of the target bone part belonging to each preset category are obtained by detecting the target medical image containing the target bone part. Therefore, it is possible to judge the preset bone state corresponding to the preset category, realize the judgment of the multi-category bone state, reduce the occurrence of missed detection or misdetection of the preset bone state, that is, improve the accuracy of detecting the target medical image, so as to realize the early detection and early treatment of the preset bone state of the target bone part, and further avoid the further deterioration of the condition.

[0080] Please refer to Figure 3 , Figure 3 isFigure 1 The flowchart of an embodiment of step S11 is shown. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 3 the shown process sequence. For example, Figure 3 as shown, in this embodiment, based on the position information of the target bone part in the original medical image, a target medical image including the target bone part is extracted from the original medical image, which specifically includes:

[0081] Step S111: Obtain the original medical image.

[0082] In one embodiment, the original medical image is a three-dimensional medical image. The three-dimensional medical image has rich geometric and texture information, etc., so that the target medical image including the target bone part extracted from the original medical image later is a three-dimensional medical image. Furthermore, when determining whether there is a fracture in the target bone part based on the three-dimensional medical image, the rich geometric and texture information of the three-dimensional medical image can be utilized to improve the accuracy of judging the preset bone state of the target bone part. Among them, the original medical image includes but is not limited to computed tomography images (CT images), magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, etc., and no specific limitation is made here. It can be understood that in other specific embodiments, the original medical image can also be a two-dimensional medical image, for example, X-ray photographic images, etc.

[0083] It should be noted that hereinafter, for the convenience of description, this application will take the target bone part as the vertebral body and the original medical image as the CT image as an example for description, but it can be understood that such a description does not limit the types of the target bone part and the original medical image.

[0084] In one embodiment, the original medical image can specifically be obtained from local storage or cloud storage. It can be understood that in other embodiments, the original medical image can also be obtained by using a medical image acquisition device to take a real-time photograph of the target bone part.

[0085] Step S112: Determine the position information of the target bone part in the original medical image.

[0086] In this embodiment, determining the position information of the target bone part in the original medical image, that is, determining the specific position of the target bone part in the original medical image, so as to be able to accurately extract the target medical image including the target bone part from the original medical image later.

[0087] In one embodiment, the position information of the target bone part in the original medical image can be determined according to the segmentation results of the target bone where the target bone part is located and the reference bone. It can be understood that in other embodiments, the position information of the target bone part in the original medical image can also be directly determined according to the segmentation result of the target bone where the target bone part is located, and no specific limitation is made here.

[0088] Please refer to Figure 4 , Figure 4 which Figure 3 is a schematic flowchart of an embodiment of step S112 shown. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 4 the Figure 4 flow sequence shown. As

[0089] shown, in this embodiment, according to the segmentation results of the target bone where the target bone part is located and the reference bone, the position information of the target bone part in the original medical image is determined, which specifically includes the following sub-steps:

[0090] Step S41: Perform instance segmentation on the original medical image to obtain a first segmentation result of the target bone and a second segmentation result of the reference bone in the original medical image.

[0091] Among them, the target bone part is located on the target bone. Since the relative positions of the target bone and the reference bone are relatively fixed, using the segmentation result of the reference bone is beneficial to separating the target bone part from the target bone. For example, taking the target bone as the vertebra and the reference bone as the rib, since the relative positions of the rib and the vertebra are relatively fixed, using the segmentation result of the rib is beneficial to separating the vertebral body from the vertebra, that is, it is beneficial to separating the appendages of the vertebra and the vertebral body.

[0092] Therefore, in this embodiment, first, perform instance segmentation on the original medical image to obtain a first segmentation result of the target bone and a second segmentation result of the reference bone in the original medical image.

[0093] Since the first segmentation result of the target bone and the second segmentation result of the reference bone are only used to subsequently determine the position information of the target bone part, so as to separate the target bone part from the target bone, that is, the accuracy requirements for the first segmentation result of the target bone and the second segmentation result of the reference bone in the obtained original medical image are not high. Therefore, in one embodiment, instance segmentation of the original medical image can be performed at a low resolution. Of course, in other embodiments, instance segmentation of the original medical image can also be performed at a high resolution as needed, which is not specifically limited herein. Among them, the specific resolution size is not limited and can be specifically set according to actual usage needs.

[0094] In one embodiment, after performing instance segmentation on the original medical image, if the first segmentation result of the target bone in the original medical image cannot be obtained, it indicates that the original medical image does not include the target bone. Since the target bone part is located on the target bone, it shows that the original medical image does not include the target bone part, and there is no need to perform subsequent detection of the target bone part and its subsequent steps. At this time, the image processing method is stopped.

[0095] Step S42: Based on the first segmentation result and the second segmentation result, determine the position information of the target bone part from the target bone in the original medical image.

[0096] In this embodiment, based on the first segmentation result and the second segmentation result, the position information of the target bone part is determined from the target bone in the original medical image. Since according to the first segmentation result of the target bone, the position of the target bone in the original medical image can be determined, and according to the second segmentation result of the reference bone, the position of the reference bone in the original medical image can be determined; moreover, the relative positions of the reference bone and the target bone are relatively fixed. Therefore, through the relative positions between the reference bone and the target bone, the position information of the target bone part can be determined from the target bone. Therefore, through the first segmentation result of the target bone and the second segmentation result of the reference bone, the position information of the target bone part can be determined from the target bone in the original medical image.

[0097] For example, taking the target bone as the vertebra and the reference bone as the rib, after performing instance segmentation on the original medical image, the first segmentation result of the vertebra and the second segmentation result of the rib are obtained, and using the second segmentation result of the rib, the position information of the vertebral body is determined from the vertebra in the original medical image, and then the vertebral body and the attachments connected to the vertebral body can be separated.

[0098] Since when there are abnormalities in the image parameters of the original medical image, the original medical image cannot be processed to obtain the position information of the target bone part in the original medical image. Therefore, as Figure 5 shown Figure 5It is a schematic flowchart of an embodiment for determining whether there are abnormalities in the image parameters of the original medical image provided by this application. In one implementation, before determining the position information of the target bone part in the original medical image, it is necessary to detect whether there are abnormalities in the image parameters of the original medical image, which specifically includes the following sub-steps:

[0099] Step S51: Detect whether there are abnormalities in the image parameters of the original medical image.

[0100] In this implementation, it is detected whether there are abnormalities in the image parameters of the original medical image. Among them, the image parameters include at least one of image modality, image slice interval, and image size. When there are no abnormalities in the image parameters of the original medical image, the type of image modality, the size of the image slice interval, and the size of the image size are not limited and can be specifically set according to actual usage needs. For example, taking the image parameters including image modality, image slice interval, and image size as an example, when the image modality is set to CT, the image slice interval ≤ 2 mm, and the image size (length / width / height) ≥ 20 mm, there are no abnormalities in the image parameters of the original medical image. Among them, when there are no abnormalities in the image parameters of the original medical image, step S52 is executed; while when there are abnormalities in the image parameters of the original medical image, the original medical image cannot be processed, and at this time, the execution of determining the position information of the target bone part in the original medical image and its subsequent steps is stopped.

[0101] In a specific implementation, the SimpleITK module can be used to detect whether there are abnormalities in the image parameters of the original medical image.

[0102] Step S52: In response to the absence of abnormalities in the image parameters, execute the steps of determining the position information of the target bone part in the original medical image and its subsequent steps.

[0103] In this implementation, in response to the absence of abnormalities in the image parameters, the steps of determining the position information of the target bone part in the original medical image and its subsequent steps are executed. That is to say, when the image parameters of the original medical image are normal, it is possible to subsequently determine the position of the target bone part in the original medical image based on the original medical image, so that the target medical image can be extracted from the original medical image.

[0104] Step S113: Based on the position information, extract the target medical image containing the target bone part from the original medical image.

[0105] In this embodiment, based on the position information of the target bone part in the original medical image, a target medical image containing the target bone part is extracted from the original medical image. Since the position information of the target bone part in the original medical image is determined, the position of the target bone part in the original medical image can be accurately located, so that the target medical image containing the target bone part can be accurately extracted according to the determined position. Compared with the original medical image, the target medical image containing the target bone part is smaller, saving running time and video memory occupancy. At the same time, the interference of other bone parts connected to the target bone part on the subsequent judgment of the bone state of the target bone part is reduced, and the accuracy of the subsequent judgment of whether the target bone part has a preset bone state is improved.

[0106] In one embodiment, the target medical image contains the complete target bone part, which is beneficial to the accurate judgment of the bone state of the target bone part. It can be understood that in other embodiments, the target medical image may also only contain a partial target bone part, which is not specifically limited here.

[0107] In one embodiment, according to the position information of the target bone part in the original medical image, that is, after determining the specific position of the target bone part in the original medical image, the original medical image can be cropped by cropping to obtain a target medical image containing the complete target bone part, so that there are fewer other bone parts connected to the target bone part in the target medical image, which is beneficial to the subsequent judgment of the bone state of the target bone part, and at the same time reduces the size of the target medical image, saving running time and video memory occupancy. For example, taking the target bone part as the vertebral body, the vertebra includes the vertebral body with a thick oval structure in the front part of the vertebra and relatively long attachments in the longitudinal direction. The existence of a large area of attachments will affect the judgment of the bone state of the vertebral body. Therefore, after determining the position of the vertebral body, the original medical image is cropped so that the obtained target medical image includes the complete vertebral body and a small part of the attachments or other vertebral bodies connected to the vertebral body, reducing the interference of the overly long attachments on the subsequent judgment of the bone state of the vertebral body, and at the same time the target medical image is relatively smaller than the original medical image, saving running time and video memory occupancy.

[0108] Due to the natural curvature of the target bone part, the radial direction of the target bone part is often not parallel to the Z-axis. For example, as Figure 6 shown, Figure 6This is a schematic diagram of a vertebral body before correction provided by the present application. The human spine has a natural curvature. In the coordinate system of the input CT image, the radial direction of the cylindrical vertebral body often is not parallel to the Z-axis. And the direction information of these target bone parts is redundant for judging the bone state of the target bone parts, which will bring additional interference to the judgment of the bone state or the training of the image processing model. Therefore, in one embodiment, the direction of the target bone part in the extracted target medical image is corrected to avoid the influence of the direction information on the target bone part on the judgment of the bone state of the target bone part.

[0109] In a specific embodiment, the perpendicular bisector of the corrected target bone part is parallel to the preset coordinate axis of the original medical image. Among them, the preset coordinate axis of the original medical image is not limited and can be specifically set according to actual use needs. For example, the preset coordinate axis is the Z-axis. For example, as Figure 7 shown, Figure 7 This is a schematic structural diagram of a corrected vertebral body provided by the present application. The perpendicular bisector of the vertebral body is extracted by using the singular value decomposition method, and the direction of the vertebral body is adjusted based on the perpendicular bisector of the vertebral body until the perpendicular bisector of the vertebral body is parallel to the Z-axis of the original medical image, thus completing the correction of the vertebral body.

[0110] Since there are multiple bone parts in the original medical image, each bone part needs to be used as a target bone part, and the target medical image corresponding to each target bone part is extracted from the original medical image, that is, multiple target medical images will be extracted at this time, that is, each target bone part corresponds to a target medical image. Subsequently, the bone state can be detected for a single target bone part, improving the accuracy of bone state detection. Therefore, in one embodiment, the target bone part is set at the preset position of the target medical image, so as to ensure the consistency of the data of each target medical image, which is helpful for judging the bone state of the target bone part, or when training the image processing model with medical images with data consistency, ensuring that the medical image data seen by the image processing model network is consistent, reducing the interference brought by unnecessary information to model training, so that a model with better detection effect can be trained under the condition of limited medical image data. Among them, the preset position is not limited and can be specifically set according to actual use needs. For example, the preset position is the central position of the target medical image, or the preset position can also be the lower left corner area position of the target medical image, etc.

[0111] In a specific embodiment, the target bone site is located at the center position of the target medical image. For example, taking the target medical image containing the target bone site extracted from the original medical image by cropping, with the preset position being the center position as an example, after determining the position of the target bone site in the original medical image, the original medical image is cropped with the center of the target bone site as the center, so that the obtained target medical image contains the complete target bone site and the center of the target medical image is the center of the target bone site, that is, the target bone site is located at the center position of the target medical image. Furthermore, when subsequently judging the bone state of the target bone site, more attention can be paid to the target bone site, while reducing the attention to other information such as the orientation information or direction information of the target bone site, thereby improving the accuracy of judging the bone state of the target bone site.

[0112] Please refer to Figure 8 , Figure 8 is Figure 1 a schematic flowchart of an embodiment of step S13 shown in the figure. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 8 the process sequence shown in the figure. As Figure 8 shown in the figure, in this embodiment, according to the prediction probability of the target bone site belonging to each preset category, it can be determined whether the target bone site of the target medical image has a preset bone state, which specifically includes:

[0113] Step S131: Based on the prediction probability of the target bone site belonging to each preset category, select the target preset category from each of the preset categories.

[0114] In this embodiment, according to the prediction probability of the target bone site belonging to each preset category, the target preset category is selected from each preset category, where the target preset category belongs to a subclass of the fracture type and the prediction probability of the target preset category is the largest among the preset categories belonging to the subclass of the fracture type. Specifically, from each preset category, select the preset categories corresponding to the subclasses of the preset bone state type; then, use the preset category with the largest prediction probability among the preset categories corresponding to the subclasses of the preset bone state type as the target preset category.

[0115] Step S132: In response to the prediction probability of the target preset category being greater than or equal to the preset probability value, determine that the target bone site has the preset bone state.

[0116] In this embodiment, in response to the prediction probability of the target preset category being greater than or equal to a preset probability value, it is determined that the target bone site has a preset bone state. Herein, the preset probability value is not limited and can be specifically set according to actual usage needs. That is to say, the prediction probability value of the target preset category is compared with the preset probability value. When the prediction probability of the target preset category is greater than or equal to the preset probability value, it is determined that the target bone site has the preset bone state; when the prediction probability of the target preset category is less than the preset probability value, it is determined that the target bone site does not have the preset bone state.

[0117] Please refer to Figures 9 - 10 , Figure 9 which is a schematic flowchart of a first embodiment of the training steps of the image processing model provided by this application. Figure 10 which is a schematic structural diagram of an embodiment of the image processing model provided by this application. It should be noted that if there are substantially the same results, this embodiment is not limited by Figure 9 the shown process sequence. As Figure 9 shown, in this embodiment, the bone state detection of the target medical image is performed using the image processing model. The training of the image processing model specifically includes the following sub-steps:

[0118] Step S91: Obtain a sample medical image containing a bone site.

[0119] The method of this embodiment is used to train the image processing model based on the sample medical image containing the bone site. In this article, the number, size, etc. of the sample medical images are not limited and can be specifically set according to actual usage needs. Among them, the sample medical image containing the bone site is labeled with the true bone state information of the bone site, and the true bone state information indicates whether the bone site has a preset bone state, so that the image processing model trained based on the sample medical image containing the bone site can detect the target bone site in the target medical image and determine whether the target bone site in the target medical image has a preset bone state.

[0120] In one embodiment, the sample medical image containing the bone site can be specifically obtained from local storage or cloud storage. It can be understood that in other embodiments, the sample medical image containing the bone site can also be obtained by real-time acquisition using a medical image acquisition device.

[0121] In one embodiment, the obtained original sample medical image including the bone part can be directly used as the sample medical image. Since the original sample medical image is large, directly judging the bone state of the bone part based on the original sample medical image will increase the amount of computation, running time, and video memory occupancy. Additionally, since the original sample medical image may include multiple bone parts, it is not conducive to accurately judging the bone state of the bone part. Therefore, in other embodiments, after obtaining the original sample medical image, the position of the bone part in the original sample medical image is determined, and then the sample medical image containing the bone part is extracted from the original sample medical image, and then the image block corresponding to the bone part is obtained. The image block corresponding to the bone part is used as the sample medical image containing the bone part, reducing the size of the sample medical image, saving subsequent running time, computational amount, and video memory occupancy, and also being conducive to improving the accurate judgment of the bone state of the bone part.

[0122] In order to make the subsequent trained image processing model have stronger generalization ability, the sample medical images should be made as diverse as possible. Therefore, in one embodiment, it is necessary to perform data augmentation processing on the sample medical images containing bone parts to generate several sample medical images, thereby diversifying the sample medical image data. In a specific embodiment, diverse sample medical image data can be generated by performing left-right flipping, rotation, cropping, or adding Gaussian noise to the sample medical images.

[0123] Since the positions of the bone parts on each sample medical image may be different or the directions of the bone parts on each sample medical image are different, and this direction information of the bone parts is redundant for the subsequent training of the image processing model and will bring additional interference to the training of the image processing model. Therefore, in one embodiment, the direction of the bone part in the extracted sample medical image is corrected. For example, the perpendicular bisector of the corrected bone part is parallel to the Z-axis coordinate axis of the original sample medical image. It can be understood that in other embodiments, the bone part is located at a preset position in the sample medical image extracted from the original sample medical image. For example, the bone part is located at the center position of the sample medical image. It can be understood that in other embodiments, the direction of the bone part in the extracted sample medical image is corrected and the bone part is located at the preset position of the sample medical image.

[0124] Step S92: Use the image processing model to detect the sample medical image to obtain the prediction probabilities of the bone part belonging to each preset category.

[0125] In this embodiment, an image processing model is used to detect a sample medical image, so as to obtain the prediction probabilities of the bone part belonging to each preset category. That is to say, latent category labels are set in the image processing model, and each latent category label corresponds to a preset category. The image processing model will detect the bone part in the sample medical image to determine the possibility that the bone part in the sample medical image may be each preset category. Among them, the number of preset categories is not limited and can be specifically set according to actual usage needs. For example, 3 preset categories corresponding to the subclasses of the preset bone state type are set, and 3 preset categories corresponding to the subclasses of the reference bone state type are set. It should be noted that since detecting the prediction probabilities of the target bone part belonging to each preset category for the target medical image is an abstract unsupervised clustering, the names of specific preset categories cannot be determined. Exemplarily, it can be unsupervised clustered into local bone fracture, compression fracture, healthy vertebral body, Schmorl's node, metastatic tumor, etc.

[0126] In one embodiment, the image processing model includes several classifiers, and each classifier corresponds to a preset category. The classifier is used to determine the prediction probability belonging to the preset category. That is to say, when using the image processing model to detect a sample medical image, each classifier will classify the bone part in the sample medical image to determine the prediction probability that the bone part belongs to the preset category corresponding to the classifier. Since image processing is a binary classification task, and there are huge intra-class differences between the bone parts of the preset bone state (for example, local bone fracture, compression fracture, etc.) and the bone parts of the reference bone state (healthy bone part, Schmorl's node, metastatic tumor, etc.), in order to improve the accuracy of the image processing model and simplify the training of the image processing model, several classifiers are set in the image processing model to force the binary classification task model to output the probabilities corresponding to each preset category, that is, the image processing model itself clusters the bone parts in each sample medical image.

[0127] In order to standardize and unify the sample medical image, so as to facilitate the detection of the sample medical image and improve the detection efficiency. Therefore, in one embodiment, before using the image processing model to detect the sample medical image and obtain the prediction probabilities of the bone part belonging to each preset category, preprocessing is performed on the sample medical image. Among them, the preprocessing includes at least one of image repositioning, image resampling, and pixel normalization. Pixel normalization can be to adjust the pixel values to a preset range based on the selected window width and window level. For example, when the selected window width and window level are [1500, 450], the pixel values of the image are normalized to between [-1, 1]; image repositioning can be to adjust the direction of the image to be consistent with the identity matrix, and image repositioning can be achieved through operations such as rotation and interpolation.

[0128] Step S93: Obtain the sample detection result of the sample medical image based on the prediction probabilities of the bone part belonging to each preset category.

[0129] In this embodiment, the image processing model obtains the sample detection result of the sample medical image based on the prediction probabilities of the bone part belonging to each preset category. That is, the image processing model can determine whether the bone part in the sample medical image has a preset bone state according to the prediction probabilities of the bone part belonging to each preset category.

[0130] In one embodiment, the sample bone state detection result of the sample medical image can be determined according to the type to which the preset category corresponding to the maximum prediction probability in the prediction probabilities belongs. It can be understood that in other embodiments, subcategories belonging to the same type can also be classified, and then the probability that the bone part has a preset bone state can be determined according to the subcategories classified into the preset bone state type, and the probability that the bone part does not have a preset bone state can be determined according to the subcategories classified into the reference bone state type. Then, the sample detection result of the sample medical image can be determined according to the probability that the bone part has a preset bone state and the probability that the bone part does not have a preset bone state.

[0131] Step S94: Adjust the network parameters of the image processing model based on the difference between the true bone state information annotated in the sample medical image and the sample detection result.

[0132] In this embodiment, the network parameters of the image processing model are adjusted based on the difference between the true bone state information annotated in the sample medical image and the sample detection result obtained by detecting the sample medical image using the image processing model. By the difference between the true bone state information annotated in the sample medical image and the sample detection result, the network parameters of the image processing model are adjusted until the image processing model converges, so that the trained image processing model can have good detection ability and certain generalization ability.

[0133] In one embodiment, the loss of the image processing model can be determined based on the difference between the true bone state information annotated in the sample medical image and the sample detection result, and the network parameters of the image processing model can be adjusted based on the loss of the image processing model through an optimization method such as gradient descent, and the above process can be repeated until the image processing model is trained to convergence using the sample medical image. Since bone state detection is a binary classification task and the image processing model itself clusters each sample medical image by setting several classifiers, in one embodiment, in order to make the preset categories corresponding to each classifier have a certain degree of distinguishability, the network parameters include the classification parameters of each classifier. Based on the difference between the true bone state information annotated in the sample medical image and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, the network parameters of the image processing model are adjusted. That is, the adjustment range of the network parameters including the classification parameters of the classifier is determined according to the difference between the true bone state information annotated in the sample medical image and the sample detection result, and within the adjustment range of the classification parameters, the classification parameters of the classifier are adjusted to maximize the difference between the classification parameters of the classifier. That is to say, since there are huge intra-class differences in both the preset bone state bone parts and the reference bone state bone parts, if the image processing model is directly trained to predict the preset bone state and the reference bone state, that is, the image processing model needs to forcefully classify different bone state situations with large intra-class differences into one category for learning, which is difficult, and the trained image processing model is prone to prediction errors, resulting in missed or false detections of bone state situations. In this solution, the image processing model predicts each preset category that is a subclass of the preset bone state and the reference bone state. Therefore, during training, learning is carried out for each preset category, and different bone state situations with large intra-class differences are classified into different categories for learning, reducing the training difficulty and also making the prediction of the trained image processing model more accurate. Subsequently, based on the relatively accurate probabilities of each preset category obtained by prediction, it is determined whether there is a preset bone state situation, and accurate identification of the preset bone state situation can be achieved, reducing missed or false detections of the preset bone state situation.

[0134] In a specific embodiment, as Figure 11 shown, Figure 11 FIG. is a schematic flowchart of an embodiment for adjusting the network parameters of the image processing model provided by the present application. Whether the difference between the classification parameters of the classifier is maximized is determined by a constrained loss function, which specifically includes the following sub-steps:

[0135] Step S941: Based on the difference between the true bone state information annotated in the sample medical image and the sample detection result, a detection loss is obtained.

[0136] In this embodiment, a detection loss is obtained based on the difference between the true bone state information and the sample detection result of the sample medical image annotation, so as to facilitate the subsequent determination of how the image processing model adjusts the network parameters.

[0137] Step S942: Determine the target adjustment range of each network parameter based on the detection loss, and within the target adjustment range, determine the parameter values of each network parameter that minimize the constraint loss function, and adjust each network parameter according to the determined parameter values.

[0138] In this embodiment, the target adjustment range of each network parameter is determined according to the detection loss, and within the target adjustment range, the parameter values of each network parameter that minimize the constraint loss function are determined, and each network parameter is adjusted according to the determined parameter values. Among them, the constraint loss function is negatively correlated with the difference between the classification parameters of each classifier. That is to say, each network parameter is adjusted according to the corresponding target adjustment range determined by the detection loss, while the classification parameters of each classifier need to additionally determine the target adjustment range according to the detection loss, and within this target adjustment range, the parameter values of each network parameter that minimize the constraint function are determined, that is, the classification parameters of each classifier are based on both the detection loss and the constraint loss function to determine the adjustment method, so as to ensure that the potential categories learned by the image processing model are meaningful. Among them, the formula of the constraint loss function is as follows:

[0139]

[0140] where w i represents the classification parameter of the i-th classifier; w j represents the classification parameter of the j-th classifier; L lm represents the constraint loss function. Minimizing the constraint loss function means minimizing the cosine value of the angle between the classification parameters, that is, maximizing the angle between the classification parameters. Among them, the value ranges of i and j are not limited, and i, j ∈ [0, 5] is an exemplary value range of i and j.

[0141] For example, taking the image processing model with four classifiers, the types of two classifiers are preset bone state types, and their corresponding classification parameters are w 1 and w 2 , respectively. The types of the other two classifiers are reference bone state types, and their corresponding classification parameters are w 3 and w 4 as an example. Assume that according to the detection loss, the target adjustment ranges of the network parameters w 1 , w 2 , w 3 and w 4 are: the value of w 1 can be w 1a and w1b , w 2 The value of w can be w 2a , w 2b , w 2c and w 2d , w 3 The value of w can be w 3a , w 3b , w 3c and w 3d , w 4 The value of w can be w 4a , w 4b , w 4c , w 4d and w 4e , traverse all the values of each classification parameter, and there are a total of 240 combination methods; calculate the constraint loss function values corresponding to each combination method respectively. Taking the combination of w 1a , w 2a , w 3a , w 4a as an example to calculate the constraint loss function. First, substitute the classification parameters in pairs into the above formula to obtain that the constraint loss function values corresponding to w 1a and w 2a are X1, the constraint loss function values corresponding to w 1a and w 2a are X2, the constraint loss function values corresponding to w 1a and w 3a are X3, the constraint loss function values corresponding to w 1a and w 4a are X4, the constraint loss function values corresponding to w 2a and w 3a are X5, the constraint loss function values corresponding to w 2a and w 4a are X6, the constraint loss function values corresponding to w 3a and w 4a are X7. Then, take the minimum value among X1, X2, X3, X4, X5, X6, and X7 as the constraint loss function value corresponding to this combination; among the 240 combination methods, take the combination method corresponding to the minimum constraint loss function value, and adjust the corresponding network parameters in the image processing model according to the parameter values of each classification parameter in this combination method. Take the combination method corresponding to the minimum constraint loss function value. Then, in this combination method, the included angle between each classification parameter is maximized, so that the discrimination between each classifier after adjusting the network parameters is increased, that is, the difference between each classifier is increased.

[0142] Please refer to Figure 12 , Figure 12It is a schematic flowchart of an embodiment of the method for training an image processing model provided by this application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 12 the process sequence shown. As Figure 12 shown, this embodiment includes:

[0143] Step S1201: Obtain a sample medical image containing a bone part.

[0144] Step S1201 is similar to step S91 and will not be elaborated here.

[0145] Step S1202: Use the image processing model to detect the sample medical image, and obtain the prediction probabilities of the bone part belonging to each preset category.

[0146] Step S1202 is similar to step S92 and will not be elaborated here.

[0147] Step S1203: Based on the prediction probabilities of the bone part belonging to each preset category, obtain the sample detection result of the sample medical image.

[0148] Step S1203 is similar to step S93 and will not be elaborated here.

[0149] Step S1204: Based on the difference between the true bone state information annotated in the sample medical image and the sample detection result, adjust the network parameters of the image processing model.

[0150] Step S1204 is similar to step S94 and will not be elaborated here.

[0151] Since there will be huge intra-class differences between the preset bone state bone parts and the reference bone state bone parts, if the image processing model is directly trained to predict the preset bone state and the reference bone state, that is, the image processing model needs to forcibly classify different bone state situations with large intra-class differences into one category for learning, which is difficult, and the trained image processing model is prone to prediction errors, resulting in missed detection or misdetection of bone state situations. In this solution, the image processing model predicts each preset category that is a subclass of the preset bone state and the reference bone state. Therefore, during training, learning is carried out for each preset category, and different bone state situations with large intra-class differences are classified into different categories for learning, reducing the training difficulty and making the prediction of the trained image processing model more accurate. Subsequently, by predicting the relatively accurate probabilities of each preset category, it is determined whether there is a preset bone state situation, which can achieve accurate identification of the preset bone state situation and reduce missed detection or misdetection of the preset bone state situation.

[0152] In one embodiment, the loss of the preset bone state model can be determined based on the difference between the true bone state information of the sample medical image annotation and the sample detection result, and the network parameters of the preset bone state model can be adjusted based on the loss of the preset bone state model by means of optimization methods such as gradient descent, and the above process can be repeated until the preset bone state model is trained to convergence using the sample medical image. Since bone state detection is a binary classification task, by setting several classifiers, the image processing model itself clusters each sample medical image. Therefore, in order to make the preset categories corresponding to each classifier have a certain distinguishability, in one embodiment, the network parameters include the classification parameters of each classifier, and based on the difference between the true bone state information of the sample medical image annotation and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, the network parameters of the image processing model are adjusted.

[0153] Please refer to Figure 13 , Figure 13 which is a schematic structural diagram of an embodiment of the image processing device provided in the present application. The image processing device 130 includes an acquisition module 1301, a detection module 1302, and a determination module 1303. The acquisition module 1301 is configured to acquire a target medical image including a target bone part; the detection module 1302 is configured to detect the target medical image to obtain the prediction probabilities that the target bone part belongs to each preset category, where at least one preset category is a subclass of the preset bone state type, and at least one preset category is a subclass of the reference bone state type; the determination module 1303 is configured to obtain the detection result of the target medical image based on the prediction probabilities that the target bone part belongs to each preset category, where the detection result of the target medical image is used to indicate whether the target bone part in the target medical image has a preset bone state.

[0154] Among them, the image processing device 130 further includes a training module 1304, and the training module 1304 is specifically configured to: acquire a sample medical image including a bone part; where the sample medical image is annotated with the true bone state information of the bone part, and the true bone state information indicates whether the bone part has a preset bone state; use the image processing model to detect the sample medical image to obtain the prediction probabilities that the bone part belongs to each preset category; obtain the sample detection result of the sample medical image based on the prediction probabilities that the bone part belongs to each preset category; and adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result.

[0155] Among them, the above image processing model includes a number of classifiers, each classifier corresponding to a preset category and used to determine the prediction probability belonging to the preset category. The network parameters include the classification parameters of each classifier; the training module 1304 is used to adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result, specifically including: adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result and in the adjustment direction of making the difference between the classification parameters of each classifier larger.

[0156] Among them, the training module 1304 is used to adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result and in the adjustment direction of making the difference between the classification parameters of each classifier larger, specifically including: obtaining a detection loss based on the difference between the true bone state information and the sample detection result; determining the target adjustment range of each network parameter based on the detection loss, and within the target adjustment range, determining the parameter values of each network parameter that minimize the constraint loss function, and adjusting each network parameter according to the determined parameter values, where the constraint loss function is negatively correlated with the difference between the classification parameters of each classifier.

[0157] Among them, the acquisition module 1301 is used to acquire a target medical image including a target bone part, specifically including: acquiring an original medical image; determining the position information of the target bone part in the original medical image; and extracting a target medical image including the target bone part from the original medical image based on the position information.

[0158] Among them, the direction of the target bone part in the extracted target medical image is corrected, and / or the target bone part is located at a preset position in the target medical image.

[0159] Among them, the perpendicular bisector of the corrected target bone part is parallel to the preset coordinate axis of the original medical image; the preset position is the central position.

[0160] Among them, the acquisition module 1301 is used to determine the position information of the target bone part in the original medical image, specifically including: performing instance segmentation on the original medical image to obtain a first segmentation result of the target bone in the original medical image and a second segmentation result of the reference bone, where the target bone part is located on the target bone; determining the position information of the target bone part from the target bone in the original medical image based on the first segmentation result and the second segmentation result; and / or the acquisition module 1301 is further used to, before determining the position information of the target bone part in the original medical image, specifically including: detecting whether there are abnormalities in the image parameters of the original medical image, where the image parameters include at least one of image modality, image slice interval, and image size; and in response to the image parameters being normal, performing the determination of the position information of the target bone part in the original medical image and its subsequent steps.

[0161] Among them, the obtaining module 1301 is further configured to, before detecting the target medical image to obtain the prediction probabilities of the target bone part belonging to each preset category, specifically include: preprocessing the target medical image, where the preprocessing includes at least one of image relocation, image resampling, and pixel normalization; and / or, detecting whether the target bone part in the target medical image includes an implant; in the case where the target bone part does not include an implant, detecting the target medical image to obtain the prediction probabilities of the target bone part belonging to each preset category and subsequent steps.

[0162] Among them, the determining module 1303 is configured to obtain the detection result of the target medical image based on the prediction probabilities of the target bone part belonging to each preset category, specifically including: selecting a target preset category from each preset category based on the prediction probabilities of the target bone part belonging to each preset category; where the target preset category belongs to a subclass of the preset bone state type and the prediction probability of the target preset category is the largest among the preset categories belonging to the subclass of the preset bone state type; in response to the prediction probability of the target preset category being greater than or equal to the preset probability value, determining that the target bone part has the preset bone state.

[0163] Among them, the above-mentioned target bone part is a vertebral body; and / or, the above-mentioned target medical image is a three-dimensional medical image.

[0164] Please refer to Figure 14 , Figure 14 which is a schematic structural diagram of an embodiment of a training device for an image processing model provided by the present application. The training device 140 of the image processing model includes an obtaining module 1401, a detecting module 1402, a determining module 1403, and an adjusting module 1404. The obtaining module 1401 is configured to obtain a sample medical image including a bone part; where the sample medical image is labeled with the true bone state information of the bone part, and the true bone state information indicates whether the bone part has the preset bone state; the detecting module 1402 is configured to use the image processing model to detect the sample medical image to obtain the prediction probabilities of the bone part belonging to each preset category, where at least one preset category is a subclass of the preset bone state type and at least one preset category is a subclass of the reference bone state type; the determining module 1403 is configured to obtain the sample detection result of the sample medical image based on the prediction probabilities of the bone part belonging to each preset category, where the sample detection result of the sample medical image is used to indicate whether the bone part in the sample medical image has the preset bone state; the adjusting module 1404 is configured to adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result.

[0165] Among them, the above image processing model includes a number of classifiers, each classifier corresponding to a preset category and used to determine the prediction probability belonging to the preset category. The network parameters include the classification parameters of each classifier. The determination module 1403 is used to adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result, specifically including: based on the difference between the true bone state information and the sample detection result, and in the adjustment direction of making the difference between the classification parameters of each classifier larger, adjusting the network parameters of the image processing model.

[0166] Please refer to Figure 15 , Figure 15 which is a schematic structural diagram of an embodiment of an electronic device provided by the present application. The electronic device 150 includes a memory 1501 and a processor 1502 that are coupled to each other. The processor 1502 is used to execute program instructions stored in the memory 1501 to implement the steps of any of the above image processing methods or the training method embodiments of the image processing model. In a specific implementation scenario, the electronic device 150 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 150 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited here.

[0167] Specifically, the processor 1502 is used to control itself and the memory 1501 to implement the steps of any of the above image processing methods or the training method embodiments of the image processing model. The processor 1502 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 1502 may be an integrated circuit chip with signal processing capabilities. The processor 1502 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 1502 may be implemented jointly by integrated circuit chips.

[0168] Please refer to Figure 16 , Figure 16It is a structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 160 of the embodiment of the present application stores a program instruction 1601, and when the program instruction 1601 is executed, it implements the method provided by any embodiment of the image processing method or the training method of the image processing model of the present application and any non-conflicting combination. Among them, the program instruction 1601 can form a program file and be stored in the above-mentioned computer-readable storage medium 160 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the methods of each implementation method of the present application. The aforementioned computer-readable storage medium 160 includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0169] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0170] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that, it includes: obtaining a target medical image containing a target bone part; using an image processing model to detect the target medical image to obtain the prediction probabilities of the target bone part belonging to each preset category, wherein at least one of the preset categories is a subclass of a preset bone state type, and at least one of the preset categories is a subclass of a reference bone state type, the image processing model includes a number of classifiers, each classifier corresponds to one of the preset categories and is used to determine the prediction probability belonging to the preset category, the network parameters of the image processing model include the classification parameters of each classifier, and during the training stage of the image processing model, according to the adjustment direction of making the differences between the classification parameters of each classifier larger, adjust the network parameters of the image processing model; based on the prediction probabilities of the target bone part belonging to each preset category, obtain the detection result of the target medical image, wherein the detection result of the target medical image is used to indicate whether there is a preset bone state in the target bone part of the target medical image.

2. The method according to claim 1, characterized in that, the method further includes: obtaining a sample medical image containing a bone part; wherein the sample medical image is labeled with the true bone state information of the bone part, and the true bone state information indicates whether there is a preset bone state in the bone part; using an image processing model to detect the sample medical image to obtain the prediction probabilities of the bone part belonging to each of the preset categories; based on the prediction probabilities of the bone part belonging to each of the preset categories, obtain the sample detection result of the sample medical image; based on the difference between the true bone state information and the sample detection result, adjust the network parameters of the image processing model.

3. The method according to claim 2, characterized in that, the adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result includes: based on the difference between the true bone state information and the sample detection result, and according to the adjustment direction of making the differences between the classification parameters of each classifier larger, adjust the network parameters of the image processing model.

4. The method according to claim 3, characterized in that, the adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result, and according to the adjustment direction of making the differences between the classification parameters of each classifier larger, includes: based on the difference between the true bone state information and the sample detection result, obtain a detection loss; based on the detection loss, determine the target adjustment range of each network parameter, and within the target adjustment range, determine the parameter values of each network parameter that minimize a constraint loss function, and adjust each network parameter according to the determined parameter values, wherein the constraint loss function is negatively correlated with the differences between the classification parameters of each classifier.

5. The method according to any one of claims 1 to 4, characterized in that, The obtaining of the target medical image including the target bone site includes: Obtaining an original medical image; Determining the position information of the target bone site in the original medical image; Based on the position information, extracting the target medical image including the target bone site from the original medical image.

6. The method according to claim 5, wherein, the direction of the target bone site in the extracted target medical image is corrected, and / or, the target bone site is located at a preset position in the target medical image.

7. The method according to claim 6, wherein, the perpendicular bisector of the corrected target bone site is parallel to the preset coordinate axis of the original medical image; the preset position is the central position.

8. The method according to claim 5, wherein, the determining the position information of the target bone site in the original medical image includes: Performing instance segmentation on the original medical image to obtain a first segmentation result of the target bone and a second segmentation result of the reference bone in the original medical image, wherein the target bone site is located on the target bone; Based on the first segmentation result and the second segmentation result, determining the position information of the target bone site from the target bone in the original medical image; and / or, before the determining the position information of the target bone site in the original medical image, the method further includes: Detecting whether there are abnormalities in the image parameters of the original medical image, wherein the image parameters include at least one of image modality, image slice interval, and image size; In response to the image parameters being normal, performing the determining the position information of the target bone site in the original medical image and its subsequent steps.

9. The method according to any one of claims 1 to 4, wherein, before the detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category, the method further includes: Performing preprocessing on the target medical image, wherein the preprocessing includes at least one of image repositioning, image resampling, and pixel normalization; and / or, Detecting whether the target bone site in the target medical image includes an implant; In the case where the target bone site does not include the implant, detecting the target medical image to obtain the prediction probabilities of the target bone site belonging to each preset category and its subsequent steps.

10. The method according to any one of claims 1 to 4, wherein, the obtaining the detection result of the target medical image based on the prediction probabilities of the target bone site belonging to each preset category includes: Based on the prediction probabilities of the target bone site belonging to each of the preset categories, selecting a target preset category from each of the preset categories; wherein the target preset category belongs to a subclass of the preset bone state type and the prediction probability of the target preset category is the largest among the preset categories belonging to the subclass of the preset bone state type. In response to the prediction probability of the target preset category being greater than or equal to a preset probability value, it is determined that the target bone site has a preset bone state.

11. The method according to claim 1, wherein, the target bone site is a vertebral body; and / or, the target medical image is a three-dimensional medical image.

12. A method for training an image processing model, wherein, it includes: Obtain a sample medical image including a bone site; wherein, the sample medical image is labeled with the true bone state information of the bone site, and the true bone state information indicates whether the bone site has a preset bone state; Use an image processing model to detect the sample medical image to obtain the prediction probability of the bone site belonging to each preset category, wherein at least one of the preset categories is a subclass of the preset bone state type, and at least one of the preset categories is a subclass of the reference bone state type. The image processing model includes several classifiers, each classifier corresponds to one of the preset categories and is used to determine the prediction probability belonging to the preset category. The network parameters of the image processing model include the classification parameters of each classifier. During the training stage of the image processing model, adjust the network parameters of the image processing model in the adjustment direction that makes the differences between the classification parameters of each classifier larger; Based on the prediction probabilities of the bone site belonging to each of the preset categories, obtain the sample detection result of the sample medical image, wherein the sample detection result of the sample medical image is used to indicate whether the bone site in the sample medical image has a preset bone state; Based on the difference between the true bone state information and the sample detection result, adjust the network parameters of the image processing model.

13. The method according to claim 12, wherein, the adjusting the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result includes: Based on the difference between the true bone state information and the sample detection result, and in the adjustment direction that makes the differences between the classification parameters of each classifier larger, adjust the network parameters of the image processing model.

14. An image processing device, wherein, it includes: An acquisition module for acquiring a target medical image including a target bone site; A detection module for using an image processing model to detect the target medical image to obtain the prediction probability of the target bone site belonging to each preset category, wherein at least one of the preset categories is a subclass of the preset bone state type, and at least one of the preset categories is a subclass of the reference bone state type. The image processing model includes several classifiers, each classifier corresponds to one of the preset categories and is used to determine the prediction probability belonging to the preset category. The network parameters of the image processing model include the classification parameters of each classifier. During the training stage of the image processing model, adjust the network parameters of the image processing model in the adjustment direction that makes the differences between the classification parameters of each classifier larger; A determination module, configured to obtain a detection result of the target medical image based on the prediction probabilities of the target bone part belonging to each preset category, where the detection result of the target medical image is used to indicate whether a preset bone state exists in the target bone part in the target medical image.

15. A training device for an image processing model, characterized in that it includes: An acquisition module, configured to acquire a sample medical image including a bone part; wherein, the sample medical image is labeled with true bone state information of the bone part, and the true bone state information indicates whether a preset bone state exists in the bone part; A detection module, configured to use the image processing model to detect the sample medical image to obtain prediction probabilities of the bone part belonging to each preset category, where at least one of the preset categories is a subclass of a preset bone state type, and at least one of the preset categories is a subclass of a reference bone state type. The image processing model includes a plurality of classifiers, each classifier corresponding to one of the preset categories and being configured to determine the prediction probability belonging to the preset category. The network parameters of the image processing model include the classification parameters of each classifier. During the training stage of the image processing model, the network parameters of the image processing model are adjusted in an adjustment direction that makes the differences between the classification parameters of each classifier larger; A determination module, configured to obtain a sample detection result of the sample medical image based on the prediction probabilities of the bone part belonging to each of the preset categories, where the sample detection result of the sample medical image is used to indicate whether a preset bone state exists in the bone part in the sample medical image; An adjustment module, configured to adjust the network parameters of the image processing model based on the difference between the true bone state information and the sample detection result.

16. An electronic device, characterized in that the electronic device includes a memory and a processor, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the image processing method according to any one of claims 1-11, or to implement the training method of the image processing model according to any one of claims 12-13.

17. A computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the image processing method according to any one of claims 1-11, or to implement the training method of the image processing model according to any one of claims 12-13.

Images