Fracture detection method and apparatus, electronic device, and storage medium

Abstract

The application provides a fracture detection method and device, electronic equipment and a storage medium. The method comprises: acquiring a chest X-ray image, and acquiring a corresponding dual-energy skeleton image based on the chest X-ray image; inputting the dual-energy skeleton image into a preset anomaly detection model to obtain position information of an abnormal area; based on the position information of the abnormal area, acquiring position information of a contralateral area of the abnormal area in a spine; and when detecting that the similarity between the abnormal area and the contralateral area in the dual-energy skeleton image is less than or equal to a preset threshold, determining that the dual-energy skeleton image has a fracture. The application improves the diagnosis efficiency, excludes the case of false fracture in which both sides of the image are abnormal, and reduces the probability of misdiagnosis.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to fracture detection methods, devices, electronic devices, and storage media. Background Technology

[0002] A fracture is a complete or partial break in the continuity of bone structure, most commonly seen in children and the elderly, but also frequently occurring in young and middle-aged adults. Fractures are a prevalent, sudden, and often complicated condition, frequently requiring emergency and nighttime visits, placing high demands on diagnostic speed and accuracy. Delayed diagnosis, missed diagnoses, or misdiagnoses can prevent timely treatment of the fracture site, potentially leading to difficult prognoses and other clinical problems, and also posing a risk of medical disputes.

[0003] Currently, the diagnosis of fractures is usually accomplished using imaging techniques, with X-ray and CT being commonly used imaging methods. Compared to CT, X-ray has advantages such as ease of operation, rapid image output, and low cost, making it the best choice for diagnosing fractures in the chest and limbs, and it is widely used.

[0004] However, X-ray images often have drawbacks such as tissue overlap, slight blurring, and uneven grayscale distribution. Furthermore, fractures are often hidden, small, obscured, and difficult to detect. In the process of differentiating fractures using X-ray images, radiologists need to repeatedly observe each bone segment with the naked eye. If the image is blurry, the window width and level need to be repeatedly adjusted, which is time-consuming and tedious. The diagnostic results are also highly subjective and prone to misdiagnosis.

[0005] Therefore, there is an urgent need to provide fracture detection methods, devices, electronic equipment, and storage media to improve existing technologies. Summary of the Invention

[0006] The purpose of this application is to provide fracture detection methods, devices, electronic equipment, and storage media that can improve diagnostic efficiency, eliminate false fractures where both sides of the image are abnormal, and reduce the probability of misdiagnosis.

[0007] In a first aspect, this application provides a method for detecting fractures, the method comprising:

[0008] Acquire a chest X-ray image, and obtain a corresponding dual-energy skeleton image based on the chest X-ray image;

[0009] The dual-energy skeleton image is input into a preset anomaly detection model to obtain the location information of the abnormal region;

[0010] Based on the location information of the abnormal region, the location information of the abnormal region on the opposite side of the spine is obtained;

[0011] When the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is less than or equal to a preset threshold, it is determined that there is a fracture in the dual-energy skeleton image.

[0012] In some optional embodiments, obtaining the corresponding dual-energy skeletal image based on the chest X-ray image includes:

[0013] The chest X-ray image is input into a preset image mapping model to obtain the corresponding dual-energy skeletal image;

[0014] The image mapping model is obtained by training a pre-defined convolutional neural network.

[0015] In some optional embodiments, obtaining the location information of the abnormal region on the contralateral side of the spine based on the location information of the abnormal region includes:

[0016] The dual-energy skeleton image is input into a preset skeleton segmentation model to obtain rib segmentation results;

[0017] Based on the location information of the abnormal region and the rib segmentation result, the abnormal rib and its contralateral rib are determined from all the ribs in the dual-energy skeleton image.

[0018] Based on the relative positional relationship between the abnormal region and the abnormal rib, the relative positional relationship between the region opposite the abnormal region and the rib opposite the abnormal rib is obtained, thereby obtaining the positional information of the region opposite the abnormal region.

[0019] In some optional embodiments, the location information of the abnormal region is used to indicate the position of the center point of the detection frame of the abnormal region, and the process of obtaining the relative positional relationship between the abnormal region and the abnormal rib includes:

[0020] The vertex that is shorter than the center point of the detection box of the abnormal region at both ends of the abnormal rib is taken as the first target vertex;

[0021] A first vector is constructed with the first target vertex as the starting point and the center point of the detection box of the abnormal region as the ending point. The first vector is used to indicate the relative positional relationship between the abnormal region and the abnormal rib.

[0022] In some optional embodiments, obtaining the relative positional relationship between the region opposite the abnormal area and the rib opposite the abnormal rib based on the relative positional relationship between the abnormal area and the abnormal rib includes:

[0023] Using the vertical axis of the image coordinate system where the dual-energy skeleton image is located as a reference, an auxiliary line is constructed through the first target vertex. The auxiliary line is used as the axis of symmetry to mirror the first vector to obtain the intermediate vector.

[0024] The vertices at both ends of the opposite rib of the abnormal rib that are symmetrical with respect to the first target vertex about the spine are taken as the second target vertex.

[0025] The intermediate vector is translated to obtain a second vector starting from the second target vertex. The second vector is used to indicate the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib.

[0026] In some optional embodiments, the method further includes: when the similarity between an abnormal region in the dual-energy skeletal image and its contralateral region is detected to be greater than the preset threshold, determining that there is no fracture in the dual-energy skeletal image.

[0027] In some optional embodiments, the training process of the anomaly detection model is as follows:

[0028] The sub-images corresponding to the detection boxes of the abnormal region with fracture and its opposite region are used as negative sample pairs, and the sub-images corresponding to the detection boxes of the abnormal region without fracture and its opposite region are used as positive sample pairs.

[0029] The anomaly detection model is obtained by training a pre-defined convolutional neural network using the positive and negative sample pairs.

[0030] Secondly, this application provides a fracture detection device, the device comprising:

[0031] An image mapping module is used to acquire chest X-ray images and acquire corresponding dual-energy skeleton images based on the chest X-ray images;

[0032] An anomaly detection module is used to input the dual-energy skeleton image into a preset anomaly detection model to obtain the location information of the abnormal region;

[0033] The contralateral acquisition module is used to acquire the location information of the abnormal region in the contralateral region of the spine based on the location information of the abnormal region.

[0034] The fracture determination module is used to determine that a fracture exists in the dual-energy skeleton image when the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is less than or equal to a preset threshold.

[0035] In some optional embodiments, the image mapping module is used for:

[0036] The chest X-ray image is input into a preset image mapping model to obtain the corresponding dual-energy skeletal image;

[0037] The image mapping model is obtained by training a pre-defined convolutional neural network.

[0038] In some optional embodiments, the contralateral acquisition module includes:

[0039] The rib segmentation unit is used to input the dual-energy skeleton image into a preset skeleton segmentation model to obtain the rib segmentation result;

[0040] A rib localization unit is used to determine the abnormal rib and its contralateral rib from all ribs in the dual-energy skeletal image based on the location information of the abnormal region and the rib segmentation result.

[0041] The contralateral positioning unit is used to obtain the relative positional relationship between the contralateral region of the abnormal region and the contralateral rib of the abnormal rib based on the relative positional relationship between the abnormal region and the abnormal rib, thereby obtaining the positional information of the contralateral region of the abnormal region.

[0042] In some optional embodiments, the location information of the abnormal region is used to indicate the position of the center point of the detection frame of the abnormal region, and the process of obtaining the relative positional relationship between the abnormal region and the abnormal rib includes:

[0043] The vertex that is shorter than the center point of the detection box of the abnormal region at both ends of the abnormal rib is taken as the first target vertex;

[0044] A first vector is constructed with the first target vertex as the starting point and the center point of the detection box of the abnormal region as the ending point. The first vector is used to indicate the relative positional relationship between the abnormal region and the abnormal rib.

[0045] In some optional embodiments, the contralateral positioning unit is used for:

[0046] Using the vertical axis of the image coordinate system where the dual-energy skeleton image is located as a reference, an auxiliary line is constructed through the first target vertex. The auxiliary line is used as the axis of symmetry to mirror the first vector to obtain the intermediate vector.

[0047] The vertices at both ends of the opposite rib of the abnormal rib that are symmetrical with respect to the first target vertex about the spine are taken as the second target vertex.

[0048] The intermediate vector is translated to obtain a second vector starting from the second target vertex. The second vector is used to indicate the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib.

[0049] In some alternative embodiments, the apparatus further includes:

[0050] The false positive detection module is used to determine that there is no fracture in the dual-energy skeletal image when the similarity between an abnormal region in the dual-energy skeletal image and its opposite region is greater than the preset threshold.

[0051] In some optional embodiments, the process of obtaining the similarity between the abnormal region in the dual-energy skeleton image and its contralateral region includes: inputting the dual-energy skeleton image into a similarity detection model to output the similarity between the abnormal region in the dual-energy skeleton image and its contralateral region.

[0052] The training process of the similarity detection model is as follows:

[0053] The sample sub-images corresponding to the detection boxes of the abnormal region with fracture and its opposite region are used as negative sample pairs, and the sample sub-images corresponding to the detection boxes of the abnormal region without fracture and its opposite region are used as positive sample pairs.

[0054] The similarity detection model is obtained by training a pre-defined convolutional neural network using the positive and negative sample pairs.

[0055] Thirdly, this application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0056] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0057] The aforementioned fracture detection method, device, electronic equipment, and storage medium, during the fracture detection process, map chest X-ray images to corresponding dual-energy skeletal imaging (DEXA) images, improving rib clarity and thus enhancing diagnostic efficiency. The DEXA images are then input into a preset anomaly detection model to obtain the location information of the abnormal area, thereby locating the abnormal area on the opposite side of the spine. The differences between the two sides (the abnormal area and its opposite side) are compared to determine if they are significant. Generally, fractures do not occur symmetrically on both sides of the ribs. Only if the difference between the two sides is significant (similarity less than or equal to a preset threshold) is a fracture confirmed in the DEXA image; otherwise, it may be a false positive. This method can rule out false fractures where both sides of the ribs appear abnormal in the image, reducing the probability of misdiagnosis. Attached Figure Description

[0058] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0059] Figure 1 This is a schematic flowchart of a fracture detection method provided in an embodiment of this application.

[0060] Figure 2 This is a schematic diagram illustrating the principle of the training process of an image mapping model provided in an embodiment of this application.

[0061] Figure 3 This is a schematic diagram of a process for locating the opposite side region provided in an embodiment of this application.

[0062] Figure 4 This is a schematic diagram of another process for locating the opposite side region provided in an embodiment of this application.

[0063] Figure 5 This is a schematic diagram illustrating the principle of locating the opposite side region provided in an embodiment of this application.

[0064] Figure 6 This is a schematic diagram of another method for locating the opposite side region provided in the embodiments of this application.

[0065] Figure 7 This is a schematic diagram illustrating another principle for locating the opposite side region provided in an embodiment of this application.

[0066] Figure 8 This is a schematic diagram of a comparative learning process provided in an embodiment of this application.

[0067] Figure 9 This is a schematic diagram illustrating the principle of comparative learning provided in an embodiment of this application.

[0068] Figure 10 This is a structural block diagram of a fracture detection device provided in an embodiment of this application.

[0069] Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0070] Figure 12 This is a schematic diagram of the structure of a program product provided in an embodiment of this application. Detailed Implementation

[0071] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] It should also be noted that, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any implementation or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other implementations or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0074] Fractures are one of the most easily misdiagnosed cases in the emergency department, with the peak period for misdiagnosis occurring between 8 PM and 2 AM. Diagnosis of fractures typically relies on imaging techniques, with X-rays and CT scans being commonly used.

[0075] Compared to CT, X-ray has advantages such as simple operation, rapid image output, and low cost, making it the best choice for diagnosing fractures in areas such as the chest and limbs, and it is widely used. However, X-ray is more difficult than CT in detecting fracture location. This is because X-ray images (i.e., X-ray films / images) often have disadvantages such as tissue overlap, slight blurring, and uneven grayscale distribution, and fractures are often hidden, small, obscured, and difficult to detect.

[0076] Current X-ray fracture image identification mainly relies on manual interpretation. Radiologists review the patient's X-ray images to locate and differentiate the fracture. This requires doctors to repeatedly observe each bone segment, and if the image is blurry, the window width and level need to be adjusted repeatedly. This process is time-consuming and tedious, and the diagnostic results are highly subjective, easily leading to misdiagnosis or missed diagnosis.

[0077] Furthermore, current research on X-ray mainly focuses on the limbs (arms and thighs), and there has been no research on the ribs of the chest. The ribs of the chest are characterized by their large number, curved shape, great variability, and severe tissue overlap. There is great potential for applying X-ray imaging to the diagnosis of fractures of the ribs of the chest.

[0078] The following section will introduce fracture detection methods, devices, electronic equipment, and storage media, which can improve diagnostic efficiency, eliminate false fractures where both sides of the image are abnormal, and reduce the probability of misdiagnosis.

[0079] See Figure 1 , Figure 1 This is a schematic flowchart of a fracture detection method provided in an embodiment of this application.

[0080] The method includes steps S101 to S104.

[0081] Step S101: Obtain a chest X-ray image and obtain a corresponding dual-energy skeleton image based on the chest X-ray image.

[0082] Chest X-ray images contain overlapping projections of various tissues, resulting in unclear rib boundaries and a low fracture detection rate. Therefore, chest X-ray images can be mapped to corresponding dual-energy skeletal images to improve rib clarity, thereby increasing the fracture detection rate and detection efficiency.

[0083] In some implementations, step S101 may include:

[0084] The chest X-ray image is input into a preset image mapping model to obtain the corresponding dual-energy skeletal image;

[0085] The image mapping model is obtained by training a pre-defined convolutional neural network.

[0086] Dual-energy skeletal images are skeletal images obtained using dual-energy subtraction imaging technology.

[0087] In some implementations, chest X-ray images can be DR images, which are technologies that directly convert X-ray image information into digital images.

[0088] Compared to traditional X-ray films, DR technology offers higher image resolution and shorter examination time, typically completing the examination of a relevant area in just a few seconds. Furthermore, the radiation dose to the human body is relatively low, with a single DR examination of a single area resulting in a radiation dose of only a few millisieverts, causing no significant harm to the human body after the examination.

[0089] Dual-energy subtraction angiography (DSA) primarily employs a two-exposure method, exposing the subject to two consecutive exposures with high and low energy outputs within 200ms to obtain two images or data. These images are then subtracted or integrated to reconstruct three images: a skeletal image, a soft tissue image, and a standard chest X-ray image. The skeletal image reduces the obstruction of bone by soft tissue, allowing for the identification of benign calcified nodules and minor fractures.

[0090] See Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of the training process of an image mapping model provided in an embodiment of this application.

[0091] In some implementations, the training process of the image mapping model is as follows:

[0092] The sample X-ray image is input into a pre-defined convolutional neural network to output the corresponding predicted skeleton image;

[0093] Based on the predicted skeletal image and the dual-energy skeletal image corresponding to the sample X-ray image, the adversarial loss of the convolutional neural network is calculated.

[0094] Based on the adversarial loss, the parameters of the convolutional neural network are updated to obtain the image mapping model.

[0095] By continuously iterating and updating the convolutional neural network, the predicted skeletal image is made as close as possible to the real dual-energy skeletal image.

[0096] In some implementations, any type of convolutional neural network can be used, including but not limited to: LinkNet, V-Net, DenseNet, and ResNet.

[0097] In some implementations, the process of calculating the adversarial loss of a convolutional neural network includes:

[0098] The predicted skeletal image and the dual-energy skeletal image corresponding to the sample X-ray image are respectively input into the loss calculation module to output the adversarial loss of the convolutional neural network.

[0099] The loss calculation module consists of four sequentially connected feature extraction sub-modules. Each feature extraction sub-module includes at least one convolutional unit (Conv) and at least one activation unit, wherein the activation unit may employ Leaky ReLU (leaking activation function).

[0100] Step S102: Input the dual-energy skeleton image into a preset anomaly detection model to obtain the location information of the abnormal region.

[0101] It should be noted that the abnormal area in this application embodiment refers to the area suspected of being a fracture. Whether the area is a real fracture area requires further testing.

[0102] In some implementations, the location information of the abnormal region is used to indicate the location of the center point of the detection box, which can be represented by coordinates.

[0103] The anomaly detection model can be trained by a pre-defined convolutional neural network, which can be any of the following detection networks, including but not limited to: Faster-RCNN, RetinaNet, FCOS, etc.

[0104] When the location information of the abnormal region output by the anomaly detection model is empty, that is, there is no suspicious fracture in the dual-energy skeleton image, the subsequent steps (S103, S104) are not executed.

[0105] Step S103: Based on the location information of the abnormal region, obtain the location information of the abnormal region on the opposite side of the spine.

[0106] It is important to note that the region opposite to the abnormal area refers to the region that is roughly symmetrical about the spine (not perfectly symmetrical, taking into account the asymmetrical distribution of ribs on both sides of the spine).

[0107] See Figure 3 , Figure 3 This is a schematic diagram of a process for locating the opposite side region provided in an embodiment of this application.

[0108] In some embodiments, step S103 may include steps S201 to S203.

[0109] Step S201: Input the dual-energy skeleton image into a preset skeleton segmentation model to obtain the rib segmentation result.

[0110] In some implementations, the rib segmentation results are used to indicate the position and labeling information of the 12 ribs.

[0111] The specific process of using the skeleton segmentation model includes:

[0112] The dual-energy skeletal image is resampled to a preset resolution using a pre-defined segmentation algorithm. The resampled image is then normalized and input into the skeletal segmentation model to automatically segment 12 ribs and label the sequence of each rib.

[0113] The training process of the skeleton segmentation model includes:

[0114] The sample dual-energy skeletal images and their corresponding labeled images are input into a pre-defined convolutional neural network for training to obtain a skeletal segmentation model. This convolutional neural network can be any of the following segmentation networks, including but not limited to: LinkNet, V-Net, DenseNet, and ResNet.

[0115] In some implementations, the convolutional neural network may employ a LinkNet network, which includes an encoder, a decoder, residual blocks, and skip connection structures.

[0116] Considering that 12 ribs are not easy to classify, during the model training process, the ribs can be divided into three major categories first. The classification information of each major category is then provided to the convolutional neural network for further subdivision of the ribs, thereby improving the segmentation ability of the convolutional neural network.

[0117] For example, in a dual-energy skeletal image, three major categories can be divided in order from top to bottom: the first to third ribs are classified as the first category, the fourth to ninth ribs as the second category, and the tenth to twelfth ribs as the third category.

[0118] Step S202: Based on the location information of the abnormal region and the rib segmentation result, determine the abnormal rib and its contralateral rib from all ribs in the dual-energy skeleton image.

[0119] The abnormal rib is the rib where the abnormal area is located. In other words, the rib on which the center point of the detection box of the abnormal area falls is the abnormal rib.

[0120] The ribs opposite to the abnormal ribs, that is, the ribs that are symmetrically distributed about the spine with respect to the abnormal ribs.

[0121] For example, if the abnormal rib is the third rib on the left side of the spine, then the opposite rib is the third rib on the right side of the spine.

[0122] It should be noted that in this embodiment of the application, when the anomaly detection model detects that multiple ribs in the dual-energy skeletal image are suspected to be fractured, it will provide multiple detection boxes for the abnormal regions. The rib at the center of each abnormal region's detection box is an abnormal rib. In subsequent fracture detection, the similarity between the abnormal region and the opposite region of each abnormal rib can be detected to determine whether the abnormal rib has actually fractured.

[0123] Step S203: Based on the relative positional relationship between the abnormal region and the abnormal rib, obtain the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib, thereby obtaining the positional information of the opposite region of the abnormal region.

[0124] In some implementations, the location information of the abnormal region is used to indicate the location of the center point of the detection box of the abnormal region.

[0125] The location and label information of the 12 ribs can be quickly obtained by using the skeletal segmentation model. Combined with the location information of the abnormal area output by the anomaly detection model, the abnormal rib and its opposite rib can be identified. Based on the relative positional relationship of the abnormal area on the abnormal rib, the relative positional relationship is mapped to the opposite rib, thereby obtaining the location information of the opposite area. This localization method relies on the rib segmentation results, is relatively simple and efficient, and has high localization accuracy.

[0126] See Figure 4 , Figure 4 This is a schematic diagram of another process for locating the opposite side region provided in an embodiment of this application.

[0127] In step S203, the process of obtaining the relative positional relationship between the abnormal area and the abnormal rib includes steps S301 to S302.

[0128] Step S301: Select the vertex at both ends of the abnormal rib that is shorter than the center point of the detection box of the abnormal area as the first target vertex.

[0129] Step S302: Construct a first vector with the first target vertex as the starting point and the center point of the detection box of the abnormal region as the ending point. The first vector is used to indicate the relative positional relationship between the abnormal region and the abnormal rib.

[0130] See Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of locating the opposite side region provided in an embodiment of this application.

[0131] exist Figure 5 In the diagram, the 11 ribs on the left side (the 12th rib is not shown) are labeled L01 to L11, and the 11 ribs on the right side (the 12th rib is not shown) are labeled R01 to R11.

[0132] The abnormal rib and its contralateral rib are marked as R05 and L05, respectively.

[0133] The two vertices of the abnormal rib are represented by PR0 and PR1 respectively. The center point of the detection box of the abnormal region is represented by Fft. The distance between PR0 and Fft is represented by D_PR0, and the distance between PR1 and Fft is represented by D_PR1. It can be seen that D_PR1 is less than D_PR0. PR1 is taken as the first target vertex, and the first vector is the vector with PR1 as the starting point and Fft as the ending point.

[0134] In some embodiments, step S203 further includes steps S303 to S305.

[0135] Step S303: Using the vertical axis of the image coordinate system where the dual-energy skeleton image is located as a reference, construct an auxiliary line through the first target vertex, and use the auxiliary line as an axis of symmetry to mirror the first vector to obtain an intermediate vector.

[0136] exist Figure 5 In the diagram, the auxiliary line is a vertical dashed line passing through PR1. The intermediate vector and the first vector are symmetrical about the auxiliary line. The starting point of the intermediate vector is PR1, and the magnitude of the intermediate vector is equal to that of the first vector, both being D_PR1.

[0137] Step S304: Select the vertices at both ends of the opposite side of the abnormal rib that are symmetrical with respect to the first target vertex about the spine as the second target vertex.

[0138] exist Figure 5 In the diagram, the two vertices of the contralateral rib are represented by PL0 and PL1 respectively. It can be seen that PL1 is the vertex on the contralateral rib that is symmetrical to PR1. PL1 is taken as the second target vertex.

[0139] Step S305: Translate the intermediate vector to obtain a second vector starting from the second target vertex. The second vector is used to indicate the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib.

[0140] The purpose of mirroring and translating the vector is to map the relative positional relationship between the abnormal region and the abnormal rib to the opposite rib of the abnormal rib, thereby quickly locating the opposite region.

[0141] exist Figure 5 In this context, the second vector, the intermediate vector, and the first vector all have the same magnitude, which is D_PR1.

[0142] The endpoint of the second vector is the center point of the detection box in the opposite region, denoted by Fff.

[0143] The method described above for obtaining the location information of the contralateral region through steps S201 to S203 can determine the contralateral region through simple geometric calculations, making it simple and efficient. However, when locating the contralateral region of the abnormal region, the vertical direction of the image coordinate system needs to be used as a reference. If the patient's posture is slightly tilted (e.g., the body is bent to the side, and the spine is tilted relative to the imaging direction), it may cause a large error.

[0144] See Figure 6 and Figure 7 , Figure 6 This is a schematic diagram of another method for locating the opposite region provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating another principle for locating the opposite side region provided in an embodiment of this application.

[0145] Considering the errors caused by spinal curvature, this application embodiment also provides another way to obtain the location information of the contralateral region, which includes steps S201, S202 and S204.

[0146] Step S201: Input the dual-energy skeleton image into a preset skeleton segmentation model to obtain the rib segmentation result.

[0147] The rib segmentation results are used to indicate the position of the spinal midline and the position and label information of the 12 ribs.

[0148] Step S202: Based on the location information of the abnormal region and the rib segmentation result, determine the abnormal rib and its contralateral rib from all ribs in the dual-energy skeleton image.

[0149] Step S204: Based on the location information of the abnormal region and the location information of the midline of the spine, obtain the location information of the region opposite to the abnormal region.

[0150] In some implementations, the location information of the abnormal region is used to indicate the location of the center point of the detection box of the abnormal region, and step S204 may include:

[0151] Using the midpoint of the spine corresponding to the abnormal rib on the midline of the spine as the center, and using the center of the circle and the center point of the detection frame of the abnormal area as the radius, an auxiliary circle is drawn. The intersection of the auxiliary circle and the opposite rib is the center point of the detection frame of the opposite area.

[0152] This method of finding the intersection point by using auxiliary circles ensures that the distance from the abnormal area to the midpoint of the spine is equal to that from the opposite side area. The positioning accuracy depends only on the symmetry of the ribs on both sides and is independent of the orientation of the image coordinate system. To a certain extent, it can eliminate the influence of the patient's body posture during imaging.

[0153] The method described above for obtaining the location information of the contralateral region through steps S201, S202, and S204 is very simple in its calculation process. However, it requires that the ribs on both sides of the spine be strictly symmetrical. If the ribs on both sides are not symmetrical, there will be a large error.

[0154] Considering the advantages and disadvantages of the two methods mentioned above, this application also provides a third method for obtaining the location information of the opposite region. This method combines the characteristics of the two methods mentioned above, as follows:

[0155] The center point of the detection box in the opposite region calculated using steps S201 to S203 is recorded as the first positioning point, and the center point of the detection box in the opposite region calculated using steps S201, S202 and S204 is recorded as the second positioning point.

[0156] The midpoint of the line connecting the first and second positioning points is used as the center point of the detection box in the opposite region.

[0157] The third method described above comprehensively optimizes the first two methods for locating the opposite area, further reducing the positioning error.

[0158] Step S104: When the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is less than or equal to a preset threshold, it is determined that there is a fracture in the dual-energy skeleton image.

[0159] The embodiments of this application do not limit the preset threshold, which may be, for example, 80%, 85%, or 90%.

[0160] The method for obtaining similarity in step S104 can be:

[0161] The dual-energy skeleton image is input into a pre-trained similarity detection model to output the similarity between the abnormal region in the dual-energy skeleton image and its opposite region.

[0162] The aforementioned fracture detection method maps chest X-ray images to corresponding dual-energy skeletal imaging (DEXA) images during the fracture detection process, improving the clarity of the ribs in the images and thus enhancing diagnostic efficiency. The DEXA images are then input into a preset anomaly detection model to obtain the location information of the abnormal area, thereby locating the abnormal area in the region opposite to the spine. The differences between the two sides (the abnormal area and its opposite region) are compared to see if they are significant. Generally, fractures do not occur symmetrically on both sides of the ribs. Only if the difference between the two sides is large (similarity less than or equal to a preset threshold) is a fracture confirmed in the DEXA image. Otherwise, it may be a false positive for fracture. This method can rule out the situation where both sides of the ribs in the image are abnormal, thus reducing the probability of misdiagnosis.

[0163] In some optional embodiments, the method further includes: when the similarity between an abnormal region in the dual-energy skeletal image and its contralateral region is detected to be greater than the preset threshold, determining that there is no fracture in the dual-energy skeletal image.

[0164] See Figure 8 and Figure 9 , Figure 8 This is a schematic diagram of a comparative learning process provided in an embodiment of this application. Figure 9 This is a schematic diagram illustrating the principle of comparative learning provided in an embodiment of this application.

[0165] In some optional embodiments, the training process of the similarity detection model includes steps S401 to S402.

[0166] Step S401: Take the sample sub-images corresponding to the detection boxes of the abnormal region with fracture and its opposite region as negative sample pairs, and take the sample sub-images corresponding to the detection boxes of the abnormal region without fracture and its opposite region as positive sample pairs.

[0167] The sample sub-image corresponding to the detection box is the portion of the image cropped from the original image (sample dual-energy skeleton image) at the position corresponding to the detection box (e.g., a rectangle).

[0168] like Figure 9 As shown, the sample sub-images of the detection boxes for true positives (in fact, there is a fracture) are represented by True positive, and the sample sub-images of the detection boxes for false positives (in fact, there is no fracture) are represented by False positive. Positive sample pairs are represented by Positive pairs, and negative sample pairs are represented by Negative pairs.

[0169] Step S402: Train a preset convolutional neural network using the positive sample pairs and the negative sample pairs to obtain the similarity detection model.

[0170] In some implementations, step S402 may include:

[0171] The contrastive loss of the convolutional neural network is calculated using the positive sample pairs and the negative sample pairs;

[0172] Based on the contrast loss, the parameters of the convolutional neural network are updated to obtain the similarity detection model.

[0173] The aforementioned contrastive learning method compares the abnormal region with the contralateral region and uses the feedback detection results of contrastive learning to update and iterate the model, thereby improving the detection accuracy of the similarity detection model.

[0174] See Figure 10 , Figure 10 This is a structural block diagram of a fracture detection device provided in an embodiment of this application.

[0175] The specific implementation method of the fracture detection device is consistent with the implementation method and the technical effect achieved as described in the embodiments of the above fracture detection method, and some contents will not be repeated.

[0176] The device includes:

[0177] Image mapping module 101 is used to acquire chest X-ray images and acquire corresponding dual-energy skeleton images based on the chest X-ray images;

[0178] Anomaly detection module 102 is used to input the dual-energy skeleton image into a preset anomaly detection model to obtain the location information of the abnormal region;

[0179] The contralateral acquisition module 103 is used to acquire the location information of the abnormal region in the contralateral region of the spine based on the location information of the abnormal region.

[0180] The fracture determination module 104 is used to determine that there is a fracture in the dual-energy skeleton image when the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is less than or equal to a preset threshold.

[0181] In some optional embodiments, the image mapping module 101 is used for:

[0182] The chest X-ray image is input into a preset image mapping model to obtain the corresponding dual-energy skeletal image;

[0183] The image mapping model is obtained by training a pre-defined convolutional neural network.

[0184] In some optional embodiments, the contralateral acquisition module 103 includes:

[0185] The rib segmentation unit is used to input the dual-energy skeleton image into a preset skeleton segmentation model to obtain the rib segmentation result;

[0186] A rib localization unit is used to determine the abnormal rib and its contralateral rib from all ribs in the dual-energy skeletal image based on the location information of the abnormal region and the rib segmentation result.

[0187] The contralateral positioning unit is used to obtain the relative positional relationship between the contralateral region of the abnormal region and the contralateral rib of the abnormal rib based on the relative positional relationship between the abnormal region and the abnormal rib, thereby obtaining the positional information of the contralateral region of the abnormal region.

[0188] In some optional embodiments, the location information of the abnormal region is used to indicate the position of the center point of the detection frame of the abnormal region, and the process of obtaining the relative positional relationship between the abnormal region and the abnormal rib includes:

[0189] The vertex that is shorter than the center point of the detection box of the abnormal region at both ends of the abnormal rib is taken as the first target vertex;

[0190] A first vector is constructed with the first target vertex as the starting point and the center point of the detection box of the abnormal region as the ending point. The first vector is used to indicate the relative positional relationship between the abnormal region and the abnormal rib.

[0191] In some optional embodiments, the contralateral positioning unit is used for:

[0192] Using the vertical axis of the image coordinate system where the dual-energy skeleton image is located as a reference, an auxiliary line is constructed through the first target vertex. The auxiliary line is used as the axis of symmetry to mirror the first vector to obtain the intermediate vector.

[0193] The vertices at both ends of the opposite rib of the abnormal rib that are symmetrical with respect to the first target vertex about the spine are taken as the second target vertex.

[0194] The intermediate vector is translated to obtain a second vector starting from the second target vertex. The second vector is used to indicate the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib.

[0195] In some alternative embodiments, the apparatus further includes:

[0196] The false positive detection module is used to determine that there is no fracture in the dual-energy skeletal image when the similarity between an abnormal region in the dual-energy skeletal image and its opposite region is greater than the preset threshold.

[0197] In some optional embodiments, the process of obtaining the similarity between the abnormal region in the dual-energy skeleton image and its contralateral region includes: inputting the dual-energy skeleton image into a similarity detection model to output the similarity between the abnormal region in the dual-energy skeleton image and its contralateral region.

[0198] The training process of the similarity detection model is as follows:

[0199] The sample sub-images corresponding to the detection boxes of the abnormal region with fracture and its opposite region are used as negative sample pairs, and the sample sub-images corresponding to the detection boxes of the abnormal region without fracture and its opposite region are used as positive sample pairs.

[0200] The similarity detection model is obtained by training a pre-defined convolutional neural network using the positive and negative sample pairs.

[0201] See Figure 11 This application also provides an electronic device, which includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0202] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0203] The memory 210 also stores a computer program, which can be executed by the processor 220, causing the processor 220 to perform the steps of the method in the embodiments of this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above method embodiments, and some contents will not be repeated.

[0204] The memory 210 may also include a utility 214 having at least one program module 215, such program module 215 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0205] Accordingly, processor 220 can execute the aforementioned computer program, and can also execute utility 214.

[0206] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0207] The electronic device can also communicate with one or more external devices 240, such as a keyboard, pointing device, Bluetooth device, etc., and with one or more devices capable of interacting with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can be performed via input / output interface 250. Furthermore, the electronic device can communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of the electronic device via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0208] This application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it implements the steps of the fracture detection method in this application. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-described fracture detection method embodiments, and some contents will not be repeated.

[0209] Figure 12This embodiment illustrates a program product for implementing the fracture detection method described above. It can employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this invention is not limited thereto. In this application, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0210] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0211] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A method for detecting fractures, characterized in that, The method includes: Acquire a chest X-ray image, and obtain a corresponding dual-energy skeleton image based on the chest X-ray image; The dual-energy skeleton image is input into a preset anomaly detection model to obtain the location information of the abnormal region; Based on the location information of the abnormal region, the location information of the abnormal region on the opposite side of the spine is obtained; When the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is less than or equal to a preset threshold, it is determined that there is a fracture in the dual-energy skeleton image. The step of obtaining the location information of the abnormal region on the opposite side of the spine based on the location information of the abnormal region includes: The dual-energy skeleton image is input into a preset skeleton segmentation model to obtain rib segmentation results; Based on the location information of the abnormal region and the rib segmentation result, the abnormal rib and its opposite rib are determined from all the ribs in the dual-energy skeleton image. The location information of the abnormal region is used to indicate the position of the center point of the detection box of the abnormal region. The rib where the center point of the detection box is located is the abnormal rib. Based on the relative positional relationship between the abnormal region and the abnormal rib, the relative positional relationship between the region opposite the abnormal region and the rib opposite the abnormal rib is obtained, thereby obtaining the positional information of the region opposite the abnormal region.

2. The fracture detection method according to claim 1, characterized in that, The process of obtaining the corresponding dual-energy skeletal image based on the chest X-ray image includes: The chest X-ray image is input into a preset image mapping model to obtain the corresponding dual-energy skeletal image; The image mapping model is obtained by training a pre-defined convolutional neural network.

3. The fracture detection method according to claim 1, characterized in that, The location information of the abnormal region is used to indicate the position of the center point of the detection frame of the abnormal region. The process of obtaining the relative positional relationship between the abnormal region and the abnormal rib includes: The vertex that is shorter than the center point of the detection box of the abnormal region at both ends of the abnormal rib is taken as the first target vertex; A first vector is constructed with the first target vertex as the starting point and the center point of the detection box of the abnormal region as the ending point. The first vector is used to indicate the relative positional relationship between the abnormal region and the abnormal rib.

4. The fracture detection method according to claim 3, characterized in that, The step of obtaining the relative positional relationship between the region opposite the abnormal area and the rib opposite the abnormal rib based on the relative positional relationship between the abnormal region and the abnormal rib includes: Using the vertical axis of the image coordinate system where the dual-energy skeleton image is located as a reference, an auxiliary line is constructed through the first target vertex. The auxiliary line is used as the axis of symmetry to mirror the first vector to obtain the intermediate vector. The vertices at both ends of the opposite rib of the abnormal rib that are symmetrical with respect to the first target vertex about the spine are taken as the second target vertex. The intermediate vector is translated to obtain a second vector starting from the second target vertex. The second vector is used to indicate the relative positional relationship between the opposite region of the abnormal region and the opposite rib of the abnormal rib.

5. The fracture detection method according to claim 1, characterized in that, The method further includes: when the similarity between an abnormal region in the dual-energy skeleton image and its opposite region is greater than the preset threshold, it is determined that there is no fracture in the dual-energy skeleton image.

6. The fracture detection method according to claim 5, characterized in that, The process of obtaining the similarity between the abnormal region and its opposite region in the dual-energy skeleton image includes: inputting the dual-energy skeleton image into a similarity detection model to output the similarity between the abnormal region and its opposite region in the dual-energy skeleton image; The training process of the similarity detection model is as follows: The sample sub-images corresponding to the detection boxes of the abnormal region with fracture and its opposite region are used as negative sample pairs, and the sample sub-images corresponding to the detection boxes of the abnormal region without fracture and its opposite region are used as positive sample pairs. The similarity detection model is obtained by training a pre-defined convolutional neural network using the positive and negative sample pairs.

7. A fracture detection device, characterized in that, The device includes: An image mapping module is used to acquire chest X-ray images and acquire corresponding dual-energy skeleton images based on the chest X-ray images; An anomaly detection module is used to input the dual-energy skeleton image into a preset anomaly detection model to obtain the location information of the abnormal region; The contralateral acquisition module is used to acquire the location information of the abnormal region in the contralateral region of the spine based on the location information of the abnormal region. The fracture determination module is used to determine that a fracture exists in the dual-energy skeleton image when the similarity between an abnormal region in the dual-energy skeleton image and its contralateral region is less than or equal to a preset threshold. The contra-side acquisition module includes: The rib segmentation unit is used to input the dual-energy skeleton image into a preset skeleton segmentation model to obtain the rib segmentation result; A rib localization unit is used to determine the abnormal rib and its opposite rib from all ribs in the dual-energy skeleton image based on the location information of the abnormal region and the rib segmentation result. The location information of the abnormal region is used to indicate the position of the center point of the detection box of the abnormal region. The rib where the center point of the detection box is located is the abnormal rib. The contralateral positioning unit is used to obtain the relative positional relationship between the contralateral region of the abnormal region and the contralateral rib of the abnormal rib based on the relative positional relationship between the abnormal region and the abnormal rib, thereby obtaining the positional information of the contralateral region of the abnormal region.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images