X-ray image bone detection method and system based on deep learning

Abstract

The present disclosure relates to the technical field of image processing, and provides an X-ray image bone detection method and system based on deep learning, which comprises the following steps: acquiring an X-ray image to be detected and performing pretreatment; inputting the pretreated image into a trained two-stage double-U network to perform coarse feature extraction on the bone region in the image, layer-by-layer refining the extracted coarse features from high level to low level to obtain multi-layer refined feature maps with increasingly rich details, and performing layer-by-layer fusion on the refined feature maps to generate a final bone region saliency map. The present disclosure detects the salient target of the bone region in the X-ray image through the two-stage double-U network, and the two-stage feature extraction and refinement operation can ensure the accuracy of the detection, and the network fuses the feature maps processed through the two stages to fully utilize the high-level semantic information contained in different levels, which can simultaneously ensure the detection accuracy and the speed of image processing.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to a method and system for skeleton detection in X-ray images based on deep learning. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] X-ray images, relying on X-ray imaging technology, can display information about objects invisible to the naked eye on film or a screen, such as bones, human tumors, tires, and defects in industrial products. From an image processing perspective, the essence is to convert the internal material distribution information of the object being inspected into pixel size, i.e., the brightness and darkness of the image. X-ray images are widely used in medicine, security inspection, and industrial defect detection to assist doctors in diagnosing illnesses and security personnel in detecting dangerous goods and defective industrial products. X-ray images can be used for bone detection. For example, by taking an X-ray image of a pig's leg with meat attached, the distribution relationship of its internal bones, muscles, and fat can be obtained. By extracting key bone information from the X-ray image using bone detection methods, bone edge extraction can be achieved, which can guide robots to perform segmentation and prepare for the task of separating bone from meat.

[0004] The inventors discovered that most existing X-ray image processing methods use image enhancement and noise reduction to improve image quality to aid in manual diagnosis; or they use morphological methods or thresholding for target detection. However, the target processing results of existing methods are difficult to obtain complete and accurate bone edges, which is insufficient to meet the needs of robot segmentation path planning in the next step. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a deep learning-based method and system for skeleton detection in X-ray images. By constructing a two-stage double-U-shaped network, the system detects skeletal regions in X-ray images. The two-stage feature extraction and thinning operations ensure detection accuracy while simultaneously maintaining image processing speed.

[0006] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0007] One or more embodiments provide a deep learning-based method for skeleton detection in X-ray images, comprising the following steps:

[0008] Acquire the X-ray image to be detected and perform preprocessing;

[0009] The preprocessed image is input into a trained two-stage dual-U network to extract coarse features from the skeletal region in the image. The extracted coarse features are then refined layer by layer from high to low level to obtain multi-layer refined feature maps with increasingly rich details. The refined feature maps are then fused layer by layer to generate the final skeletal region saliency map.

[0010] One or more embodiments provide a deep learning-based X-ray image skeleton detection system, comprising:

[0011] Data acquisition module: configured to acquire and preprocess the X-ray image to be detected;

[0012] Data processing module: It is configured to input the preprocessed image into the trained two-stage double U-shaped network, perform coarse feature extraction on the skeletal region in the image, refine the extracted coarse features layer by layer from high level to low level to obtain multi-layer refined feature maps with increasingly rich details, and fuse the refined feature maps layer by layer to generate the final saliency map of the skeletal region.

[0013] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps described in the above method.

[0014] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps described in the above method.

[0015] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0016] In this disclosure, a two-stage dual-U-shaped network is used to detect salient targets in the skeletal region of X-ray images. The two-stage feature extraction and thinning operations can ensure the accuracy of detection. At the same time, the network integrates the feature maps processed in the two stages to make full use of the high-level semantic information contained in different levels, which can simultaneously ensure the detection accuracy and the speed of image processing. It can solve the problems of false detection and incomplete detection, as well as large memory consumption in threshold segmentation methods.

[0017] The advantages of this disclosure, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0018] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0019] Figure 1 This is a flowchart of the X-ray image skeleton detection method in Embodiment 1 of this disclosure;

[0020] Figure 2 This is a schematic diagram of the image preprocessing method in Embodiment 1 of this disclosure;

[0021] Figure 3 This is a schematic diagram of the two-stage double U-shaped network framework in Embodiment 1 of this disclosure;

[0022] Figure 4 This is a schematic diagram of the operation of the double-U-shaped network, an important component of the network in Embodiment 1 of this disclosure;

[0023] Figure 5 This is a schematic diagram of the residual convolutional block of the backbone network, an important component of the network in Embodiment 1 of this disclosure. Detailed Implementation

[0024] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0026] It should be noted that the terminology used herein is for descriptive purposes only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0027] Example 1

[0028] In one or more of the technical solutions disclosed in the embodiments, such as Figures 1-5 As shown, a deep learning-based method for skeleton detection in X-ray images includes the following steps:

[0029] Step 1: Acquire the X-ray image to be detected and perform preprocessing to remove a large amount of noise present during X-ray image acquisition;

[0030] Step 2: Input the preprocessed image into the trained two-stage double U-shaped network to extract coarse features from the skeletal region in the image. Then, refine the extracted coarse features layer by layer from high level to low level to obtain multi-layer refined feature maps with increasingly rich details. Finally, fuse the refined feature maps layer by layer to generate the final saliency map of the skeletal region.

[0031] The two-stage dual-U-shaped network includes a single-encoder-dual-decoder U-shaped network, a salient and marginal U-shaped network, and a feature map fusion module.

[0032] The two-stage dual-U-shaped network obtains multiple salient and edge coarse feature maps at different scales through a single-encoder-dual-decoder U-shaped network. The two types of coarse feature maps are refined layer by layer from high level to low level through independent salient and edge U-shaped networks, resulting in increasingly detailed refined feature maps. Finally, the feature map fusion module fuses the refined feature maps layer by layer to generate the final skeletal region salient map.

[0033] In this embodiment, a two-stage dual-U-shaped network is used to detect salient targets in the skeletal region of X-ray images. The two-stage feature extraction and thinning operations can ensure the accuracy of detection. At the same time, the network integrates the feature maps processed in the two stages to make full use of the high-level semantic information contained in different levels, which can simultaneously ensure the detection accuracy and the speed of image processing. It can solve the problems of false detection and incomplete detection, as well as large memory consumption in threshold segmentation methods.

[0034] Applying this technology to bone detection in pig leg X-ray images can effectively assist in the next step of extracting the edges and key points of the pig leg bones, resulting in more accurate and real-time detection.

[0035] Optional, a two-stage double-U network structure diagram, such as Figure 3 and Figure 4 As shown in the figure. Conv represents the convolutional layer.

[0036] In some embodiments, the single-encoder-dual-decoder U-shaped network includes multiple encoders Eni cascaded in sequence, each encoder being connected to a salient coarse feature decoder SDei and an edge feature decoder BDei. In this embodiment, the encoding and decoding are each set to 5 layers, i = 1, 2, 3, 4, 5.

[0037] Optionally, the encoder Eni can employ convolutional blocks of a residual network (ResNet-50). The En1 convolutional block can consist of 7×7 convolutional layers with a stride of 2 and an encoded feature size of (w / 2)×(h / 2)×64. The En2 convolutional block can have a stride of 2, and the pooling convolutions consist of 3×3 max-pooling layers and 3 sets of residual convolutions, where the residual convolutions are as follows... Figure 5 The En3 convolutional block consists of 4 sets of residual convolutions, with a coding feature size of (w / 4)×(h / 4)×256; the En4 convolutional block consists of 6 sets of residual convolutions, with a coding feature size of (w / 16)×(h / 16)×1024; and the En5 convolutional block consists of 3 sets of residual convolutions, with a coding feature size of (w / 32)×(h / 32)×2048.

[0038] Optionally, the salient coarse feature decoder SDei and the edge feature decoder BDei use the same decoding modules. Each decoding module includes two convolutional layers, a BatchNorm layer, and an activation module connected in sequence. The activation module can use the ReLU activation function. The input of each decoder is a fusion feature of the bilinear upsampled output of the previous stage and the output of the corresponding stage in the encoder. The decoding feature sizes of the five levels from high to low can be set to (w / 32)×(h / 32)×64, (w / 16)×(h / 16)×64, (w / 8)×(h / 8)×64, (w / 4)×(h / 4)×64, and (w / 2)×(h / 2)×64, respectively.

[0039] In some embodiments, the salient and edge U-shaped networks include a salient feature refinement U-shaped network and an edge feature refinement U-shaped network, and can employ the same network structure. The salient feature refinement U-shaped network includes multiple cascaded encoders (SUEni) and corresponding sequentially connected decoders (SUDei). A top-level refinement module (TopRefine) is also provided between the encoders and decoders. The top-level refinement module (TopRefine) can include multiple convolutional layers connected sequentially end to end.

[0040] The salient feature refinement U-shaped network consists of multiple cascaded encoders BUEni and corresponding sequentially connected decoders BUDei. A top-level refinement module, TopRefine, is also located between the encoders and decoders. TopRefine can include multiple convolutional layers connected sequentially end-to-end.

[0041] In some embodiments, the feature map fusion module includes a pixel addition module, a fusion convolution module, and a bilinear upsampling module.

[0042] The fused output prediction map is obtained by restoring the saliency and edge feature maps of the two-stage double U-shaped network and the fused feature map to the original image size through prediction convolution and bilinear upsampling, and the resulting output prediction map is the saliency map of the skeletal region.

[0043] Furthermore, the training process of the two-stage double-U network includes the following steps:

[0044] Step S1: Perform pixel-level skeletal region labeling on the acquired X-ray images to form a training set;

[0045] S1.1: Images can be captured by an X-ray machine at different angles;

[0046] S1.2: Automatically label each pixel using a threshold segmentation algorithm, and then manually correct the bone edge region pixel by pixel to obtain the ground truth map label.

[0047] Step S2: Preprocess the images in the training set;

[0048] Specifically, the preprocessing includes: mean subtraction, random horizontal flipping for data augmentation, and finally, using the data as network input to train a two-stage double U-shaped network.

[0049] Step S3: Input the preprocessed image into a two-stage dual U-shaped network for training. Then, perform coarse feature extraction of salience and edges based on a single encoder-dual decoder U-shaped network, refine the salience and edges based on independent salience and edge U-shaped network features to obtain refined feature maps of salience and edges, and perform layer-by-layer fusion of the refined feature maps to generate the final salience map of the skeletal region.

[0050] During training, the processing in the two-stage dual-U-shaped network includes a coarse feature extraction stage based on a single-encoder-dual-decoder U-shaped network, a feature refinement stage based on independent salient and marginal U-shaped networks, and a fusion processing of salient and marginal information, as detailed below:

[0051] In the coarse feature extraction stage, the input image is encoded in five levels from low to high levels through a single-encoder-dual-decoder U-shaped network. This is called single encoding. Then, the dual-decoder U-shaped network is used to decode the salient and edge coarse features layer by layer from high to low levels.

[0052] In the feature refinement stage, salient and marginal U-shaped networks are used to gradually refine the coarse features from high to low levels. The features are then further refined by the refinement module at the top of the network to obtain abstract semantic information, resulting in a refined feature map with increasingly rich details.

[0053] In the fusion operation, the feature map fusion module gradually integrates and upsamples the salient and edge-refined feature maps of the corresponding scale by using pixel addition, fusion convolution and bilinear upsampling layers to obtain the skeletal region information map.

[0054] Step S4: By supervising the saliency refinement feature map, edge refinement feature map, and fused skeletal region saliency map, calculate the cross-entropy loss and IOU loss of each feature map and the labeled label (label edge), optimize the convolution parameters of the model, and obtain the trained two-stage double U-shaped network.

[0055] The following section uses the bone detection of pig leg X-ray images as an example to illustrate the training process of a two-stage dual-U-shaped network.

[0056] Specifically, SA1: Obtain X-ray images of pig legs with arbitrary image size w×h, label the images, determine the labels, and then construct training and testing sets.

[0057] The specific process of step SA1 is as follows:

[0058] SA1.1: The dataset was formed by X-ray images of different pig forelegs and hind legs.

[0059] SA1.2: Automatically label each pixel using a threshold segmentation algorithm, and then manually correct the bone edge region pixel by pixel to obtain the ground truth map label.

[0060] SA1.3: Reference Figure 2 We employ random horizontal flipping to enhance the training set and improve training accuracy, while normalizing by subtracting the mean.

[0061] Step SA2 involves preprocessing the images in the training set. The specific process is as follows:

[0062] SA2.1: Calculate the average value of all image pixels. Subtract the mean value from the input image before training the network. Normalizing the pixel values ​​can improve the calculation accuracy of the network.

[0063] SA2.2: Randomly flip the normalized image and the labeled image horizontally, and then input them into the network to complete the training.

[0064] SA3: Input the training set of preprocessed pig leg X-ray images into, for example... Figure 3 The training is performed in the two-stage double U-shaped network shown.

[0065] The input image is first encoded by a series of convolutional operations in a single-encoding backbone network to obtain multi-resolution features from low to high levels. Then, the first-stage dual-decoding U-shaped network decodes from high to low levels to obtain salient and coarse edge features at five levels. The two types of coarse feature maps are then refined by the second-stage dual U-shaped network to obtain feature maps with rich details. This process uses multiple 7×7, 3×3, and 1×1 convolutions to extract the semantic information contained in the feature maps at different levels.

[0066] Taking five layers as an example, the specific process of step SA3 is as follows:

[0067] SA3.1: In the first stage of single-encoding feature processing, the preprocessed input image (w×h) obtains coarse encoded features through the first five convolutional blocks of the ResNet-50 residual network: the En1 convolutional block consists of a 7×7 convolutional layer with a stride of 2 and an encoded feature size of (w / 2)×(h / 2)×64; the En2 convolutional block consists of a max-pooling layer with a stride of 2 and a pooling convolution of 3×3, and 3 sets of residual convolutions, where the residual convolutions are as follows... Figure 5The En3 convolutional block consists of 4 sets of residual convolutions, with a coding feature size of (w / 4)×(h / 4)×256; the En4 convolutional block consists of 6 sets of residual convolutions, with a coding feature size of (w / 16)×(h / 16)×1024; and the En5 convolutional block consists of 3 sets of residual convolutions, with a coding feature size of (w / 32)×(h / 32)×2048.

[0068] SA3.2: In the first stage of dual decoding feature processing, salient and edge feature decoders (such as...) Figure 3 (Academics SDei and BDei), it consists of five sets of decoding convolutions. Structurally, it is symmetrical to the encoder, but the module design differs between the two. The decoding module consists of two convolutional layers, a BatchNorm layer, and a ReLU activation function. The input to each module is a fusion feature of the bilinear upsampled output of the previous stage and the output of the corresponding stage in the encoder. The decoding feature sizes of the five stages from high to low are (w / 32)×(h / 32)×64, (w / 16)×(h / 16)×64, (w / 8)×(h / 8)×64, (w / 4)×(h / 4)×64, and (w / 2)×(h / 2)×64, respectively.

[0069] SA3.3: The second-stage salient feature refinement U-shaped network also adopts an encoder-decoder structure. The encoder SUEn1 is equipped with convolutional layers (3×3 kernels, stride of 2, and 64 channels), BatchNorm layers, and ReLU activation functions. The convolutional layers with a stride of 2 can halve the feature size, and the output size is (w / 4)×(h / 4)×64. The coarse features of the corresponding level and the encoded features of the previous level are concatenated as the inputs to the four decoders SUEn2 to SUEn5. Therefore, the number of input channels of the convolutional layers of these four decoders is set to 128, and the other parameters and number of layers are the same as SUEn1. The corresponding output feature sizes are (w / 8)×(h / 8)×64, (w / 16)×(h / 16)×64, (w / 32)×(h / 32)×64, and (w / 64)×(h / 64)×64. A top-level saliency feature refinement module is designed between the encoding and decoding structures. This module consists of four convolutional blocks with parameters set to 3×3 kernels, a stride of 1, and 64 channels. Top-level refinement does not change the feature size, and the output size is (w / 64)×(h / 64)×64. The SUDe5 decoder first concatenates the top-level refined features with the corresponding layer's encoded features. Then, it uses a deconvolutional layer (4×4 kernels, stride of 2) to decode the features as (w / 32)×(h / 32)×64. This is followed by a BatchNorm layer and a ReLU activation function for normalization and non-linear processing to accelerate the training process. SUDe4 to SUDe1 follow the same procedure, resulting in output saliency feature sizes of (w / 16)×(h / 16)×64, (w / 8)×(h / 8)×64, (w / 4)×(h / 4)×64, and (w / 2)×(h / 2)×64.

[0070] SA3.4: In the second stage of edge feature refinement, similar to S3.4, the output refined edge feature size is (w / 32)×(h / 32)×64, (w / 16)×(h / 16)×64, (w / 8)×(h / 8)×64, (w / 4)×(h / 4)×64, (w / 2)×(h / 2)×64.

[0071] SA3.5: The two-class refined feature map fusion module has pixel addition, fusion convolution and bilinear upsampling layer. The fusion convolution parameters are 3×3 kernel, stride of 3, and number of channels of 64. After that, a BatchNorm layer and ReLU activation function are set. The final output feature map size of the fusion module is (w / 2)×(h / 2)×64.

[0072] SA3.6: Reference Figure 3 Significant refined features S were obtained from two feature refinement networks and a fusion module. 1 r, edge refinement feature B 1r and fused features are then used to obtain the output feature map S through predictive convolution (3×3 kernel, stride 1, number of channels 1) and bilinear upsampling. The output salient feature map S is then obtained. wo And edge feature map B, with size w×h×1.

[0073] SA3.7: A two-stage double U-shaped network outputs a feature map S through supervision, and outputs a salient feature map Ss. wo Using edge feature map B, calculate the network's cross-entropy loss and IOU loss to optimize the network. The loss calculation formula is as follows:

[0074]

[0075] Where L represents the total network loss.

[0076] L bce Represents the cross-entropy loss, and its calculation formula is:

[0077]

[0078] Where g(x,y)∈{0,1} represents the truth map label, and p(x,y)∈[0,1] represents the predicted probability value of the pixel.

[0079] The IOU loss was initially used to measure the similarity between two sets. When applied to skeletal region (salient feature) detection tasks, it measures the similarity between the predicted map and the ground truth map (between the maps themselves). The calculation formula is as follows:

[0080]

[0081] This embodiment utilizes a two-stage dual-U-shaped network to solve the problem of skeletal region detection in pig leg X-ray images. Acquired images are pixel-level labeled to form training and testing sets. During training, the images are randomly horizontally flipped to expand the training set. In the first stage, a single-encoder-dual-decoder U-shaped structure is used to extract salient and coarse edge features to locate the skeletal region. In the second stage, a five-level coarse feature network based on independent salient and edge features is used to gradually refine the five levels of coarse features from high to low levels, resulting in a feature map rich in detail. The feature fusion process further explores the hierarchical correlation between salient and edge features, ensuring the integrity and accuracy of the feature map. Cross-entropy loss and IOU loss are introduced in the loss calculation stage to supervise the salient and edge feature maps. This effectively assists in the next step of extracting pig leg bone edges and edge key points, while meeting the requirements of accuracy and real-time performance. It solves the problems of false detections and incomplete detection, as well as large memory consumption, that exist in threshold segmentation methods.

[0082] Example 2

[0083] Based on Embodiment 1, this embodiment provides a deep learning-based X-ray image skeleton detection system, including:

[0084] Data acquisition module: configured to acquire and preprocess the X-ray image to be detected;

[0085] Data processing module: It is configured to input the preprocessed image into the trained two-stage double U-shaped network, perform coarse feature extraction on the skeletal region in the image, refine the extracted coarse features layer by layer from high level to low level to obtain multi-layer refined feature maps with increasingly rich details, and fuse the refined feature maps layer by layer to generate the final saliency map of the skeletal region.

[0086] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0087] Example 3

[0088] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it performs the steps described in the method of Embodiment 1.

[0089] Example 4

[0090] This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps described in the method of Embodiment 1.

[0091] The electronic devices proposed in this disclosure can be mobile terminals and non-mobile terminals. Non-mobile terminals include desktop computers, and mobile terminals include smartphones (such as Android phones, iOS phones, etc.), smart glasses, smartwatches, smart bracelets, tablets, laptops, personal digital assistants, and other mobile internet devices capable of wireless communication.

[0092] It should be understood that in this disclosure, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0093] The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store information about the device type.

[0094] In implementation, each step of the above method can be completed by integrated logic circuits in the processor hardware or by instructions in software. The steps of the method disclosed herein can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here. Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0095] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0096] In the embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0097] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0099] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A deep learning-based method for skeleton detection in X-ray images, characterized in that, Includes the following steps: Acquire the X-ray image to be detected and perform preprocessing; The preprocessed image is input into a trained two-stage double U-shaped network to extract coarse features from the skeletal region in the image. The extracted coarse features are then refined layer by layer from high level to low level to obtain multi-layer refined feature maps with increasingly rich details. The refined feature maps are then fused layer by layer to generate the final skeletal region saliency map. The two-stage dual-U-shaped network includes a single-encoder-dual-decoder U-shaped network, salient and edge U-shaped networks, and a feature map fusion module. The single-encoder-dual-decoder U-shaped network includes multiple encoders Eni cascaded in sequence, each encoder Eni is connected to a salient coarse feature decoder SDei and an edge feature decoder BDei. The feature map fusion module includes a pixel addition module, a fusion convolution module, and a bilinear upsampling module. The salient and edge U-shaped networks include a salient feature refinement U-shaped network and an edge feature refinement U-shaped network, which adopt the same network structure. The salient feature refinement U-shaped network or the edge feature refinement U-shaped network includes multiple cascaded encoders and corresponding decoders connected in sequence. A top-level refinement module is also set between the encoder and the decoder. The processing in the two-stage dual-U-shaped network includes a coarse feature extraction stage based on a single-encoder-dual-decoder U-shaped network, a feature refinement stage based on independent salient and marginal U-shaped networks, and a fusion processing of salient and marginal information, as detailed below: In the coarse feature extraction stage, the input image is encoded in five levels from low to high levels through a single-encoder-dual-decoder U-shaped network. This is called single encoding. Then, the dual-decoder U-shaped network is used to decode the salient and edge coarse features layer by layer from high to low levels. In the feature refinement stage, salient and marginal U-shaped networks are used to gradually refine the coarse features from high to low levels. The features are then further refined by the refinement module at the top of the network to obtain abstract semantic information, resulting in a refined feature map with increasingly rich details. In the fusion operation, the feature map fusion module gradually integrates and upsamples the salient and edge-refined feature maps of the corresponding scale by using pixel addition, fusion convolution and bilinear upsampling layers to obtain the skeletal region information map.

2. The X-ray image skeleton detection method based on deep learning as described in claim 1, characterized in that: The salient coarse feature decoder SDei and the edge feature decoder BDei use the same decoding module. Each decoding module includes two convolutional layers, a BatchNorm layer, and an activation module connected in sequence.

3. The X-ray image skeleton detection method based on deep learning as described in claim 1, characterized in that: The top-level refinement module TopRefine consists of multiple convolutional layers connected end-to-end in sequence.

4. The X-ray image skeleton detection method based on deep learning as described in claim 1, characterized in that: The training process of a two-stage double-U network includes the following steps: The acquired X-ray images were labeled with pixel-level skeletal regions to form a training set; Preprocess the images in the training set; The preprocessed image is input into a two-stage dual U-shaped network for training. Coarse feature extraction of saliency and edges is performed sequentially based on a single encoder-dual decoder U-shaped network. Refined feature maps of saliency and edges are obtained based on independent saliency and edge U-shaped network features. Finally, the refined feature maps are fused layer by layer to generate the final saliency map of the skeletal region. By supervising the refinement of salient feature maps, edge refinement feature maps, and the fused skeletal region salient map, the cross-entropy loss and IOU loss of each feature map with the labeled labels are calculated, and the convolution parameters of the model are optimized to obtain a trained two-stage double U-shaped network.

5. The X-ray image skeleton detection method based on deep learning as described in claim 4, characterized in that, Pixel-level skeletal region labeling is achieved by automatically labeling each pixel using a threshold segmentation algorithm, and then manually correcting the skeletal edge regions pixel by pixel to obtain ground truth map labels. Alternatively, the preprocessing may include: a mean reduction operation and random horizontal flipping for data augmentation.

6. A deep learning-based X-ray image skeleton detection system, characterized in that, include: Data acquisition module: configured to acquire and preprocess the X-ray image to be detected; Data processing module: It is configured to input the preprocessed image into the trained two-stage double U-shaped network, perform coarse feature extraction on the skeletal region in the image, refine the extracted coarse features layer by layer from high level to low level to obtain multi-layer refined feature maps with increasingly rich details, and fuse the refined feature maps layer by layer to generate the final saliency map of the skeletal region. The two-stage dual-U-shaped network includes a single-encoder-dual-decoder U-shaped network, salient and edge U-shaped networks, and a feature map fusion module. The single-encoder-dual-decoder U-shaped network includes multiple encoders Eni cascaded in sequence, each encoder Eni is connected to a salient coarse feature decoder SDei and an edge feature decoder BDei. The feature map fusion module includes a pixel addition module, a fusion convolution module, and a bilinear upsampling module. The salient and edge U-shaped networks include a salient feature refinement U-shaped network and an edge feature refinement U-shaped network, which adopt the same network structure. The salient feature refinement U-shaped network or the edge feature refinement U-shaped network includes multiple cascaded encoders and corresponding decoders connected in sequence. A top-level refinement module is also set between the encoder and the decoder. The processing in the two-stage dual-U-shaped network includes a coarse feature extraction stage based on a single-encoder-dual-decoder U-shaped network, a feature refinement stage based on independent salient and marginal U-shaped networks, and a fusion processing of salient and marginal information, as detailed below: In the coarse feature extraction stage, the input image is encoded in five levels from low to high levels through a single-encoder-dual-decoder U-shaped network. This is called single encoding. Then, the dual-decoder U-shaped network is used to decode the salient and edge coarse features layer by layer from high to low levels. In the feature refinement stage, salient and marginal U-shaped networks are used to gradually refine the coarse features from high to low levels. The features are then further refined by the refinement module at the top of the network to obtain abstract semantic information, resulting in a refined feature map with increasingly rich details. In the fusion operation, the feature map fusion module gradually integrates and upsamples the salient and edge-refined feature maps of the corresponding scale by using pixel addition, fusion convolution and bilinear upsampling layers to obtain the skeletal region information map.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the steps of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the steps of any one of claims 1-5.

Images