Fracture detection method and device, electronic equipment and storage medium
Through the characteristic extraction and boundary box prediction of medical imaging, the bone loss function is constructed and the fracture detection model is trained, the problem of fracture detection accuracy and efficiency in existing technologies is solved, and high accuracy and high efficiency fracture detection is achieved.
Patent Information
- Application Number
- CN202411447041.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-01-28
AI Technical Summary
Existing fracture detection technology is difficult to achieve high accuracy and efficiency when facing complex situations such as subtle fracture characteristics, complex bone structure, and possible overlap and rotation, and doctors are prone to fatigue and subjective judgment when analyzing images. The impact of the increase of the risk of missed and missed detection.
By conducting characteristic extraction and border prediction of medical imaging, the bone loss function is constructed, determining the co -value and position encoding information of the adjacent features of bone images in the training sample image data, generate the results of bone prediction, and train the fracture detection model based on the bone loss function training function. Realize the high adaptability, high accuracy and high efficiency of medical image.
It improves the accuracy and efficiency of bone detection, reduces the risk of missed and misunderstandings, and enhances the adaptability of models in different data sets and tasks.
Smart Images
Figure CN119379636A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of bone fracture detection, and in particular to a bone fracture detection method, device, electronic equipment and storage medium. Background Art
[0002] With the rapid development of medical technology, fractures are common injuries of the skeletal system, and their accurate and timely diagnosis is of great significance for formulating treatment plans and evaluating prognosis. Therefore, various imaging technologies are widely used in fracture detection to improve the accuracy and efficiency of diagnosis.
[0003] In the existing technology, a variety of deep learning methods have been used in fracture detection, greatly improving the efficiency of clinical diagnosis of fractures. These technologies provide clinicians with indispensable visual evidence when diagnosing and determining fracture conditions, including key information such as the specific location of the fracture, the severity of the injury, and the morphology of the fracture line. However, due to the subtlety of fracture characteristics and the complexity of bone structure, such as the dispersion and irregular shape of fracture fragments, as well as possible overlapping, rotation and other complex situations, it poses challenges to orthopedic surgeons in diagnosing fracture types and assessing the extent of injuries. Clinicians are required to have a long period of clinical diagnosis experience and rich knowledge learning. At the same time, doctors bear a huge workload when analyzing images, and even experienced doctors are easily affected by fatigue and subjective judgment, which increases the risk of missed detection and misdetection.
[0004] Therefore, there is an urgent need for a fracture detection method that can achieve highly adaptive, accurate and efficient detection of medical images by performing feature extraction and bounding box prediction on medical images. Summary of the invention
[0005] In view of this, it is necessary to provide a fracture detection method, device, electronic device and storage medium, which can achieve highly adaptive, accurate and efficient detection of medical images by performing feature extraction and bounding box prediction on medical images.
[0006] In order to solve the above technical problems, on the one hand, the present invention provides a fracture detection method, comprising: A bone loss function is constructed according to the coordinates of the corner points of the predicted bounding box and the actual bounding box of the bone image, their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; Determine the coherence value and position coding information of adjacent features of the bone image in the training sample image data; Generate a skeleton prediction result according to the coherent value and position encoding information of the adjacent features; A fracture detection model is trained according to the bone loss function, the bone prediction result and the training sample image data, and fracture detection is performed on the bone image to be detected based on the trained fracture detection model.
[0007] In one possible implementation, a bone loss function is constructed based on the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor, including: Constructing an IoU loss function based on the overlapping area and non-overlapping area of the predicted bounding box and the actual bounding box of the skeleton image; The size factor is determined according to the MPDIoU loss function, and the bone loss function is constructed according to the IoU loss function, the size factor, the corner point coordinates of the predicted bounding box and the corner point coordinates corresponding to the actual bounding box, the height and width of the bone image, and the size factor.
[0008] In a possible implementation, the corner point coordinates include the upper left corner coordinates and the lower right corner coordinates, and the expression of the skeleton loss function is: , , in, is the IoU loss function value, To predict the bounding box, is the actual bounding box, is the distance between the upper left corner coordinates of the predicted bounding box and the upper left corner coordinates of the actual bounding box, is the distance between the lower right corner coordinates of the predicted bounding box and the lower right corner coordinates of the actual bounding box, is the height of the bone image, is the width of the bone image, is the size factor.
[0009] In a possible implementation, determining the coherent value and position encoding information of adjacent features of a bone image includes: Obtain bone images; Performing offline data enhancement processing on the bone image to generate training sample image data; Extracting feature information of the training sample image data and determining the coherence value of adjacent features; Position encoding is performed on the feature information to generate position encoding information.
[0010] In a possible implementation manner, extracting feature information of the training sample image data and determining coherent values of adjacent features includes: Extracting feature information of the training sample image data based on dynamic snake convolution; Based on bilinear interpolation, the deformation offsets between adjacent features are calculated in sequence to determine the coherence values of adjacent features.
[0011] In a possible implementation manner, position encoding is performed on the feature information to generate position encoding information, including: Based on the ConvS2S model, extracting the sequence dynamic features of the training sample image data; Based on a learnable position coding model, learnable position coding is added to the sequence dynamic features to generate position coding information.
[0012] In a possible implementation, training a fracture detection model according to the bone loss function, the bone prediction result and the training sample image data includes: An initial fracture detection model is constructed based on the RT-DETR large model, dynamic snake convolution, learnable position encoding model, and bone loss function; Iteratively train the fracture detection model according to the training sample image data to generate a bone prediction result; Based on the bone loss function, a bone loss function value between a bone prediction result and an actual bone result in the training sample image data is calculated; The bone loss function value is compared with a preset threshold value to determine the optimal model parameters and generate a final fracture detection model.
[0013] In a second aspect, the present invention further provides a fracture detection device, comprising: A loss function construction module is used to construct a bone loss function according to the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and a size factor; A feature information module, used to determine the coherence value and position coding information of adjacent features of the bone image in the training sample image data; A skeleton prediction module, used for generating a skeleton prediction result according to the coherent value and position coding information of the adjacent features; The fracture detection module is used to train a fracture detection model according to the bone loss function, the bone prediction result and the training sample image data, and perform fracture detection on the bone image to be detected based on the trained fracture detection model.
[0014] In a third aspect, the present invention also provides an electronic device comprising a memory and a processor, wherein the memory is used to store programs and data; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the fracture detection method as described above, and / or to implement the fracture detection as described above.
[0015] In a fourth aspect, the present invention further provides a computer storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the fracture detection method as described above.
[0016] The beneficial effects of the present invention are as follows: first, a bone loss function is constructed according to the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; then, the coherence value and position coding information of the adjacent features of the bone image in the training sample image data are determined; then, the bone prediction result is generated according to the coherence value and position coding information of the adjacent features; finally, a fracture detection model is trained according to the bone loss function, the bone prediction result, and the training sample image data, and fracture detection is performed on the bone image to be detected based on the trained fracture detection model. The present invention adopts a size factor to optimize the loss function, constructs a bone loss function by the overlapping area and non-overlapping area of the predicted bounding box and the actual bounding box, the width and height of the bone image, and the size factor, predicts the bone bounding box according to the coherence value and position coding information of the adjacent features of the bone image, calculates the loss function value between the predicted result and the actual image according to the predicted bounding box, the actual bounding box, and the height and width of the bone image, and combines the size factor, iteratively trains the fracture detection model according to the loss function, and performs fracture detection on the bone image to be detected according to the trained fracture detection model. The present invention evaluates the fracture status by predicting the overlapping area and non-overlapping area between the bounding box and the actual bounding box, the width and height of the bone image, and the size factor, thereby improving the model's adaptability to detection on different data sets and tasks and improving the accuracy and efficiency of bone detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 A schematic diagram of an embodiment of a fracture detection method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of an embodiment of step S101; Figure 3 For the present invention Figure 1 A schematic diagram of a flow chart of an embodiment of step S102; Figure 4 A comparison chart of the medical image samples before and after enhancement provided by the present invention; Figure 5For the present invention Figure 3 A schematic diagram of a flow chart of an embodiment of step S303; Figure 6 For the present invention Figure 3 A flowchart of an embodiment of step S304; Figure 7 For the present invention Figure 1 A schematic flow chart of an embodiment of step S104; Figure 8 The structural diagram of the improved fracture detection model based on the RT-DETR large model provided by the present invention; Fig. 9 A schematic diagram of the structure of an embodiment of the fracture detection electronic device provided by the present invention; Fig.10 A schematic structural diagram of an embodiment of a fracture detection storage medium provided by the present invention. DETAILED DESCRIPTION
[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0020] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.
[0021] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.
[0022] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0023] Before describing the embodiments, the following definitions are given for the relevant terms: The MPDIoU loss function is a loss function for bounding box regression, which aims to solve the problem that the existing loss function cannot be effectively optimized when the predicted bounding box and the true bounding box have the same aspect ratio but completely different width and height values; Piecewise chaotic mapping is a piecewise defined chaotic system that divides the entire definition domain into multiple subintervals and applies different nonlinear transformations on each subinterval; The ConvS2S model is a new sequence-to-sequence learning model that aims to address some of the limitations of traditional RNN (recurrent neural network) in sequence modeling. By introducing convolutional neural networks (CNN) into the Seq2Seq framework, the model achieves parallel processing of sequence data, improves computational efficiency, and can better capture the relationship between distant elements in the sequence.
[0024] The present invention provides a fracture detection method, device, electronic device and storage medium, which are respectively described below.
[0025] Figure 1 A schematic diagram of an embodiment of the fracture detection method provided by the present invention is shown in FIG. Figure 1 As shown, fracture detection methods include: S101, constructing a bone loss function according to the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; S102, determining the coherence value and position coding information of adjacent features of the bone image; S103, generating a skeleton prediction result according to the coherence value of adjacent features and the position coding information; S104. Based on the bone loss function, the bone loss function value between the bone prediction result and the actual bone image is calculated, and fracture detection is performed according to the bone loss function value.
[0026] Specifically, this embodiment is an improvement on the existing RT-DETR large model. As an advanced target detection model, the performance of the RT-DETR model depends largely on the choice of loss function. This embodiment uses the size factor to optimize the MPDIoU loss function to replace the traditional GIoU loss function or IoU loss function. The optimized loss function can control the size of the bounding box through the size factor, so that the model can adapt to bounding boxes of different sizes. When the predicted bounding box does not match the overlapping area of the actual bounding box, the size factor is used to provide effective conclusion information, thereby improving the detection ability of the model on different data sets and tasks. The performance of the model on the bounding box regression task is improved.
[0027] This embodiment constructs a bone loss function based on the corner coordinates of the predicted bounding box and the actual bounding box of the bone image, their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; determines the coherence value and position coding information of the adjacent features of the bone image in the training sample image data; then, generates a bone prediction result based on the coherence value and position coding information of the adjacent features; trains a fracture detection model based on the bone loss function, the bone prediction result, and the training sample image data, and performs fracture detection on the bone image to be detected based on the trained fracture detection model. This embodiment uses a size factor to optimize the loss function, constructs a bone loss function through the overlapping area and non-overlapping area of the predicted bounding box and the actual bounding box, the width and height of the bone image, and the size factor, predicts the bone bounding box based on the coherence value and position coding information of the adjacent features of the bone image, calculates the loss function value between the predicted result and the actual image based on the predicted bounding box, the actual bounding box, and the height and width of the bone image, and combines the size factor, iteratively trains the fracture detection model based on the loss function, and performs fracture detection on the bone image to be detected based on the trained fracture detection model.
[0028] This embodiment optimizes the loss function by using a size factor, and evaluates the loss function value of the model prediction result and the actual result according to the overlapping area and non-overlapping area between the predicted bounding box and the actual bounding box, the width and height of the bone image, and the size factor, thereby improving the model's adaptability to detection on different data sets and tasks, and improving the adaptability, accuracy, and efficiency of bone detection.
[0029] In some embodiments of the present invention, Figure 2 As shown, Figure 2 The present invention provides Figure 1 The flowchart of an embodiment of step S101 in the embodiment includes: S201, constructing an IoU loss function according to the overlapping area and non-overlapping area of the predicted bounding box and the actual bounding box of the skeleton image; S202. Determine the size factor according to the MPDIoU loss function, and construct a bone loss function according to the IoU loss function, the size factor, the IoU loss function, the corner point coordinates of the predicted bounding box and the corner point coordinates corresponding to the actual bounding box, the height and width of the bone image, and the size factor.
[0030] Specifically, the MPDIoU loss function is an extended optimization of the standard IoU (intersection over union). IoU loss is one of the commonly used loss functions in target detection, which is used to measure the degree of overlap between the predicted bounding box and the actual bounding box. The calculation formula of the IoU loss function is: , in, is the IoU loss function value, To predict the bounding box, is the actual bounding box.
[0031] Furthermore, according to the coordinates of the upper left corner and lower right corner of the predicted bounding box and the actual bounding box, the Euclidean distance between the two bounding box corner points is calculated. The calculation formula is: , , in, is the Euclidean distance between the upper left corner coordinates of the predicted bounding box and the upper left corner coordinates of the actual bounding box, is the Euclidean distance between the lower right corner coordinates of the predicted bounding box and the lower right corner coordinates of the actual bounding box, is the coordinate of the upper left corner of the predicted bounding box, is the coordinate of the lower right corner of the predicted bounding box, is the coordinate of the upper left corner of the actual bounding box, is the lower right corner coordinate of the actual bounding box, is the size factor, which is a coefficient greater than 0 and has a value range of [0.5, 1.5]. It is dynamically adjusted according to different bone image samples and training stages, and can also be determined by hyperparameter optimization technology.
[0032] This embodiment makes full use of the geometric characteristics of the bounding box based on the MPDIoU loss function by predicting the coordinates of the corner points of the bounding box and the actual bounding box, and introduces a size factor on this basis to control the size of the actual bounding box so that the size of the bounding box can be better matched according to different sample characteristics (such as high IoU or low IoU) when calculating the loss. The optimized loss function takes into account that when the predicted bounding box overlaps with the actual bounding box but does not completely match, it can provide more effective gradient information, thereby improving the model's adaptive detection capabilities on different data sets and tasks, and improving the accuracy and adaptability of detection.
[0033] In some embodiments of the present invention, the corner point coordinates include the upper left corner coordinates and the lower right corner coordinates, and the expression of the skeleton loss function is: , , in, is the IoU loss function value, To predict the bounding box, is the actual bounding box, is the distance between the upper left corner coordinates of the predicted marker box and the upper left corner coordinates of the actual bounding box, is the distance between the lower right corner coordinates of the predicted marker box and the lower right corner coordinates of the actual bounding box, is the height of the bone image, is the width of the bone image, is the size factor.
[0034] In some embodiments of the present invention, Figure 3 As shown, Figure 3 The present invention provides Figure 1 The flowchart of an embodiment of step S102 in the embodiment includes: S301, obtaining bone images; S302, performing offline data enhancement processing on the bone image to generate training sample image data; S303, extracting feature information of the training sample image data, and determining the coherence value of adjacent features; S304: Position-encode the feature information to generate position-encoded information.
[0035] It should be noted that in the process of training the optimized RTDETR large model, large-scale data is needed to train the model. However, in medical image processing tasks, due to the patient's privacy and the need for experts in the field to manually annotate the detection images, the acquired medical image data is very limited. Therefore, this embodiment uses offline data enhancement to expand the medical image data.
[0036] Specifically, the albumentations library is used to perform offline data enhancement on the collected training set images. Methods such as cropping, flipping, translation, scaling, and brightness contrast are used to expand the training data set. The images after offline data enhancement are compared with the original images. Figure 4 As shown, Figure 4 This is a comparison chart of the medical image samples before and after enhancement provided by the present invention.
[0037] This embodiment uses offline data enhancement technology to provide the model with richer samples during the training process, which helps to reduce overfitting of specific types of samples, improve the generalization ability of the model on unseen data, and thus improve the accuracy and efficiency of model detection.
[0038] In some embodiments of the present invention, Figure 5 As shown, Figure 5 The present invention provides Figure 3 The flowchart of an embodiment of step S303 in the embodiment includes: S501, extracting feature information of training sample image data based on dynamic snake convolution; S502: Based on bilinear interpolation, the deformation offsets between adjacent features are determined in sequence, and the coherence values of the adjacent features are determined.
[0039] It should be noted that, considering the subtlety of human fractures and the slenderness and complexity of bones, dynamic snake convolution (DSConv) is used to replace the traditional fixed convolution kernel in the RTDETR large model. Dynamic snake convolution has higher flexibility and adaptability, can better adapt to the different shapes and structures of the target, and improve the ability to capture features. The coherence value of adjacent features is a quantitative value that reflects the coherence between features.
[0040] Specifically, the standard convolution kernel is linearly arranged on the x-axis and y-axis. This embodiment takes a convolution kernel of 9 grids as an example. In the layout of the x-axis, the position of each cell is ,in, , indicating that from the center cell The horizontal expansion distance outward. Each cell in the convolution kernel The selection of is a gradual cumulative process. Starting from the central cell , as you expand outward, each new cell The position is based on the previous cell The position plus an offset , the offset Accumulate to ensure that the shape of the convolution kernel remains in a straight line arrangement. The expression for the change of the convolution kernel in the x-axis direction is: , in, For The center The positions of adjacent cells, for The x- and y-coordinates of the cell, is the y-axis offset; The expression of the change of the convolution kernel in the y-axis direction is: , in, For The center The positions of adjacent cells, for The x- and y-coordinates of the cell, is the x-axis offset.
[0041] Furthermore, bilinear interpolation is used to decompose the decimal part of the calculated convolution kernel position coordinates. The specific formula is expressed as follows: , in, is the decimal position of the convolution kernel position coordinates, for all integer spatial positions, is the bilinear interpolation kernel.
[0042] Furthermore, the convolution kernel position coordinates are decomposed into two one-dimensional kernels according to the decimal position of the convolution kernel position coordinates. Through this method, the next position of each target to be processed is selected in turn for observation to ensure the continuity of the observation. The features of the bone image are extracted through this continuous movement, and the coherence values between the features are obtained clearly and intuitively.
[0043] This embodiment gradually accumulates the offset through the dynamic snake convolution technology to maintain the consistency of the changes in the entire set, thereby improving the recognition accuracy and stability of small targets. The use of bilinear interpolation further enhances the fluency and accuracy of the convolution process.
[0044] In some embodiments of the present invention, Figure 6 As shown, Figure 6 The present invention provides Figure 3 The flowchart of an embodiment of step S304 in the embodiment includes: S601, extracting sequence dynamic features of training sample image data based on ConvS2S; S602: Based on the learnable position coding, add the learnable position coding to the sequence dynamic feature to generate the position coding information.
[0045] Specifically, the AIFI module of the RTDETR large model controls the model's ability to process local and global information by introducing an attention mechanism. This embodiment uses a learnable relative position encoding layer to replace the traditional fixed position encoding. The relative position encoding layer is used to learn the position encoding of the bone image. According to the maximum index value that the embedding layer needs to learn and the embedding vector dimension of each position index and other parameters, the parameter layer of the feature position is determined, and the position parameter layer is mapped to the position vector. The obtained position vector is added to the sequence dynamic feature obtained by the convolution sequence, or directly combined with other parts of the sequence dynamic feature, wherein the position vector has the same dimension as the sequence dynamic feature, and finally the sequence dynamic feature position encoding information is obtained. In the initial stage of model training, the parameter layer of the position encoding layer is initialized to the minimum random value to break the parameter symmetry and provide a non-trivial starting point. As the iterative training proceeds, these parameters are continuously updated through the loss function, so that the model can learn the optimal position encoding representation.
[0046] This embodiment improves the AIFI module of the RTDETR large model and introduces a learnable position encoding layer, thereby improving the model's ability to process spatial information, enabling the model to better adapt to different tasks and data, and improving the accuracy and adaptability of model detection.
[0047] In some embodiments of the present invention, Figure 7 As shown, Figure 7The present invention provides Figure 1 The flowchart of an embodiment of step S104 in the embodiment includes: S701, build an initial fracture detection model based on the RT-DETR large model, dynamic snake convolution, learnable position encoding model and bone loss function; S702, iteratively training a fracture detection model according to the training sample image data to generate a bone prediction result; S703, based on the bone loss function, calculating the bone loss function value of the bone prediction result and the actual bone result in the training sample image data; S704: Compare the bone loss function value with a preset threshold, determine the optimal model parameters, and generate a final fracture detection model.
[0048] Specifically, the result of the fracture detection model constructed in this embodiment is shown in FIG. Figure 8 As shown, Figure 8 This is a structural diagram of the improved fracture detection model based on the RT-DETR large model provided by the present invention. Specifically, the fracture detection model integrates the dynamic snake convolution (DSConv) module in the backbone network BackBone, adopts a learnable position encoding method (LPE) to replace the sine and cosine encoding of the original model, and optimizes the minimum point distance (MPDIoU) intersection-over-union loss function by introducing a size factor ratio, thereby significantly improving the overall performance of the detection algorithm.
[0049] Furthermore, based on the IoU loss function, the ratio of the intersection area of the predicted bounding box and the actual bounding box to the union area of the predicted bounding box and the actual bounding box is calculated; the IoU loss function value is obtained according to this ratio to dynamically adjust the size of the size factor. For example, when the IoU loss function value is high, it means that the size of the predicted bounding box is significantly different from the size of the actual bounding box. At this time, the size factor needs to be expanded so that the predicted bounding box fully matches the actual bounding box; the bone loss function value calculated during the iterative training process is compared with the preset threshold to determine the optimal model parameters and its corresponding fracture detection model.
[0050] In order to better implement the fracture detection method in the embodiment of the present invention, based on the fracture detection method, correspondingly, Fig. 9 As shown, the embodiment of the present invention further provides a fracture detection device 900 including: A loss function construction module 901 is used to construct a bone loss function according to the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; The feature information module 902 is used to determine the coherence value and position coding information of the adjacent features of the bone image in the training sample image data; A skeleton prediction module 903, used to generate a skeleton prediction result according to the coherent values and position coding information of adjacent features; The fracture detection module 904 is used to train a fracture detection model according to the bone loss function, the bone prediction result and the training sample image data, and perform fracture detection on the bone image to be detected based on the trained fracture detection model.
[0051] The fracture detection device 900 provided in the above embodiment can implement the technical solution described in the above fracture detection method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above fracture detection method embodiment, which will not be repeated here.
[0052] In the embodiment of the present invention, the fracture detection device may be an independent server, or a server network or server cluster composed of servers. For example, the fracture detection device described in the embodiment of the present invention includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.
[0053] The present invention also provides a fracture detection electronic device, such as Fig.10 As shown, Fig.10 This is a structural block diagram of an embodiment of a fracture detection device provided by the present invention. The fracture detection device 1000 can be a computing device such as a mobile terminal, a desktop computer, a notebook, a PDA, and a server. The fracture detection device 1000 includes a processor 1001 and a memory 1002, wherein the memory 1002 stores a fracture detection program 1003.
[0054] In some embodiments, the memory 1002 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 1002 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory 1002 may also include both an internal storage unit of the computer device and an external storage device. The memory 1002 is used to store application software and various types of data installed in the computer device, such as program codes installed in the computer device. The memory 1002 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the fracture detection program 1003 may be executed by the processor 1001, thereby realizing the fracture detection method, device, electronic device and storage device of each embodiment of the present invention.
[0055] In some embodiments, the processor 1001 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 1002, such as executing a fracture detection program.
[0056] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0057] The fracture detection method, device, electronic device and storage device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A fracture detection method, characterized in that: include: A bone loss function is constructed according to the coordinates of the corner points of the predicted bounding box and the actual bounding box of the bone image, their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor; Determine the coherence value and position coding information of adjacent features of the bone image in the training sample image data; Generate a skeleton prediction result according to the coherent value and position encoding information of the adjacent features; A fracture detection model is trained according to the bone loss function, the bone prediction result and the training sample image data, and fracture detection is performed on the bone image to be detected based on the trained fracture detection model.
2. The fracture detection method according to claim 1, characterized in that: The bone loss function is constructed based on the corner coordinates of the predicted bounding box and the actual bounding box of the bone image, their overlapping area and non-overlapping area, the width and height of the bone image, and the size factor, including: Constructing an IoU loss function based on the overlapping area and non-overlapping area of the predicted bounding box and the actual bounding box of the skeleton image; The size factor is determined according to the MPDIoU loss function, and the bone loss function is constructed according to the IoU loss function, the size factor, the corner point coordinates of the predicted bounding box and the corner point coordinates corresponding to the actual bounding box, the height and width of the bone image, and the size factor.
3. The fracture detection method according to claim 2, characterized in that: The corner point coordinates include the upper left corner coordinates and the lower right corner coordinates, and the calculation formula of the bone loss function is: ， ， in, is the IoU loss function value, To predict the bounding box, is the actual bounding box, is the distance between the upper left corner coordinates of the predicted bounding box and the upper left corner coordinates of the actual bounding box, is the distance between the lower right corner coordinates of the predicted bounding box and the lower right corner coordinates of the actual bounding box, is the height of the bone image, is the width of the bone image, is the size factor.
4. The fracture detection method according to claim 1, characterized in that: Determine the coherent values and position encoding information of adjacent features of the bone image in the training sample image data, including: Obtain bone images; Performing offline data enhancement processing on the bone image to generate training sample image data; Extracting feature information of the training sample image data and determining the coherence value of adjacent features; Position encoding is performed on the feature information to generate position encoding information.
5. The fracture detection method according to claim 4, characterized in that: Extracting feature information of the training sample image data and determining the coherence value of adjacent features includes: Extracting feature information of the training sample image data based on dynamic snake convolution; Based on bilinear interpolation, the deformation offsets between adjacent features are calculated in sequence to determine the coherence values of adjacent features.
6. The fracture detection method according to claim 5, characterized in that: Position encoding the feature information to generate position encoding information includes: Based on the ConvS2S model, extracting the sequence dynamic features of the training sample image data; Based on a learnable position coding model, learnable position coding is added to the sequence dynamic features to generate position coding information.
7. The fracture detection method according to claim 1, characterized in that: According to the bone loss function, the bone prediction result and the training sample image data, a fracture detection model is trained, including: An initial fracture detection model is constructed based on the RT-DETR large model, dynamic snake convolution, learnable position encoding model, and bone loss function; Iteratively train the fracture detection model according to the training sample image data to generate a bone prediction result; Based on the bone loss function, a bone loss function value between a bone prediction result and an actual bone result in the training sample image data is calculated; The bone loss function value is compared with a preset threshold value to determine the optimal model parameters and generate a final fracture detection model.
8. A fracture detection device, characterized in that: include: A loss function construction module is used to construct a bone loss function according to the corner coordinates of the predicted bounding box and the actual bounding box of the bone image and their overlapping area and non-overlapping area, the width and height of the bone image, and a size factor; A feature information module, used to determine the coherence value and position coding information of adjacent features of the bone image; A skeleton prediction module, used for generating a skeleton prediction result according to the coherent value and position coding information of the adjacent features; The fracture detection module is used to calculate the bone loss function value between the bone prediction result and the actual bone image based on the bone loss function, and perform fracture detection according to the bone loss function value.
9. An electronic device for detecting fractures, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the fracture detection method according to any one of claims 1 to 7 are implemented.
10. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are executed by a computer, the computer is caused to execute the bone fracture detection method according to any one of claims 1 to 7.