Deep learning-based acetabular fracture classification prediction method and system

Abstract

The present disclosure provides a deep learning-based acetabular fracture classification prediction method and system, belonging to the technical field of deep learning. The method comprises: processing first data based on the basic information of the target person to obtain first target data, and processing second data to obtain second target data; wherein the first data is X-ray detection data of the acetabular position of the target person, and the second data is CT three-dimensional reconstruction data of the acetabular position of the target person; identifying the first target data and the second target data based on a deep learning model to obtain acetabular fracture classification features; and sending the acetabular fracture classification features to a first device, which is used to display the acetabular fracture classification features. The deep learning-based acetabular fracture classification prediction method and system provided by the present disclosure can improve the accuracy of acetabular fracture feature recognition and provide auxiliary information for doctors.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of deep learning, more specifically, it relates to a deep learning-based acetabular fracture classification prediction method and system. BACKGROUND

[0002] Acetabular fracture is a serious hip injury, often caused by high-energy trauma, such as traffic accidents, falls from a great height, etc.

[0003] Traditional fracture classification methods, such as Letournel-Judet classification and AO classification, mainly rely on doctors to manually identify X-ray, CT and other images. However, due to the complex anatomical structure of the acetabulum and the overlapping shadows, the traditional method of relying on doctors to manually interpret images for acetabular fracture classification is prone to inaccurate judgment of fracture lines, fracture locations, etc. in the image.

[0004] Therefore, a method is needed to assist doctors in judging the type of acetabular fracture. SUMMARY

[0005] The purpose of the present disclosure is to provide a deep learning-based acetabular fracture classification prediction method and system to improve the accuracy of acetabular fracture feature recognition and provide auxiliary information for doctors.

[0006] The first aspect of the embodiment of the present disclosure provides a deep learning-based acetabular fracture classification prediction method, comprising: processing first data based on the basic information of a target person to obtain first target data, and processing second data to obtain second target data; wherein the first data is X-ray detection data of the acetabular position of the target person, and the second data is CT three-dimensional reconstruction data of the acetabular position of the target person;

[0007] Based on a deep learning model, the first target data and the second target data are identified to obtain acetabular fracture classification features;

[0008] The acetabular fracture classification features are sent to a first device, and the first device is used to display the acetabular fracture classification features.

[0009] The second aspect of the embodiment of the present disclosure provides a deep learning-based acetabular fracture classification prediction system, comprising:

[0010] A preprocessing module is configured to process first data based on the basic information of a target person to obtain first target data, and process second data to obtain second target data; wherein the first data is X-ray detection data of the acetabular position of the target person, and the second data is CT three-dimensional reconstruction data of the acetabular position of the target person;

[0011] The identification module is configured to identify the first target data and the second target data based on a deep learning model to obtain a hip fracture classification feature.

[0012] The central control module is configured to send the hip fracture classification feature to a first device, and the first device is configured to display the hip fracture classification feature.

[0013] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the deep learning-based hip fracture classification prediction method when executing the computer program.

[0014] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the deep learning-based hip fracture classification prediction method when executed by a processor.

[0015] The deep learning-based hip fracture classification prediction method and system provided by the embodiments of the present disclosure have the following beneficial effects:

[0016] The present disclosure can more accurately identify the characteristics of the hip fracture, such as the position, direction, and length of the fracture line, by preprocessing the X-ray detection data and the CT three-dimensional reconstruction data based on the basic information of the person and comprehensively analyzing the X-ray detection data and the CT three-dimensional reconstruction data based on the deep learning model. Compared with the traditional manual interpretation method, the accuracy of judging the fracture line and the fracture position in the patient's examination image is improved, and the time spent by doctors in image observation is shortened, thereby improving the medical efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0018] Figure 1 A flowchart of the deep learning-based hip fracture classification prediction method provided by an embodiment of the present disclosure is shown in the figure.

[0019] Figure 2 A flowchart of early fusion and late fusion in the deep learning-based hip fracture classification prediction method provided by an embodiment of the present disclosure is shown in the figure.

[0020] Figure 3 A structure block diagram of the deep learning-based hip fracture classification prediction system provided by an embodiment of the present disclosure is shown in the figure.

[0021] Figure 4 A schematic block diagram of an electronic device according to an embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0022] In the following description, specific details are set forth such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the present embodiments of the present disclosure. However, persons skilled in the art will understand that the present disclosure can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and methods have not been described in detail in order to avoid obscuring the present disclosure.

[0023] X-ray or X-light referred to in the present disclosure refers to Rontgen rays, and CT refers to Computed Tomography, i.e., electronic computed tomography. CT is essentially a scan using X-rays, and the difference is that CT is a slice type X-ray irradiation of the human body from multiple angles to obtain a large amount of cross-sectional image data, and a computer processes and reconstructs the data to finally form a three-dimensional image or a tomographic image of an arbitrary plane. The principle will not be described again.

[0024] In order to make the objects, technical solutions and advantages of the present disclosure clearer, the following will be described by specific embodiments in conjunction with the accompanying drawings.

[0025] Reference is made to Figure 1 , Figure 1 A flowchart of a deep learning-based acetabular fracture classification prediction method according to an embodiment of the present disclosure is provided. The method comprises:

[0026] S101: processing first data based on basic information of a target person to obtain first target data, and processing second data to obtain second target data; wherein the first data is X-ray detection data of the acetabular position of the target person, and the second data is CT three-dimensional reconstruction data of the acetabular position of the target person.

[0027] In the present embodiment, the target person refers to a patient receiving acetabular fracture diagnosis, and the basic information can be the age, body mass index (Body Mass Index, BMI), gender, etc. of the target person.

[0028] The first target data is X-ray detection data of the acetabular position of the target person after noise reduction processing, which can be an X-ray image after noise reduction processing, for example. The second target data is CT three-dimensional reconstruction data of the acetabular position of the target person after noise reduction processing, which can be a CT three-dimensional image after noise reduction processing, for example.

[0029] The data source of the first data is an X-ray machine, and the data source of the second data is a CT machine.

[0030] In this embodiment, considering that the X-ray detection data and the CT three-dimensional reconstruction data of personnel in different physical conditions are different, different processing methods should be taken for personnel in different physical conditions.

[0031] For example, for a child or a teenager detection personnel, the corresponding radiation dose in the current medical scene will be correspondingly reduced, which inevitably reduces part of the image quality, resulting in part of the noise in the detection result. For doctors, they can make subjective judgment to exclude the influence of part of the noise according to experience to obtain the fracture classification result. However, for computers, the collected data should be processed, such as noise reduction, so that whether the fracture and the fracture classification related features appear in the data can be accurately recognized.

[0032] S102: identifying the first target data and the second target data based on a deep learning model to obtain acetabular fracture classification features.

[0033] In this embodiment, after the foregoing noise reduction processing, the first target data and the second target data can be input into the deep learning model. The deep learning model can be one or multiple, but it should be noted that whether the deep learning model is one or multiple, it is obtained by training the corresponding training set.

[0034] For example, the deep learning model has only one, when the input is the first target data and the second target data, the training set should also include the X-ray detection data and the CT three-dimensional reconstruction data, and the output is the acetabular fracture classification features corresponding to the above two kinds of data.

[0035] In this embodiment, the acetabular fracture classification features refer to data related to the doctor's judgment of the acetabular fracture classification result, for example, the direction, length, width or position of the fracture line, and the size or shape of the fracture block, etc. The specific acetabular fracture classification features, their corresponding descriptions and the role of assisting doctors in acetabular fracture classification are shown in Table 1.

[0036] Table 1: Acetabular fracture classification features and corresponding descriptions

[0037]

[0038] S103: sending the acetabular fracture classification features to the first device, and the first device is used to display the acetabular fracture classification features.

[0039] In the embodiment, the first device is a terminal device receiving the acetabular fracture classification features, and the specific type can be various, such as a computer in a hospital imaging department, a mobile terminal of a doctor, such as a computer, a smart phone, etc. The first device has a display function and can display the received acetabular fracture classification feature data.

[0040] In the embodiment, the X-ray detection data is taken as an example, and the specific display form can be marking the fracture line in the X-ray detection data (X-ray image), and marking the length, width or suspected fracture line, etc. The acetabular fracture classification features referred to in the disclosure are extracted based on the X-ray data and CT three-dimensional reconstruction data of the target person, and are used to assist the doctor in judging whether the acetabular fracture occurs and providing the related features for judging the acetabular fracture classification.

[0041] From the above, it can be concluded that the disclosure can more accurately identify the features of the acetabular fracture, such as the position, direction and length of the fracture line, by preprocessing the X-ray detection data and the CT three-dimensional reconstruction data based on the basic information of the person and comprehensively analyzing the X-ray detection data and the CT three-dimensional reconstruction data based on the deep learning model. Compared with the traditional manual interpretation method, the accuracy of judging the fracture line and the fracture position in the patient's examination image is improved, and the time spent by the doctor in image observation is shortened, thereby improving the medical efficiency.

[0042] In an embodiment of the disclosure, the basic information of the target person includes: age and body mass index;

[0043] The second data is processed to obtain second target data, including:

[0044] In response to the age of the target person being less than a preset age and the body mass index being less than or equal to a preset index, the second data is processed based on a first noise reduction mode to obtain second target data;

[0045] In response to the body mass index of the target person being greater than a preset index and the age being greater than or equal to a preset age, the second data is processed based on a second noise reduction mode to obtain second target data;

[0046] In response to the age of the target person being less than a preset age and the body mass index being greater than a preset index, the second data is processed based on a third noise reduction mode to obtain second target data;

[0047] Among them, the noise reduction modes of the first noise reduction mode, the second noise reduction mode and the third noise reduction mode are different.

[0048] In this embodiment, different noise reduction modes are set for different basic information of the personnel. The preset age in this embodiment refers to a specified age limit, which is determined based on prior knowledge in the medical field, and can be set to 12-14 years old, or 8-9 years old, etc. The preset index in this embodiment refers to a BMI index that will affect the X-ray detection data or CT three-dimensional reconstruction data, which can be set to 28.

[0049] When the age of the target personnel is less than the preset age, the corresponding radiation dose will be reduced when the doctor performs the acetabular fracture examination, and correspondingly, there will be a certain impact on the image level, such as noise. When the body mass index of the target personnel is greater than the preset index, it indicates that the target personnel is obese, and there may be noise or artifacts in the X-ray detection data and CT three-dimensional reconstruction data.

[0050] In this embodiment, the first noise reduction mode can be adaptive median filtering, or wavelet transform-based noise reduction, etc. Taking wavelet transform-based noise reduction as an example, wavelet transform can decompose the image into different scales and directions. For the noise generated by low-dose scanning, this method can remove noise interference while preserving the high-frequency details of the image (such as bone texture), which is suitable for the case where the bone features are obvious but the image quality is affected by the radiation dose.

[0051] The second noise reduction mode can be non-local mean filtering. Non-local mean filtering itself can remove noise by utilizing the self-similarity of the image. In the filtering process, the pixel information conforming to the bone features is preserved, and the noise and artifacts are better removed, and the true features of the bone are preserved.

[0052] The third noise reduction mode can be a combination of the two aforementioned noise reduction modes, i.e., a combination of adaptive median filtering and non-local mean filtering, or a combination of wavelet transform-based noise reduction and non-local mean filtering. The order of the two noise reduction modes is not limited in this disclosure.

[0053] In a preferred embodiment of the present disclosure, the third noise reduction mode first performs adaptive median filtering or wavelet transform-based noise reduction in the time dimension, and then performs non-local mean filtering. Taking wavelet transform-based noise reduction as an example, the wavelet transform-based noise reduction method can decompose the CT image into different scales and directions. By processing the wavelet coefficients, the noise can be removed while preserving the high-frequency details of the image (such as acetabular bone texture). The present disclosure first performs this kind of noise reduction, which can effectively remove the noise generated by low-dose scanning and preserve important features of the image. Then, when non-local mean filtering is performed, the noise level of the image has been reduced, and the bone features are well preserved. Non-local mean filtering can further utilize the self-similarity of the image to smooth the image, improve the overall quality of the image, and will not cause excessive damage to the previously preserved high-frequency details.

[0054] If the non-local mean filtering is performed first, the image is globally smoothed, and some high-frequency details of the image are blurred in the process. When the wavelet transform-based noise reduction is performed later, although the image can be further refined, the high-frequency information has been lost to some extent due to the previous non-local mean filtering, and the original high-frequency details of the CT image cannot be completely restored, resulting in that the final image is not as good as the case where the wavelet transform-based noise reduction is performed first in terms of retaining the details such as bone texture.

[0055] In this embodiment, only the second data, i.e., the CT three-dimensional reconstruction data of the acetabulum position of the target person, is subjected to noise reduction processing. Meanwhile, as known from the foregoing, CT is essentially a scanning process using X-rays, and therefore it should be understood that the noise reduction mode and principle of the first data and the second data are consistent, i.e., the first data is processed to obtain the first target data, including:

[0056] In response to the age of the target person being less than the preset age and the body mass index being less than or equal to the preset index, the first data is processed based on the first noise reduction mode to obtain the first target data;

[0057] In response to the body mass index of the target person being greater than the preset index and the age being greater than or equal to the preset age, the first data is processed based on the second noise reduction mode to obtain the first target data;

[0058] In response to the age of the target person being less than the preset age and the body mass index being greater than the preset index, the first data is processed based on the third noise reduction mode to obtain the first target data;

[0059] The first noise reduction mode, the second noise reduction mode, and the third noise reduction mode are different.

[0060] When the age of the target person is greater than or equal to the preset age and the body mass index is less than or equal to the preset index, the radiation dose is within the normal range, and it will not affect the X-ray detection data and the CT three-dimensional reconstruction data, and therefore the noise reduction processing can not be performed.

[0061] That is, in response to the age of the target person being greater than or equal to the preset age and the body mass index being less than or equal to the preset index, the first data is taken as the first target data, and the second data is taken as the target data.

[0062] In this embodiment, the process and principle of the noise reduction processing of the first data are not described again.

[0063] From the above, the present disclosure selects a suitable noise reduction mode according to the age and body mass index of the target person, helps to optimize the image quality and reduce noise and artifacts, and thus improves the recognition accuracy of the subsequent deep learning model on the acetabular fracture features.

[0064] In an embodiment of the present disclosure, the first target data and the second target data are identified based on a deep learning model to obtain acetabular fracture classification features, including:

[0065] The target fusion period is determined based on the similarity of the first target data and the second target data in multiple dimensions, and the target fusion period is the fusion period of the first target data and the second target data.

[0066] The first target data and the second target data are identified based on the deep learning model corresponding to the target fusion period to obtain acetabular fracture classification features.

[0067] In the present embodiment, the target fusion period includes early fusion and late fusion, and the target fusion period is determined based on the similarity of the first target data and the second target data in multiple dimensions. The nature of the first target data and the second target data is image, so the dimensions can include image quality, resolution, data distribution, etc. The data distribution refers to the distribution of the aforementioned acetabular fracture classification features on the image, which can be reflected in the number or position distribution.

[0068] In the present embodiment, the target fusion period can be determined in the following manner:

[0069] The target fusion period is determined based on the similarity of the first target data and the second target data in multiple dimensions, including:

[0070] In response to the similarity of the first target data and the second target data in multiple dimensions being greater than the target similarity, the early fusion is taken as the target fusion period.

[0071] In response to the similarity of any dimension of the first target data and the second target data being less than the target similarity, the late fusion is taken as the target fusion period.

[0072] In the present embodiment, the similarity calculation method of each dimension can be different. For example, for the image quality dimension, the mean square error of the two images can be calculated to obtain the similarity. For the resolution dimension, the difference between the resolutions of the two images can be determined, for example, one of the resolutions is taken as the denominator, and the greater the result, the smaller the similarity. The similarity can be obtained according to the mapping table relationship. The similarity of the data distribution dimension can be compared by histogram, for example, the Bhattacharyya distance or the Chi-square distance can be calculated to obtain the similarity.

[0073] It should be noted that the similarity of each dimension described above should be set to different target similarities according to its calculation method, or the similarity of each dimension described above is normalized, at which time a target similarity can be corresponded.

[0074] If the X-ray and CT three-dimensional reconstruction data have high similarity in image quality, resolution, data distribution, etc., the early fusion of the first target data and the second target data is more appropriate. Because at this time, after directly fusing the two kinds of data, the deep learning model is easier to learn the unified feature representation.

[0075] For example, when the basic parameters such as the gray scale range and pixel spacing of the X-ray image and the CT image are similar, early fusion can make full use of these similarities, while extracting features from both kinds of data and mining the complementary information between them.

[0076] On the contrary, if the two kinds of data differ greatly in these aspects, late fusion is more advantageous. Because late fusion can first preprocess and extract features from data of different modalities, and adopt methods suitable for the characteristics of each kind of data, it can better handle the differences between data and avoid the difficulty in training the deep learning model due to data differences in early fusion.

[0077] From the above, it can be concluded that the disclosure determines the target fusion period based on the similarity of the first target data (X-ray image) and the second target data (CT three-dimensional reconstruction data) in multiple dimensions, so that the disclosure can dynamically select the optimal fusion strategy, make full use of the complementary information between the two kinds of data, avoid the negative effects of data differences, and thus improve the recognition performance of the deep learning model for the acetabular fracture classification features.

[0078] Figure 2 An early fusion and late fusion process schematic diagram is provided in the deep learning-based acetabular fracture classification prediction method of an embodiment of the disclosure. Referring to Figure 2 In an embodiment of the disclosure, the target fusion period includes early fusion;

[0079] The deep learning model corresponding to the target fusion period is used to identify the first target data and the second target data to obtain the acetabular fracture classification features, including:

[0080] In response to the target fusion period being early fusion, the first target data and the second target data are fused to obtain third target data;

[0081] The third target data is identified based on the first deep learning model to obtain the acetabular fracture classification features.

[0082] In the embodiment, when it is judged that the target fusion period is the early fusion, it is explained that the first target data and the second target data have high similarity, and they are directly fused into the third target data, so that the first deep learning model can comprehensively utilize the information of the two kinds of data from the beginning. The advantage of this is that the model can learn features in a unified data space, mine the complementary information between the two kinds of data, avoid the information fragmentation problem caused by processing the data separately, and thus more effectively learn the features related to the acetabular fracture classification.

[0083] It should be noted that, since in the early fusion, the first target data is essentially two-dimensional image data, and the second target data is essentially three-dimensional image data, the three-dimensional image data can be converted into two-dimensional image for fusion, or the two-dimensional X-ray image can be converted into three-dimensional voxel representation, for example, adding a dimension to the two-dimensional image and regarding it as a three-dimensional voxel data with a thickness of 1, and then the fusion can be performed in the voxel space with the CT three-dimensional data. In the fusion process, the voxel values can be weighted and summed or other fusion operations can be performed to integrate the information of the two kinds of data.

[0084] The training set of the first deep learning model is a data group composed of a plurality of X-ray detection data and corresponding CT detection data, and a plurality of acetabular fracture classification features corresponding to the data group.

[0085] Reference Figure 2 In another embodiment of the present disclosure, in addition to the aforementioned early fusion, there is also late fusion, and the data processing manner of the deep learning model corresponding to the late fusion is different from the data processing manner of the deep learning model corresponding to the early fusion, that is, the target fusion period includes late fusion.

[0086] Based on the deep learning model corresponding to the target fusion period, the first target data and the second target data are identified to obtain the acetabular fracture classification feature, comprising:

[0087] In response to the target fusion period being late fusion, the first target data is identified based on the second deep learning model to obtain the X-ray classification feature;

[0088] The second target data is identified based on the third deep learning model to obtain the CT classification feature;

[0089] The X-ray classification feature and the CT classification feature are weighted and fused to obtain the comprehensive classification feature;

[0090] The comprehensive classification feature is classified based on the classifier to obtain the acetabular fracture classification feature;

[0091] The training set of the second deep learning model is different from the training set of the third deep learning model.

[0092] In the present embodiment, if the target fusion period is the late fusion, it indicates that the first target data and the second target data are quite different. At this time, the first target data and the second target data are processed by the second deep learning model and the third deep learning model respectively, because different types of data may need different processing methods and feature extraction methods.

[0093] The X-ray image and the CT three-dimensional reconstruction image have great differences in imaging principle, data structure and information characteristics. Different models are trained using different training sets, so that each deep learning model can better adapt to the characteristics of the corresponding data and more accurately extract the acetabular fracture classification features. The training set of the second deep learning model is composed of multiple X-ray detection data and the corresponding acetabular fracture classification features, and the training set of the third deep learning model is composed of multiple CT detection data and the corresponding acetabular fracture classification features.

[0094] After obtaining the X-ray classification features and the CT classification features, they are fused to obtain comprehensive classification features. Different types of data have different contributions to the acetabular fracture classification. Through weighting, the two types of feature information can be more reasonably integrated. Finally, the comprehensive classification features are classified using a classifier, because the comprehensive classification features are only a feature vector containing multiple information, which needs to be output through a classifier, because the features output by the deep learning model are an abstract representation of the acetabular fracture image, which only describes various attributes and characteristics of the image. For example, the fracture line position, fracture line length, displacement and other features mentioned above cannot directly determine the type of fracture or give a clinical diagnosis result, so a classifier is needed to map them to visual data that can be understood by doctors. The role of the classifier is to convert abstract feature data into visual data that can be understood by doctors.

[0095] In the early fusion process described above, the final result output also needs to use a classifier, and the classifier itself is a component of the deep learning model, so during the training process, the classifier can learn how to output the acetabular fracture classification features, which will not be described in detail in the present disclosure. The deep learning model referred to in the present disclosure can be a convolutional neural network or a recurrent neural network.

[0096] From the above, in the early fusion, the present disclosure can utilize the information of the two types of data comprehensively from the beginning by directly fusing the first target data (X-ray image) and the second target data (CT three-dimensional reconstruction image), which helps the model to learn more comprehensive feature representation and thus improves the accuracy of acetabular fracture classification. The late fusion strategy is aimed at the case where the data difference is large. By processing the X-ray image and the CT three-dimensional reconstruction image respectively and using different deep learning models for feature extraction, it is ensured that the model can adapt to different types of data and enhance the generalization ability of the present disclosure.

[0097] In an embodiment of the present disclosure, the deep learning-based acetabular fracture classification prediction method further comprises: in response to the number of fracture line features in the X-ray classification features and / or the CT classification features being greater than a first number, reducing a reference value of the X-ray classification weight based on a first step, to obtain the X-ray classification weight;

[0098] in response to the number of fracture line features in the X-ray classification features and the CT classification features both being less than a second number, increasing the reference value of the X-ray classification weight based on a second step, to obtain the X-ray classification weight;

[0099] wherein the first number is greater than the second number, and the X-ray classification weight is the weight corresponding to the X-ray classification features. The first number and the second number can be determined based on actual conditions or experience.

[0100] In this embodiment, it is considered that when the acetabular fracture is relatively complex, a single X-ray image may be difficult to fully and accurately display the details and spatial structure of the fracture. At this time, the CT three-dimensional reconstruction data can provide more detailed fracture information, including the size, shape, position of the fracture fragments and their spatial relationship, etc. Therefore, for complex fractures, the weight of the CT classification features in the late fusion can be appropriately increased.

[0101] For relatively simple acetabular fractures, such as simple linear fractures or fractures with fewer fracture fragments and no obvious displacement, an X-ray image may be able to provide sufficient information for accurate classification. In this case, the reliability of the X-ray classification features is high, and their weight in the late fusion can be increased accordingly.

[0102] Therefore, when the number of fracture line features in the X-ray classification features and / or the CT classification features is greater than the first number, it indicates that the fracture is relatively complex, and when the number of fracture line features in the X-ray classification features and / or the CT classification features is less than the second number, it indicates that the fracture is relatively simple. The adjustment can be made according to the logic described above.

[0103] The formula of the first step can be and the formula of the second step can be wherein, represents the first step, represents the second step, represents a first proportion coefficient, represents a second proportion coefficient, represents a natural constant, represents the number of fracture line features in the X-ray classification features and the CT classification features, represents the first number, represents the second quantity. The first proportionality coefficient and the second proportionality coefficient can be preset according to experience.

[0104] The reference value of the X-ray typing weight is determined based on an experiment or defaults to 0.5. The sum of the X-ray typing weight and the CT typing weight is 1, and the CT typing weight is the weight corresponding to the CT typing feature, that is, when the reference value of the X-ray typing weight is adjusted, the reference value of the CT typing weight should also be adjusted in the opposite direction with the same step.

[0105] In response to the number of the fracture line feature in the X-ray typing feature and the CT typing feature being less than or equal to the first quantity and greater than or equal to the second quantity, the reference value of the X-ray typing weight is taken as the X-ray typing weight, and the reference value of the CT typing weight is taken as the CT typing weight. At this time, the number of the fracture line is in the moderate interval, and the reference values of the two weights can not be adjusted.

[0106] From the above, it can be concluded that the disclosure adjusts the weights of the two typing features according to the number of the fracture line feature, thereby ensuring more accurate typing prediction results in complex and simple fracture cases. In the case of complex fracture, the disclosure increases the weight of the CT typing feature to obtain more detailed fracture information; in the case of simple fracture, the weight of the X-ray typing feature is increased to take full advantage of its high reliability, which helps to improve the accuracy and reliability of the extraction of the acetabular fracture analysis feature.

[0107] The acetabular fracture typing prediction method based on deep learning corresponding to the above embodiment, Figure 3 The structure block diagram of the acetabular fracture typing prediction system based on deep learning provided by an embodiment of the disclosure is shown. For ease of illustration, only parts related to the embodiments of the disclosure are shown. For parts not related to the embodiments of the disclosure, reference Figure 3 The acetabular fracture typing prediction system based on deep learning 20 includes a preprocessing module 21, an identification module 22, and a central control module 23.

[0108] The preprocessing module 21 is configured to process the first data based on the basic information of the target person to obtain first target data, and process the second data to obtain second target data; the first data is X-ray detection data of the acetabular position of the target person, and the second data is CT three-dimensional reconstruction data of the acetabular position of the target person.

[0109] The identification module 22 is configured to identify the first target data and the second target data based on a deep learning model to obtain acetabular fracture typing features.

[0110] The central control module 23 is configured to send the acetabular fracture typing features to a first device, and the first device is configured to display the acetabular fracture typing features.

[0111] In an embodiment of the present disclosure, the basic information includes age and body mass index;

[0112] The preprocessing module 21 is specifically configured to, in response to the age of the target person being less than a preset age and the body mass index being less than or equal to a preset index, process the second data based on a first noise reduction manner to obtain second target data;

[0113] In response to the body mass index of the target person being greater than the preset index and the age being greater than or equal to the preset age, the second data is processed based on a second noise reduction manner to obtain the second target data;

[0114] In response to the age of the target person being less than the preset age and the body mass index being greater than the preset index, the second data is processed based on a third noise reduction manner to obtain the second target data;

[0115] The first noise reduction manner, the second noise reduction manner and the third noise reduction manner are different in noise reduction manner.

[0116] In an embodiment of the present disclosure, the identification module 22 is specifically configured to determine a target fusion period based on the similarity of the first target data and the second target data in multiple dimensions, the target fusion period being a fusion period of the first target data and the second target data;

[0117] The first target data and the second target data are identified based on a deep learning model corresponding to the target fusion period to obtain the acetabular fracture classification feature.

[0118] In an embodiment of the present disclosure, the identification module 22 is specifically further configured to, in response to the similarity of the first target data and the second target data in multiple dimensions being greater than a target similarity, take early fusion as the target fusion period;

[0119] In response to the similarity of any dimension of the first target data and the second target data in multiple dimensions being less than the target similarity, take late fusion as the target fusion period.

[0120] In an embodiment of the present disclosure, the target fusion period includes early fusion;

[0121] The identification module 22 is specifically further configured to, in response to the target fusion period being early fusion, fuse the first target data and the second target data to obtain third target data;

[0122] The third target data is identified based on a first deep learning model to obtain the acetabular fracture classification feature.

[0123] In an embodiment of the present disclosure, the target fusion period includes late fusion;

[0124] The identification module 22 is further used to identify the first target data based on the second deep learning model in response to the late fusion of the target fusion period, and obtain X-ray typing features;

[0125] The second target data is identified based on the third deep learning model to obtain CT classification features;

[0126] The comprehensive classification features are obtained by weighted fusion of X-ray classification features and CT classification features;

[0127] Based on the classifier, the comprehensive classification features are classified to obtain the classification features of acetabular fractures;

[0128] The training set for the second deep learning model is different from that for the third deep learning model.

[0129] In one embodiment of this disclosure, the deep learning-based acetabular fracture classification prediction system 20 further includes: a weight adjustment module, used to obtain X-ray classification weights by reducing a reference value of the X-ray classification weights based on a first step length in response to the number of fracture line features in the X-ray classification features and / or CT classification features being greater than a first number.

[0130] Since the number of fracture line features in both X-ray and CT classification features is less than the second number, the reference value of the X-ray classification weight is increased based on the second step size to obtain the X-ray classification weight.

[0131] Among them, the first quantity is greater than the second quantity, and the X-ray typing weight is the weight corresponding to the X-ray typing feature.

[0132] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of the preprocessing module 21, the identification module 22, and the central control module 23 are shown.

[0133] It should be appreciated that in the embodiments of the present disclosure, the processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0134] The input device 302 can include a touchpad, a fingerprint collection sensor (for collecting fingerprint information and direction information of a fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0135] The memory 304 can include a read-only memory and a random access memory, and provide instructions and data for the processor 301. A portion of the memory 304 can also include a non-volatile random access memory. For example, the memory 304 can also store device type information.

[0136] In specific implementations, the processor 301, the input device 302, and the output device 303 described in the embodiments of the present disclosure can execute the implementation manners described in the first and second embodiments of the method for predicting acetabular fracture classification based on deep learning provided by the embodiments of the present disclosure, and can also execute the implementation manners of the electronic device described in the embodiments of the present disclosure, which will not be described here.

[0137] In another embodiment of the present disclosure, a computer readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also instruct related hardware to complete the above-mentioned processes. The computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0138] The computer readable storage medium can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0139] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here.

[0141] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, or can be in electrical, mechanical or other forms.

[0142] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present disclosure.

[0143] In addition, each functional unit in the various embodiments of the present disclosure can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0144] The above is merely specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present disclosure, and these modifications or replacements should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A deep learning-based acetabular fracture classification prediction method, characterized in that, The method comprises: processing first data based on basic information of a target person to obtain first target data, and processing second data to obtain second target data, wherein the first data is X-ray detection data of a position of an acetabulum of the target person, and the second data is CT three-dimensional reconstruction data of the position of the acetabulum of the target person; in response to similarities of the first target data and the second target data in multiple dimensions being greater than a target similarity, determining early fusion as the target fusion period; in response to a similarity of the first target data and the second target data in any dimension of the multiple dimensions being less than the target similarity, determining late fusion as the target fusion period; the target fusion period is a fusion period of the first target data and the second target data; identifying the first target data and the second target data based on a deep learning model corresponding to the target fusion period to obtain an acetabular fracture classification feature; sending the acetabular fracture classification feature to a first device, the first device being configured to display the acetabular fracture classification feature; the identifying the first target data and the second target data based on the deep learning model corresponding to the target fusion period to obtain the acetabular fracture classification feature comprises: in response to the target fusion period being early fusion, fusing the first target data and the second target data to obtain third target data; identifying the third target data based on a first deep learning model to obtain an acetabular fracture classification feature; in response to the target fusion period being late fusion, identifying the first target data based on a second deep learning model to obtain an X-ray classification feature; identifying the second target data based on a third deep learning model to obtain a CT classification feature; performing weighted fusion on the X-ray classification feature and the CT classification feature to obtain a comprehensive classification feature; classifying the comprehensive classification feature based on a classifier to obtain an acetabular fracture classification feature; wherein a training set of the second deep learning model is different from a training set of the third deep learning model; in response to a number of fracture line features in the X-ray classification feature and / or the CT classification feature being greater than a first number, reducing a reference value of an X-ray classification weight based on a first step to obtain the X-ray classification weight; in response to the number of fracture line features in the X-ray classification feature and the CT classification feature both being less than a second number, increasing the reference value of the X-ray classification weight based on a second step to obtain the X-ray classification weight; wherein the first number is greater than the second number, and the X-ray classification weight is a weight corresponding to the X-ray classification feature. 2.The deep learning-based acetabular fracture classification prediction method of claim 1, wherein, The basic information comprises age and body mass index. The processing the second data to obtain the second target data comprises: in response to the age of the target person being less than a preset age and the body mass index being less than or equal to a preset index, processing the second data based on a first noise reduction manner to obtain the second target data. In response to the body mass index of the target person being greater than a preset index and the age being greater than or equal to a preset age, the second data is processed based on a second noise reduction mode to obtain second target data; In response to the age of the target person being less than a preset age and the body mass index being greater than a preset index, the second data is processed based on a third noise reduction mode to obtain second target data; The first noise reduction mode, the second noise reduction mode and the third noise reduction mode are different.

3. A deep learning-based acetabular fracture classification prediction system, characterized by, Comprise: The preprocessing module is configured to process first data based on basic information of a target person to obtain first target data, and process second data to obtain second target data; the first data is X-ray detection data of a target person's acetabulum position, and the second data is CT three-dimensional reconstruction data of the target person's acetabulum position; The identification module is configured to, in response to the similarity of the first target data and the second target data in multiple dimensions being greater than a target similarity, determine early fusion as the target fusion period; In response to the similarity of any dimension of the first target data and the second target data being less than the target similarity, determine late fusion as the target fusion period; the target fusion period is a fusion period of the first target data and the second target data; The identification module is configured to, in response to the target fusion period being early fusion, fuse the first target data and the second target data to obtain third target data; and identify the third target data based on a first deep learning model to obtain an acetabular fracture classification feature In response to the target fusion period being late fusion, identify the first target data based on a second deep learning model to obtain an X-ray classification feature; identify the second target data based on a third deep learning model to obtain a CT classification feature; weight fuse the X-ray classification feature and the CT classification feature to obtain a comprehensive classification feature; and classify the comprehensive classification feature based on a classifier to obtain an acetabular fracture classification feature; the training set of the second deep learning model is different from the training set of the third deep learning model; The weight adjustment module is configured to, in response to the number of fracture line features in the X-ray classification feature and / or the CT classification feature being greater than a first number, reduce a reference value of an X-ray classification weight based on a first step to obtain an X-ray classification weight; In response to the number of fracture line features in the X-ray classification feature and the CT classification feature being less than a second number, increase the reference value of the X-ray classification weight based on a second step to obtain the X-ray classification weight; The first number is greater than the second number, and the X-ray classification weight is a weight corresponding to the X-ray classification feature. ​ ​ 4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 4. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 2.

Images