Orthopedic disease auxiliary detection method, device, equipment and medium

Abstract

The present invention relates to the field of artificial intelligence technology, and provides an orthopedic disease auxiliary detection method, device, equipment and medium. On the one hand, an image feature learning module in an orthopedic disease auxiliary detection model is used to extract features of a to-be-processed image to obtain image features, so as to learn multi-level features of the to-be-processed image; on the other hand, a feature enhancement module in the orthopedic disease auxiliary detection model is used to enhance the learned image features to obtain enhanced features, so as to aggregate features of different granularities based on the importance of features at each level, thereby improving the accuracy of subsequent predictions; on the other hand, the enhanced features are input into a prediction module in the orthopedic disease auxiliary detection model to predict the type and location of the orthopedic disease in the to-be-processed image, and an orthopedic disease auxiliary detection report is generated to be sent to the triggerer of the orthopedic disease auxiliary detection instruction, thereby assisting relevant medical personnel in quickly reading films, thereby improving the diagnostic efficiency while ensuring the diagnostic accuracy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an orthopedic disease auxiliary detection method, device, equipment and medium. Background Art

[0002] At present, there are many types of orthopedic diseases, such as fractures, osteophytes, arthritis, osteoporosis and other types. These diseases may occur in the limbs, spine, hips and other parts of the human body. Especially for orthopedic diseases such as minor fractures and osteophytes that are not easy to detect, diagnosing orthopedic diseases through bone images usually requires in-depth analysis by experienced doctors, which is time-consuming and laborious.

[0003] Therefore, how to assist doctors in effectively diagnosing orthopedic diseases has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above, it is necessary to provide an auxiliary detection method, device, equipment and medium for orthopedic diseases, aiming to solve the problem of time-consuming and labor-intensive diagnosis of orthopedic diseases.

[0005] An orthopedic disease auxiliary detection method, the orthopedic disease auxiliary detection method comprising:

[0006] In response to an orthopedic disease auxiliary detection instruction, parsing the orthopedic disease auxiliary detection instruction to obtain an image to be processed;

[0007] Retrieving a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module;

[0008] Using the image feature learning module to extract features from the image to be processed to obtain image features;

[0009] Using the feature enhancement module to enhance the image features to obtain enhanced features;

[0010] Inputting the enhanced features into the prediction module, and determining the predicted type and predicted location of the orthopedic disease in the image to be processed according to the output data of the prediction module;

[0011] An orthopedic disease auxiliary detection report is generated according to the predicted type and the predicted position, and the orthopedic disease auxiliary detection report is sent to the triggerer of the orthopedic disease auxiliary detection instruction.

[0012] According to a preferred embodiment of the present invention, the image feature learning module has the same structure as the backbone network of the Sparse R-CNN network; the image feature learning module is used to extract features of the image to be processed to obtain image features, including:

[0013] Obtaining the number of features output by the backbone network in the orthopedic disease auxiliary detection model as the target number;

[0014] The image to be processed is input into the image feature learning module for feature learning to obtain the image features of the target number of levels.

[0015] According to a preferred embodiment of the present invention, the step of enhancing the image features using the feature enhancement module to obtain enhanced features includes:

[0016] Using the feature enhancement module to transfer semantic information from high-level features in the image features to low-level features to obtain at least one semantically enhanced feature;

[0017] The at least one semantic enhancement feature is fused by using the feature enhancement module, and the position information is transferred from the low-level feature to the high-level feature to obtain the enhanced feature.

[0018] According to a preferred embodiment of the present invention, the step of transferring semantic information from high-level features in the image features to low-level features by using the feature enhancement module to obtain at least one semantically enhanced feature comprises:

[0019] Get the image features Among them, F m represents the image features of the mth level, and M represents the number of targets;

[0020] The following formula is used to transfer semantic information from the high-level feature to the low-level feature to obtain the semantic enhancement feature:

[0021]

[0022]

[0023] Among them, f M-1 represents the semantic enhancement feature of the M-1th level; f m represents the semantic enhancement feature of the mth level; conv represents the convolution operation; up represents the upsampling operation; w M,M-1 Represents the adaptive fusion of the image features F at the Mth level M and the image feature F at level M-1 M-1 Time F M The weight of M-1,M-1 represents the adaptive fusion F M With F M-1 Time F M-1 The weight of m+1,m Represents the adaptive fusion of the semantic enhancement feature f at the m+1th level m+1 With Fm Time m+1 The weight of m,m represents the adaptive fusion f m+1 With F m Time F m The weight of ; ∈ represents a constant for normalizing the fusion features.

[0024] According to a preferred embodiment of the present invention, the step of fusing the at least one semantic enhancement feature using the feature enhancement module and transferring position information from the low-level feature to the high-level feature to obtain the enhanced feature includes:

[0025] The semantic enhancement feature is enhanced by the following formula: With the image features The low-level features are fused and the position information is transferred from the low-level features to the high-level features to obtain the enhanced features.

[0026]

[0027]

[0028]

[0029] Among them, P 1 Indicates the enhanced features of level 1; P m represents the enhanced features of the mth level; P M represents the enhanced features of the Mth level; w 21 Represents the adaptive fusion of the first level image features F 1 And the semantic enhancement feature f at the second level 2 Time 2 The weight of 11 represents the adaptive fusion F 1 With f 2 Time F 1 The weight of m-1,m Represents the enhanced feature P of the adaptive fusion m-1th level m-1 、F m With f m Time m-1 The weight of ′ m,m represents adaptive fusion P m-1 、F m With f m Time F m The weight of ′ m ′ ,m represents adaptive fusion P m-1 、F mWith f m Time m The weight of M-1,M Represents the enhanced feature P of the adaptive fusion M-1 level M-1 With F M Time M-1 The weight of M,M represents adaptive fusion P M-1 With F M Time F M The weight of .

[0030] According to a preferred embodiment of the present invention, the prediction module has the same structure as the detection head of the Sparse R-CNN network; the step of inputting the enhanced features into the prediction module and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module comprises:

[0031] Obtain the P from the enhanced features 1 Other enhanced features are taken as features to be processed;

[0032] Inputting the features to be processed into the prediction module;

[0033] The output data of the classification branch of the prediction module is obtained to predict the prediction type, and the output data of the regression branch of the prediction module is obtained to predict the prediction position.

[0034] According to a preferred embodiment of the present invention, before calling the pre-trained orthopedic disease auxiliary detection model, the method further includes:

[0035] Get the initial network built based on the Sparse R-CNN network;

[0036] The weight of each candidate box corresponding to the initial network is configured using the following formula: i =log(x i ×y i ), where Z i represents the weight of the i-th candidate box, x i Indicates the width of the i-th candidate box, y i Represents the height of the i-th candidate box; i is a positive integer;

[0037] Use the following formula to configure the normalized weight term coefficient according to the weight of each candidate box: Among them, α represents the weight term coefficient, N b Indicates the number of candidate boxes in a batch of samples during training, Z k Represents the weight of the kth candidate box;

[0038] The following formula is used to adjust the initial loss function of the initial network according to the weight term coefficient to obtain the current loss function of the initial network: total =αL sparse Among them, L total Represents the current loss function, L sparse represents the initial loss function;

[0039] Training the initial network based on the current loss function;

[0040] When the current loss function reaches convergence, the training is stopped, and the currently obtained model is determined as the orthopedic disease auxiliary detection model.

[0041] An auxiliary detection device for orthopedic diseases, comprising:

[0042] A parsing unit, configured to respond to an orthopedic disease auxiliary detection instruction and parse the orthopedic disease auxiliary detection instruction to obtain an image to be processed;

[0043] A retrieval unit, used to retrieve a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module;

[0044] An extraction unit, used for extracting features from the image to be processed using the image feature learning module to obtain image features;

[0045] An enhancement unit, configured to perform enhancement processing on the image features using the feature enhancement module to obtain enhanced features;

[0046] A prediction unit, used for inputting the enhanced features into the prediction module, and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module;

[0047] A generating unit is used to generate an orthopedic disease auxiliary detection report according to the predicted type and the predicted position, and send the orthopedic disease auxiliary detection report to the trigger of the orthopedic disease auxiliary detection instruction.

[0048] A computer device, comprising:

[0049] a memory storing at least one instruction; and

[0050] A processor executes the instructions stored in the memory to implement the orthopedic disease auxiliary detection method.

[0051] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the orthopedic disease auxiliary detection method.

[0052] It can be seen from the above technical solutions that, on the one hand, the present invention utilizes the image feature learning module in the orthopedic disease auxiliary detection model to extract features from the image to be processed to obtain image features, so as to learn the multi-level features of the image to be processed; on the other hand, the feature enhancement module in the orthopedic disease auxiliary detection model is used to enhance the learned image features to obtain enhanced features, so as to aggregate features of different granularities based on the importance of features at each level, thereby improving the accuracy of subsequent predictions; on the other hand, the enhanced features are input into the prediction module in the orthopedic disease auxiliary detection model to predict the type and location of orthopedic diseases in the image to be processed, and an orthopedic disease auxiliary detection report is generated to be sent to the triggerer of the orthopedic disease auxiliary detection instruction, thereby assisting relevant medical personnel to quickly read the film, while ensuring the accuracy of the diagnosis and improving the diagnostic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention.

[0054] Figure 2 It is a functional module diagram of a preferred embodiment of the orthopedic disease auxiliary detection device of the present invention.

[0055] Figure 3 It is a structural schematic diagram of a computer device of a preferred embodiment of the present invention for implementing the auxiliary detection method for orthopedic diseases. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0058] The orthopedic disease auxiliary detection method is applied to one or more computer devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and their hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0059] The computer device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0060] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0061] The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0062] Among them, Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0064] The network where the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0065] S10, in response to the orthopedic disease auxiliary detection instruction, parsing the orthopedic disease auxiliary detection instruction to obtain an image to be processed.

[0066] In this embodiment, the orthopedic disease auxiliary detection instruction can be triggered by relevant medical personnel to assist in targeted diagnosis according to the actual needs of the medical personnel.

[0067] In this embodiment, the image to be processed may include any type of image, such as X-ray imaging.

[0068] S11, retrieve a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module.

[0069] It can be understood that in order to further accurately diagnose orthopedic diseases that are not easy to detect, such as minor fractures and osteophytes, considering that the features corresponding to the large-size candidate boxes contain rich detail information and semantic information, and this information can provide effective contextual information for diagnosing small-size diseases, this embodiment optimizes the loss function of the model.

[0070] Specifically, before calling the pre-trained orthopedic disease auxiliary detection model, the method further includes:

[0071] Get the initial network built based on the Sparse R-CNN network;

[0072] The weight of each candidate box corresponding to the initial network is configured using the following formula: i =log(x i ×y i ), where Z i represents the weight of the i-th candidate box, x i Indicates the width of the i-th candidate box, y i Represents the height of the i-th candidate box; i is a positive integer;

[0073] Use the following formula to configure the normalized weight term coefficient according to the weight of each candidate box: Among them, α represents the weight term coefficient, N b Indicates the number of candidate boxes in a batch of samples during training, Z k Represents the weight of the kth candidate box;

[0074] The following formula is used to adjust the initial loss function of the initial network according to the weight term coefficient to obtain the current loss function of the initial network: total =αL sparse Among them, L total Represents the current loss function, L sparse represents the initial loss function;

[0075] Training the initial network based on the current loss function;

[0076] When the current loss function reaches convergence, the training is stopped, and the currently obtained model is determined as the orthopedic disease auxiliary detection model.

[0077] In the above embodiment, the loss function is optimized using the size of the lesion area. Specifically, a weight term is added to the loss function to give a higher weight to the large-sized candidate frame, so that the weight term is closely related to the size of the candidate frame. By increasing the importance of the features corresponding to the large-sized candidate frame, the learning of the features corresponding to the large-sized candidate frame is strengthened, so as to make full use of the information contained in the features corresponding to the large-sized candidate frame to enhance the features corresponding to the small-sized candidate frame, thereby obtaining contextual information that is helpful for detecting small-sized orthopedic diseases that are not easy to detect, such as microfractures and osteophytes, and improving the detection performance and accuracy of small-sized orthopedic diseases.

[0078] S12, using the image feature learning module to extract features of the image to be processed to obtain image features.

[0079] In this embodiment, the image feature learning module has the same structure as the backbone network of the Sparse R-CNN network; the image feature learning module is used to extract features of the image to be processed to obtain image features, including:

[0080] Obtaining the number of features output by the backbone network in the orthopedic disease auxiliary detection model as the target number;

[0081] The image to be processed is input into the image feature learning module for feature learning to obtain the image features of the target number of levels.

[0082] Through the above embodiments, the image features of the image to be processed can be fully learned.

[0083] S13, utilizing the feature enhancement module to perform enhancement processing on the image features to obtain enhanced features.

[0084] In this embodiment, the use of the feature enhancement module to enhance the image features to obtain enhanced features includes:

[0085] Using the feature enhancement module to transfer semantic information from high-level features in the image features to low-level features to obtain at least one semantically enhanced feature;

[0086] The at least one semantic enhancement feature is fused by using the feature enhancement module, and the position information is transferred from the low-level feature to the high-level feature to obtain the enhanced feature.

[0087] Specifically, the using the feature enhancement module to transfer semantic information from high-level features in the image features to low-level features to obtain at least one semantically enhanced feature includes:

[0088] Get the image features Among them, F m represents the image features of the mth level, and M represents the number of targets;

[0089] The following formula is used to transfer semantic information from the high-level feature to the low-level feature to obtain the semantic enhancement feature:

[0090]

[0091]

[0092] Among them, f M-1 represents the semantic enhancement feature of the M-1th level; f m represents the semantic enhancement feature of the mth level; conv represents the convolution operation; up represents the upsampling operation; w M,M-1 Represents the adaptive fusion of the image features F at the Mth level M and the image feature F at level M-1 M-1 Time F M The weight of M-1,M-1 represents the adaptive fusion F M With F M-1 Time F M-1 The weight of m+1,m Represents the adaptive fusion of the semantic enhancement feature f at the m+1th level m+1 With F m Time m+1 The weight of m,m represents the adaptive fusion f m+1 With F m Time F m The weight of ; ∈ represents a constant for normalizing the fusion features.

[0093] When the semantic information is transmitted from the high-level feature to the low-level feature, due to the use of the first-level image feature F 1The parameter redundancy caused by the operation is higher than the improvement of subsequent orthopedic disease detection performance, so F 1 Does not participate in feature propagation.

[0094] Specifically, the step of fusing the at least one semantic enhancement feature by using the feature enhancement module and transferring position information from the low-level feature to the high-level feature to obtain the enhanced feature includes:

[0095] The semantic enhancement feature is enhanced by the following formula: With the image features The low-level features are fused and the position information is transferred from the low-level features to the high-level features to obtain the enhanced features.

[0096]

[0097]

[0098]

[0099] Among them, P 1 Indicates the enhanced features of level 1; P m represents the enhanced features of the mth level; P M represents the enhanced features of the Mth level; w 21 Represents the adaptive fusion of the first level image features F 1 And the semantic enhancement feature f at the second level 2 Time 2 The weight of 11 represents the adaptive fusion F 1 With f 2 Time F 1 The weight of m-1,m Represents the enhanced feature P of the adaptive fusion m-1th level m-1 、F m With f m Time m-1 The weight of ′ m,m represents adaptive fusion P m-1 、F m With f m Time F m The weight of ′ m ′ ,m represents adaptive fusion P m-1 、F m With f m Time m The weight of M-1,M Represents the enhanced feature P of the adaptive fusion M-1 levelM-1 With F M Time M-1 The weight of M,M represents adaptive fusion P M-1 With F M Time F M The weight of .

[0100] For example, when the image feature learning module learns 4 levels of image features {F 1 ,F 2 ,F 3 ,F 4}, firstly, according to the importance of different scale features for diagnosing orthopedic diseases, image features of different scales are adaptively aggregated from top to bottom, that is, semantic information is transferred from the high-level features to the low-level features, so as to transfer the semantic information of the high-level features to obtain at least one semantic enhancement feature, and the formula is as follows:

[0101]

[0102]

[0103] Among them, f 3 represents the semantic enhancement feature of the third level, f 2 represents the semantic enhancement feature of the second level, w 43 represents the adaptive fusion F 4 With F 3 Time F 4 The weight, w 33 represents the adaptive fusion F 4 With F 3 Time F 3 The weight, w 32 represents the adaptive fusion f 3 With F 2 Time 3 The weight, w 22 represents the adaptive fusion f 3 With F 2 Time F 2 The weight of ∈ represents the constant for normalizing the fusion feature, such as ∈ can be taken as 0.0001;

[0104] Furthermore, features of different scales are adaptively fused from bottom to top, that is, the position information is transmitted from the low-level features to the high-level features to transmit the position information of the low-level features, so as to obtain the enhanced multi-scale enhanced features {P 1 ,P 2 ,P 3 ,P 4}, the formula is as follows:

[0105]

[0106]

[0107]

[0108]

[0109] Among them, w 21 represents the adaptive fusion F 1 With f 2 Time 2 The weight, w 11 represents the adaptive fusion F 1 With f 2 Time F 1 The weight, w 12 represents adaptive fusion P 1 、F 2 With f 2 Time 1 The weight, w2 ′ 2 indicates adaptive fusion P 1 、F 2 With f 2 Time F 2 The weight of w2″2 represents the adaptive fusion P 1 、F 2 With f 2 Time 2 The weight, w 23 represents adaptive fusion P 2 、F 3 With f 3 Time 2 The weight, w3 ′ 3 represents adaptive fusion P 2 、F 3 With f 3 Time F 3 The weight of w3″3 represents the adaptive fusion P 2 、F 3 With f 3 Time 3 The weight, w 34 represents adaptive fusion P 3 With F 4 Time 3 The weight, w 44 represents adaptive fusion P 3 With F 4 Time F 4 The weight of .

[0110] Typically, high-level features rich in semantic information are used to diagnose larger diseases, and low-level features rich in detail information are used to diagnose smaller diseases. However, since smaller disease features contain less information, the diagnostic accuracy is not high.

[0111] Therefore, this embodiment integrates features of different granularities and levels through the feature enhancement module constructed, so that the features at each level simultaneously aggregate semantic information and location information. That is, by fully integrating characteristic information of different scales to improve the accuracy of diagnosing small-sized diseases, it is possible to effectively diagnose small fractures, osteophytes and other orthopedic diseases that are not easy to detect.

[0112] S14, inputting the enhanced features into the prediction module, and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module.

[0113] In this embodiment, the prediction module has the same structure as the detection head of the Sparse R-CNN network; the step of inputting the enhanced features into the prediction module and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module includes:

[0114] Obtain the P from the enhanced features 1 Other enhanced features are taken as features to be processed;

[0115] Inputting the features to be processed into the prediction module;

[0116] The output data of the classification branch of the prediction module is obtained to predict the prediction type, and the output data of the regression branch of the prediction module is obtained to predict the prediction position.

[0117] Among them, since P 1 More noise is introduced, which is not conducive to the detection of orthopedic diseases. Therefore, the P 1 Other enhanced features are considered as features to be processed.

[0118] For example: Following the above example, when the enhanced multi-scale enhanced features {P 1 ,P 2 ,P 3 ,P 4}, with {P 2 ,P 3 ,P 4} is the input data of the prediction module.

[0119] In the above embodiment, the type and location of orthopedic diseases can be assisted in detection based on artificial intelligence means.

[0120] S15, generating an orthopedic disease auxiliary detection report according to the predicted type and the predicted position, and sending the orthopedic disease auxiliary detection report to the triggerer of the orthopedic disease auxiliary detection instruction.

[0121] The trigger may include relevant medical personnel.

[0122] In this embodiment, the orthopedic disease auxiliary detection report may include an identifier of the predicted type (such as a minor fracture can be identified as 0, and a osteophyte can be identified as 1), as well as a special mark for the predicted position (such as the predicted position can be selected with a mark box, or the predicted position can be marked in a highlighted manner) to assist the triggerer in quickly reading the film, thereby reducing the burden on medical staff. At the same time, due to a series of optimization strategies such as feature learning, feature enhancement, and loss function adjustment adopted in the previous text, the prediction accuracy of orthopedic diseases is effectively improved, thereby being able to more effectively assist medical staff in accurately diagnosing orthopedic diseases.

[0123] It can be seen from the above technical solutions that, on the one hand, the present invention utilizes the image feature learning module in the orthopedic disease auxiliary detection model to extract features from the image to be processed to obtain image features, so as to learn the multi-level features of the image to be processed; on the other hand, the feature enhancement module in the orthopedic disease auxiliary detection model is used to enhance the learned image features to obtain enhanced features, so as to aggregate features of different granularities based on the importance of features at each level, thereby improving the accuracy of subsequent predictions; on the other hand, the enhanced features are input into the prediction module in the orthopedic disease auxiliary detection model to predict the type and location of orthopedic diseases in the image to be processed, and an orthopedic disease auxiliary detection report is generated to be sent to the triggerer of the orthopedic disease auxiliary detection instruction, thereby assisting relevant medical personnel to quickly read the film, while ensuring the accuracy of the diagnosis and improving the diagnostic efficiency.

[0124] like Figure 2 As shown, it is a functional module diagram of a preferred embodiment of the auxiliary detection device for orthopedic diseases of the present invention. The auxiliary detection device for orthopedic diseases 11 includes a parsing unit 110, a retrieval unit 111, an extraction unit 112, an enhancement unit 113, a prediction unit 114, and a generation unit 115. The module / unit referred to in the present invention refers to a series of computer program segments that can be executed by a processor and can perform fixed functions, which are stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0125] The parsing unit 110 is used to respond to the orthopedic disease auxiliary detection instruction and parse the orthopedic disease auxiliary detection instruction to obtain an image to be processed;

[0126] The retrieval unit 111 is used to retrieve a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module;

[0127] The extraction unit 112 is used to extract features of the image to be processed using the image feature learning module to obtain image features;

[0128] The enhancement unit 113 is used to enhance the image features using the feature enhancement module to obtain enhanced features;

[0129] The prediction unit 114 is used to input the enhanced features into the prediction module, and determine the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module;

[0130] The generating unit 115 is used to generate an orthopedic disease auxiliary detection report according to the predicted type and the predicted position, and send the orthopedic disease auxiliary detection report to the triggerer of the orthopedic disease auxiliary detection instruction.

[0131] It can be seen from the above technical solutions that, on the one hand, the present invention utilizes the image feature learning module in the orthopedic disease auxiliary detection model to extract features from the image to be processed to obtain image features, so as to learn the multi-level features of the image to be processed; on the other hand, the feature enhancement module in the orthopedic disease auxiliary detection model is used to enhance the learned image features to obtain enhanced features, so as to aggregate features of different granularities based on the importance of features at each level, thereby improving the accuracy of subsequent predictions; on the other hand, the enhanced features are input into the prediction module in the orthopedic disease auxiliary detection model to predict the type and location of orthopedic diseases in the image to be processed, and an orthopedic disease auxiliary detection report is generated to be sent to the triggerer of the orthopedic disease auxiliary detection instruction, thereby assisting relevant medical personnel to quickly read the film, while ensuring the accuracy of the diagnosis and improving the diagnostic efficiency.

[0132] like Figure 3 The figure is a schematic diagram of the structure of a computer device of a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention.

[0133] The computer device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as an orthopedic disease auxiliary detection program.

[0134] Those skilled in the art will appreciate that the schematic diagram is merely an example of the computer device 1 and does not constitute a limitation on the computer device 1. The computer device 1 may be a bus-type structure or a star-type structure. The computer device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the computer device 1 may also include input and output devices, network access devices, etc.

[0135] It should be noted that the computer device 1 is only an example, and other existing or future electronic products that are suitable for the present invention should also be included in the protection scope of the present invention and included here by reference.

[0136] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 can also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 1. Further, the memory 12 can also include both an internal storage unit of the computer device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the computer device 1, such as the code of the orthopedic disease auxiliary detection program, but also can be used to temporarily store data that has been output or is to be output.

[0137] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the computer device 1, and uses various interfaces and lines to connect the various components of the entire computer device 1, and executes or executes programs or modules stored in the memory 12 (for example, executing orthopedic disease auxiliary detection programs, etc.), and calls data stored in the memory 12 to execute various functions of the computer device 1 and process data.

[0138] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the auxiliary detection method for orthopedic diseases, for example Figure 1 Steps shown.

[0139] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a parsing unit 110, a retrieval unit 111, an extraction unit 112, an enhancement unit 113, a prediction unit 114, and a generation unit 115.

[0140] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, and includes a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute the part of the orthopedic disease auxiliary detection method described in each embodiment of the present invention.

[0141] If the module / unit integrated in the computer device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0142] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0143] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0144] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0145] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by only one straight line, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0146] Although not shown, the computer device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The computer device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0147] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0148] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the computer device 1 and to display a visual user interface.

[0149] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0150] Figure 3 Only the computer device 1 having components 12-13 is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0151] Combination Figure 1 , the memory 12 in the computer device 1 stores a plurality of instructions to implement an orthopedic disease auxiliary detection method, and the processor 13 can execute the plurality of instructions to implement:

[0152] In response to an orthopedic disease auxiliary detection instruction, parsing the orthopedic disease auxiliary detection instruction to obtain an image to be processed;

[0153] Retrieving a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module;

[0154] Using the image feature learning module to extract features from the image to be processed to obtain image features;

[0155] Using the feature enhancement module to enhance the image features to obtain enhanced features;

[0156] Inputting the enhanced features into the prediction module, and determining the predicted type and predicted location of the orthopedic disease in the image to be processed according to the output data of the prediction module;

[0157] An orthopedic disease auxiliary detection report is generated according to the predicted type and the predicted position, and the orthopedic disease auxiliary detection report is sent to the triggerer of the orthopedic disease auxiliary detection instruction.

[0158] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0159] It should be noted that the data involved in this case were all obtained legally.

[0160] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0161] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0162] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0164] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0165] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0166] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the present invention can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for auxiliary detection of orthopedic diseases, characterized in that: The orthopedic disease auxiliary detection method comprises: In response to an orthopedic disease auxiliary detection instruction, parsing the orthopedic disease auxiliary detection instruction to obtain an image to be processed; Retrieving a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module; Using the image feature learning module to extract features from the image to be processed to obtain image features; Using the feature enhancement module to enhance the image features to obtain enhanced features; Inputting the enhanced features into the prediction module, and determining the predicted type and predicted location of the orthopedic disease in the image to be processed according to the output data of the prediction module; Generate an orthopedic disease auxiliary detection report according to the predicted type and the predicted position, and send the orthopedic disease auxiliary detection report to the triggerer of the orthopedic disease auxiliary detection instruction; Before calling the pre-trained orthopedic disease auxiliary detection model, an initial network constructed based on the Sparse R-CNN network is obtained; The weight of each candidate box corresponding to the initial network is configured using the following formula: i =log(x i ×y i ), where Z i represents the weight of the i-th candidate box, x i Indicates the width of the i-th candidate box, y i Represents the height of the i-th candidate box; i is a positive integer; Use the following formula to configure the normalized weight term coefficient according to the weight of each candidate box: Among them, α represents the weight term coefficient, N b Indicates the number of candidate boxes in a batch of samples during training, Z k Represents the weight of the kth candidate box; The following formula is used to adjust the initial loss function of the initial network according to the weight term coefficient to obtain the current loss function of the initial network: total =αL sparse Among them, L total Represents the current loss function, L sparse represents the initial loss function; Training the initial network based on the current loss function; When the current loss function reaches convergence, the training is stopped, and the currently obtained model is determined as the orthopedic disease auxiliary detection model.

2. The orthopedic disease auxiliary detection method according to claim 1, characterized in that: The image feature learning module has the same structure as the backbone network of the Sparse R-CNN network; the image feature learning module is used to extract features of the image to be processed to obtain image features, including: Obtaining the number of features output by the backbone network in the orthopedic disease auxiliary detection model as the target number; The image to be processed is input into the image feature learning module for feature learning to obtain the image features of the target number of levels.

3. The orthopedic disease auxiliary detection method according to claim 2, characterized in that: The enhancing process of the image features by using the feature enhancement module to obtain the enhanced features comprises: Using the feature enhancement module to transfer semantic information from high-level features in the image features to low-level features to obtain at least one semantically enhanced feature; The at least one semantic enhancement feature is fused by using the feature enhancement module, and the position information is transferred from the low-level feature to the high-level feature to obtain the enhanced feature.

4. The orthopedic disease auxiliary detection method according to claim 3, characterized in that: The step of using the feature enhancement module to transfer semantic information from high-level features in the image features to low-level features to obtain at least one semantic enhancement feature includes: Get the image features Among them, F m represents the image features of the mth level, and M represents the number of targets; The following formula is used to transfer semantic information from the high-level feature to the low-level feature to obtain the semantic enhancement feature: Among them, f M-1 represents the semantic enhancement feature of the M-1th level; f m represents the semantic enhancement feature of the mth level; conv represents the convolution operation; up represents the upsampling operation; w M,M-1 Represents the adaptive fusion of the image features F at the Mth level M and the image feature F at level M-1 M-1 Time F M The weight of M-1,M-1 represents the adaptive fusion F M With F M-1 Time F M-1 The weight of m+1,m Represents the adaptive fusion of the semantic enhancement feature f at the m+1th level m+1 With F m Time m+1 The weight of m,m represents the adaptive fusion f m+1 With F m Time F m The weight of ; ∈ represents a constant for normalizing the fusion features.

5. The orthopedic disease auxiliary detection method according to claim 4, characterized in that: The step of fusing the at least one semantic enhancement feature using the feature enhancement module and transferring position information from the low-level feature to the high-level feature to obtain the enhanced feature includes: The semantic enhancement feature is enhanced by the following formula: With the image features The low-level features are fused and the position information is transferred from the low-level features to the high-level features to obtain the enhanced features. Among them, P 1 Indicates the enhanced features of level 1; P m represents the enhanced features of the mth level; P M represents the enhanced features of the Mth level; w 21 Represents the adaptive fusion of the first level image features F 1 And the semantic enhancement feature f at the second level 2 Time 2 The weight of 11 represents the adaptive fusion F 1 With f 2 Time F 1 The weight of m-1,m Represents the enhanced feature p of the adaptive fusion m-1th level m-1 、F m With f m Time m-1 The weight of m,m represents adaptive fusion P m-1 、F m With f m Time F m The weight of m,m represents adaptive fusion P m-1 、F m With f m Time m The weight of M-1,M Represents the enhanced feature P of the adaptive fusion M-1 level M-1 With F M Time M-1 The weight of M,M represents adaptive fusion P M-1 With F M Time F M The weight of .

6. The orthopedic disease auxiliary detection method according to claim 5, characterized in that: The prediction module has the same structure as the detection head of the SparseR-CNN network; the step of inputting the enhanced features into the prediction module and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module comprises: Obtain the P from the enhanced features 1 Other enhanced features are taken as features to be processed; Inputting the features to be processed into the prediction module; The output data of the classification branch of the prediction module is obtained to predict the prediction type, and the output data of the regression branch of the prediction module is obtained to predict the prediction position.

7. An auxiliary detection device for orthopedic diseases, characterized in that: The orthopedic disease auxiliary detection device comprises: A parsing unit, configured to respond to an orthopedic disease auxiliary detection instruction and parse the orthopedic disease auxiliary detection instruction to obtain an image to be processed; A retrieval unit, used to retrieve a pre-trained orthopedic disease auxiliary detection model; wherein the orthopedic disease auxiliary detection model includes an image feature learning module, a feature enhancement module and a prediction module; An extraction unit, used for extracting features from the image to be processed using the image feature learning module to obtain image features; An enhancement unit, configured to perform enhancement processing on the image features using the feature enhancement module to obtain enhanced features; A prediction unit, used for inputting the enhanced features into the prediction module, and determining the predicted type and predicted position of the orthopedic disease in the image to be processed according to the output data of the prediction module; A generating unit, configured to generate an orthopedic disease auxiliary detection report according to the predicted type and the predicted position, and send the orthopedic disease auxiliary detection report to a triggerer of the orthopedic disease auxiliary detection instruction; Before calling the pre-trained orthopedic disease auxiliary detection model, an initial network constructed based on the Sparse R-CNN network is obtained; The weight of each candidate box corresponding to the initial network is configured using the following formula: i =log(x i ×y i ), where Z i represents the weight of the i-th candidate box, x i Indicates the width of the i-th candidate box, y i Represents the height of the i-th candidate box; i is a positive integer; Use the following formula to configure the normalized weight term coefficient according to the weight of each candidate box: Among them, α represents the weight term coefficient, N b Indicates the number of candidate boxes in a batch of samples during training, Z k Represents the weight of the kth candidate box; The following formula is used to adjust the initial loss function of the initial network according to the weight term coefficient to obtain the current loss function of the initial network: total =αL sparse Among them, L total Represents the current loss function, L sparse represents the initial loss function; Training the initial network based on the current loss function; When the current loss function reaches convergence, the training is stopped, and the currently obtained model is determined as the orthopedic disease auxiliary detection model.

8. A computer device, characterized in that: The computer device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the orthopedic disease auxiliary detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the orthopedic disease auxiliary detection method as described in any one of claims 1 to 6.

Images