Bone joint multi-disease screening intelligent evaluation method based on deep learning
Through deep learning-based methods, the features of bone joint X-ray images and text information are extracted and fusion are solved, and the problem of inaccurate screening of multiple diseases of bone joints in the prior art is solved, and more accurate multiple diseases are achieved Smart screening assessment.
Patent Information
- Application Number
- CN202411404599.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-02-14
AI Technical Summary
It is difficult to accurately conduct intelligent screening and evaluation of multiple diseases of osteoarthritis, especially when screening multiple diseases at the same time, the problem of inaccurate identification is more prominent.
Using a deep learning-based method, by obtaining the patient's bone joint X-ray image and text information, the preset deep learning network model extracts image features and text features, and performs feature fusion, and finally outputs multiple diseases of osteoarthritis. Check the results of intelligent evaluation.
A more accurate intelligent evaluation of screening for multiple bone and joint diseases is achieved, and the intelligent evaluation results of multiple bone and joint diseases and their corresponding levels of disease screening are simultaneously identified.
Smart Images
Figure CN119446550A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of multi-disease screening for bone and joint diseases, and in particular, relates to an intelligent assessment method, system, device and computer-readable storage medium for multi-disease screening for bone and joint diseases based on deep learning. Background Art
[0002] At present, the screening of single bone and joint diseases is mainly based on the identification of lesions based on bone and joint images. However, the identification of lesions based solely on images may result in inaccurate identification. Moreover, it is impossible to perform intelligent evaluation of multi-disease screening at the same time.
[0003] Therefore, how to more accurately conduct intelligent assessment for screening of multiple bone and joint diseases is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0004] The embodiments of the present application provide a method, system, device and computer-readable storage medium for intelligent assessment of multiple bone and joint diseases screening based on deep learning, which can more accurately perform intelligent assessment of multiple bone and joint diseases screening.
[0005] In a first aspect, the present application provides an intelligent assessment method for screening multiple bone and joint diseases based on deep learning, including:
[0006] Obtaining bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information;
[0007] Input bone and joint X-ray images and text information into the preset deep learning network model to output intelligent assessment results for multiple bone and joint disease screening;
[0008] Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features;
[0009] The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
[0010] Optional, the corresponding grades of hip femoral head necrosis include:
[0011] Level 1: The femoral head has a complete appearance and the joint space is normal;
[0012] Level 2: Bone sclerosis, focal osteoporosis or subtle cystic changes in the femoral head, no subchondral fracture, fracture in the necrotic area, and collapse of the femoral head;
[0013] Grade III: subchondral fracture, fracture in the necrotic area, and / or femoral head collapse;
[0014] Grade IV: Narrowing of the joint space, acetabular changes, and joint destruction;
[0015] The corresponding grades for hip osteoarthritis or knee osteoarthritis include:
[0016] Level 0: Normal;
[0017] Grade 1: Mild osteophytes;
[0018] Grade II: obvious osteophytes, but without involving the joint space;
[0019] Grade 3: moderate narrowing of the joint space;
[0020] Grade 4: The joint space is significantly narrowed and the subchondral bone is sclerotic.
[0021] Optionally, an image feature extraction network for extracting image features, including:
[0022] Use refined global attention mechanism to extract image features from bone and joint X-ray images;
[0023] Among them, the refined global attention mechanism includes:
[0024] Refine the channel attention mechanism, perform dilated convolutions with different hole sizes on the feature map, and dilated convolutions with different hole sizes share weights;
[0025] The channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution. Each dilated convolution kernel only calculates the features of one channel to obtain the image features. Then, based on the image features, the refined channel attention is obtained.
[0026] Refined spatial attention mechanism, which divides the feature map into several groups, calculates the attention weight for each group, and obtains refined spatial attention;
[0027] The refined channel attention is dot-multiplied with the refined spatial attention to obtain the refined global attention.
[0028] Optionally, the channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution. Each dilated convolution kernel only calculates the features of one channel to obtain the image features. Then, based on the image features, the refined channel attention is obtained, including:
[0029] Add the element-by-element sum of several feature maps outputted by dilated convolution operations with different dilated sizes;
[0030] The summed result is convolved channel by channel again to fuse features of different scales, and then goes through global average pooling, fully connected layer compression of the number of feature channels, fully connected layer restoration of the number of channels and Sigmoid function nonlinear activation to obtain refined channel attention.
[0031] Optionally, a text feature extraction network for extracting text features, including:
[0032] Use ALBERT technology to convert text information into text feature word vectors;
[0033] Use the text feature extraction network to extract the text feature word vector and obtain the optimal feature vector;
[0034] Among them, the text feature extraction network includes a convolutional neural network with attention mechanism and a bidirectional GRU neural network;
[0035] Convolutional neural network and bidirectional GRU network are used to extract local feature vectors and global feature vectors of text respectively, and finally the feature vectors of the two are fused to output the optimal feature vector.
[0036] Optionally, a convolutional neural network and a bidirectional GRU neural network are used to extract local feature vectors and global feature vectors of the text, respectively, including:
[0037] The text feature word vector is input into the convolutional neural network using the mini-batch gradient descent method. The convolution layer extracts multiple different feature expressions of the feature word through convolution kernels of different sizes. The pooling layer compresses and reduces the dimension of the output features of the convolution layer to retain the main features of the feature word. Finally, the pooling layer outputs the extracted features to the fully connected layer to obtain local features.
[0038] At the same time, the text feature word vector is input into the bidirectional GRU neural network using the mini-batch gradient descent method to obtain the global features.
[0039] Optionally, image features and text features are fused, including:
[0040] Set weight coefficients for image features and text features respectively;
[0041] Weighted feature fusion is performed based on image features, text features and their corresponding weight coefficients.
[0042] In a second aspect, the present application embodiment provides an intelligent assessment system for screening multiple bone and joint diseases based on deep learning, including:
[0043] An information acquisition module is used to acquire bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information;
[0044] The multi-disease screening intelligent assessment module is used to input bone and joint X-ray images and text information into a preset deep learning network model and output the multi-disease screening intelligent assessment results for bone and joints;
[0045] Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features;
[0046] The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
[0047] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
[0048] When the processor executes the computer program instructions, an intelligent assessment method for screening multiple bone and joint diseases based on deep learning is implemented.
[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, an intelligent assessment method for screening multiple bone and joint diseases based on deep learning is implemented.
[0050] The deep learning-based intelligent assessment method, system, device and computer-readable storage medium for screening of multiple bone and joint diseases in the embodiments of the present application can more accurately perform intelligent assessment for screening of multiple bone and joint diseases.
[0051] The deep learning-based intelligent assessment method for screening multiple bone and joint diseases includes:
[0052] Obtaining bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information;
[0053] Input bone and joint X-ray images and text information into the preset deep learning network model to output intelligent assessment results for multiple bone and joint disease screening;
[0054] Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features;
[0055] The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 It is a flowchart of an intelligent assessment method for screening multiple bone and joint diseases based on deep learning provided by an embodiment of the present application;
[0058] Figure 2 It is a flowchart of image feature extraction using a refined global attention mechanism provided by an embodiment of the present application;
[0059] Figure 3 It is a flowchart of obtaining refined channel attention provided by an embodiment of the present application;
[0060] Figure 4 It is a schematic diagram of a process of obtaining local features by a convolutional neural network provided by an embodiment of the present application;
[0061] Figure 5 It is a schematic diagram of a process of obtaining global features by a bidirectional GRU neural network provided by an embodiment of the present application;
[0062] Figure 6 It is a structural schematic diagram of an intelligent assessment system for screening multiple bone and joint diseases based on deep learning provided by an embodiment of the present application;
[0063] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0064] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0065] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0066] In order to solve the problems of the prior art, the embodiments of the present application provide a method, system, device and computer-readable storage medium for intelligent assessment of multiple bone and joint diseases based on deep learning. The following first introduces the intelligent assessment method for multiple bone and joint diseases based on deep learning provided by the embodiments of the present application.
[0067] Figure 1 The flowchart of the intelligent assessment method for multiple bone and joint diseases screening based on deep learning provided by one embodiment of the present application is shown. Figure 1 As shown in the figure, the deep learning-based intelligent assessment method for screening multiple bone and joint diseases includes:
[0068] S101, obtaining a bone and joint X-ray image and text information of a patient; wherein the bone and joint X-ray image includes a hip joint X-ray image or a knee joint X-ray image; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information;
[0069] S102, inputting bone and joint X-ray images and text information into a preset deep learning network model, and outputting intelligent assessment results for bone and joint multi-disease screening;
[0070] Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features;
[0071] The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
[0072] In one embodiment, the corresponding grades of hip femoral head necrosis include:
[0073] Level 1: The femoral head has a complete appearance and the joint space is normal;
[0074] Level 2: Bone sclerosis, focal osteoporosis or subtle cystic changes in the femoral head, no subchondral fracture, fracture in the necrotic area, and collapse of the femoral head;
[0075] Grade III: subchondral fracture, fracture in the necrotic area, and / or femoral head collapse;
[0076] Grade IV: Narrowing of the joint space, acetabular changes, and joint destruction;
[0077] The corresponding grades for hip osteoarthritis or knee osteoarthritis include:
[0078] Level 0: Normal;
[0079] Grade 1: Mild osteophytes;
[0080] Grade II: obvious osteophytes, but without involving the joint space;
[0081] Grade 3: moderate narrowing of the joint space;
[0082] Grade 4: The joint space is significantly narrowed and the subchondral bone is sclerotic.
[0083] like Figure 2 As shown, in one embodiment, an image feature extraction network for extracting image features includes:
[0084] Use refined global attention mechanism to extract image features from bone and joint X-ray images;
[0085] Among them, the refined global attention mechanism includes:
[0086] Refine the channel attention mechanism, perform dilated convolutions with different hole sizes on the feature map, and dilated convolutions with different hole sizes share weights;
[0087] The channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution. Each dilated convolution kernel only calculates the features of one channel to obtain the image features. Then, based on the image features, the refined channel attention is obtained.
[0088] Refined spatial attention mechanism, which divides the feature map into several groups, calculates the attention weight for each group, and obtains refined spatial attention;
[0089] The refined channel attention is dot-multiplied with the refined spatial attention to obtain the refined global attention.
[0090] like Figure 3 As shown, in one embodiment, the channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution, and each dilated convolution kernel only calculates the features of one channel to obtain the image features, and then obtains the refined channel attention according to the image features, including:
[0091] Add the element-by-element sum of several feature maps outputted by dilated convolution operations with different dilated sizes;
[0092] The summed result is convolved channel by channel again to fuse features of different scales, and then goes through global average pooling, fully connected layer compression of the number of feature channels, fully connected layer restoration of the number of channels and Sigmoid function nonlinear activation to obtain refined channel attention.
[0093] In one embodiment, a text feature extraction network for extracting text features includes:
[0094] Use ALBERT technology to convert text information into text feature word vectors;
[0095] Use the text feature extraction network to extract the text feature word vector and obtain the optimal feature vector;
[0096] Among them, the text feature extraction network includes a convolutional neural network with attention mechanism and a bidirectional GRU neural network;
[0097] Convolutional neural network and bidirectional GRU network are used to extract local feature vectors and global feature vectors of text respectively, and finally the feature vectors of the two are fused to output the optimal feature vector.
[0098] like Figure 4 and Figure 5 As shown, Figure 4 It is a schematic diagram of a process of obtaining local features by a convolutional neural network provided by an embodiment of the present application; Figure 5 It is a flowchart of a bidirectional GRU neural network obtaining global features provided by an embodiment of the present application.
[0099] In one embodiment, a local feature vector and a global feature vector of a text are extracted using a convolutional neural network and a bidirectional GRU neural network, respectively, including:
[0100] The text feature word vector is input into the convolutional neural network using the mini-batch gradient descent method. The convolution layer extracts multiple different feature expressions of the feature word through convolution kernels of different sizes. The pooling layer compresses and reduces the dimension of the output features of the convolution layer to retain the main features of the feature word. Finally, the pooling layer outputs the extracted features to the fully connected layer to obtain local features.
[0101] At the same time, the text feature word vector is input into the bidirectional GRU neural network using the mini-batch gradient descent method to obtain the global features.
[0102] In one embodiment, the image features and the text features are fused, including:
[0103] Set weight coefficients for image features and text features respectively;
[0104] Weighted feature fusion is performed based on image features, text features and their corresponding weight coefficients.
[0105] Figure 6 It is a structural schematic diagram of an intelligent assessment system for screening multiple bone and joint diseases based on deep learning provided by an embodiment of the present application;
[0106] The deep learning-based intelligent assessment system for multi-disease screening of bone and joint diseases includes:
[0107] The information acquisition module 601 is used to acquire the bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information;
[0108] The multi-disease screening intelligent assessment module 602 is used to input bone and joint X-ray images and text information into a preset deep learning network model and output the bone and joint multi-disease screening intelligent assessment results;
[0109] Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features;
[0110] The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
[0111] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.
[0112] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.
[0113] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0114] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 702 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 702 may be inside or outside the electronic device. In a particular embodiment, the memory 702 may be a non-volatile solid-state memory.
[0115] In one embodiment, the memory 702 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0116] The processor 701 reads and executes the computer program instructions stored in the memory 702 to implement any one of the deep learning-based intelligent assessment methods for screening multiple bone and joint diseases in the above-mentioned embodiments.
[0117] In one example, the electronic device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.
[0118] The communication interface 703 is mainly used to implement communication between various modules, systems, units and / or devices in the embodiments of the present application.
[0119] Bus 710 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0120] In addition, in combination with the bone and joint multi-disease screening intelligent assessment method based on deep learning in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the bone and joint multi-disease screening intelligent assessment methods based on deep learning in the above embodiments is implemented.
[0121] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0122] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0123] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0124] The above reference is according to the method of the embodiment of the present application, the flowchart and / or block diagram of the system and computer program product described various aspects of the present application.It should be understood that each square block in the flowchart and / or block diagram and the combination of each square block in the flowchart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing system, to produce a kind of machine, so that these instructions executed by the processor of the computer or other programmable data processing system enable the realization of the function / action specified in one or more square blocks of the flowchart and / or block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square block in the block diagram and / or the flowchart and the combination of the square blocks in the block diagram and / or the flowchart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.
[0125] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A deep learning-based intelligent assessment method for screening multiple bone and joint diseases, characterized in that: include: Obtaining bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information; Input bone and joint X-ray images and text information into the preset deep learning network model to output intelligent assessment results for multiple bone and joint disease screening; Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features; The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
2. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 1 is characterized in that: The corresponding grades of hip femoral head necrosis include: Level 1: The femoral head has a complete appearance and the joint space is normal; Level 2: Bone sclerosis, focal osteoporosis or subtle cystic changes in the femoral head, no subchondral fracture, fracture in the necrotic area, and collapse of the femoral head; Grade III: subchondral fracture, fracture in the necrotic area, and / or femoral head collapse; Grade IV: Narrowing of the joint space, acetabular changes, and joint destruction; The corresponding grades for hip osteoarthritis or knee osteoarthritis include: Level 0: Normal; Grade 1: Mild osteophytes; Grade II: obvious osteophytes, but without involving the joint space; Grade 3: moderate narrowing of the joint space; Grade 4: The joint space is significantly narrowed and the subchondral bone is sclerotic.
3. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 1 is characterized in that: The image feature extraction network used to extract image features includes: Use refined global attention mechanism to extract image features from bone and joint X-ray images; Among them, the refined global attention mechanism includes: Refine the channel attention mechanism, perform dilated convolutions with different hole sizes on the feature map, and dilated convolutions with different hole sizes share weights; The channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution. Each dilated convolution kernel only calculates the features of one channel to obtain the image features. Then, based on the image features, the refined channel attention is obtained. Refined spatial attention mechanism, which divides the feature map into several groups, calculates the attention weight for each group, and obtains refined spatial attention; The refined channel attention is dot-multiplied with the refined spatial attention to obtain the refined global attention.
4. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 3 is characterized in that: The channel-by-channel convolution in the depthwise separable convolution is combined with the dilated convolution. Each dilated convolution kernel only calculates the features of one channel to obtain the image features. Then, based on the image features, the refined channel attention is obtained, including: Add the element-by-element sum of several feature maps outputted by dilated convolution operations with different dilated sizes; The summed result is convolved channel by channel again to fuse features of different scales, and then goes through global average pooling, fully connected layer compression of the number of feature channels, fully connected layer restoration of the number of channels and Sigmoid function nonlinear activation to obtain refined channel attention.
5. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 1 is characterized in that: The text feature extraction network used to extract text features includes: Use ALBERT technology to convert text information into text feature word vectors; Use the text feature extraction network to extract the text feature word vector and obtain the optimal feature vector; Among them, the text feature extraction network includes a convolutional neural network with attention mechanism and a bidirectional GRU neural network; Convolutional neural network and bidirectional GRU network are used to extract local feature vectors and global feature vectors of text respectively, and finally the feature vectors of the two are fused to output the optimal feature vector.
6. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 5 is characterized in that: Convolutional neural network and bidirectional GRU neural network are used to extract local feature vectors and global feature vectors of text, including: The text feature word vector is input into the convolutional neural network using the mini-batch gradient descent method. The convolution layer extracts multiple different feature expressions of the feature word through convolution kernels of different sizes. The pooling layer compresses and reduces the dimension of the output features of the convolution layer to retain the main features of the feature word. Finally, the pooling layer outputs the extracted features to the fully connected layer to obtain local features. At the same time, the text feature word vector is input into the bidirectional GRU neural network using the mini-batch gradient descent method to obtain the global features.
7. The deep learning-based intelligent assessment method for screening multiple bone and joint diseases according to claim 1 is characterized in that: The image features and text features are fused, including: Set weight coefficients for image features and text features respectively; Weighted feature fusion is performed based on image features, text features and their corresponding weight coefficients.
8. A deep learning-based intelligent assessment system for screening multiple bone and joint diseases, characterized in that: The system comprises: An information acquisition module is used to acquire bone and joint X-ray images and text information of the patient; wherein the bone and joint X-ray images include hip joint X-ray images or knee joint X-ray images; the text information includes bone and joint questionnaire information and historical case information, and the bone and joint questionnaire information includes the patient's identity information and bone and joint function assessment information; The multi-disease screening intelligent assessment module is used to input bone and joint X-ray images and text information into a preset deep learning network model and output the multi-disease screening intelligent assessment results for bone and joints; Among them, the deep learning network model is used to: extract image features and text features respectively; perform feature fusion of image features and text features; The intelligent assessment results of multiple bone and joint disease screening include: hip femoral head necrosis, hip osteoarthritis, hip dysplasia, hip fracture, knee osteoarthritis, knee fracture and their corresponding levels of disease screening intelligent assessment results.
9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the deep learning-based intelligent assessment method for screening multiple bone and joint diseases as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the deep learning-based intelligent assessment method for screening multiple bone and joint diseases as described in any one of claims 1 to 7.
Citation Information
Cited By
Hip multi-disease automatic classification method and system
CN121278474A