Intelligent detection method, device, electronic equipment and readable medium for knee joint images

By improving the YOLOx neural network structure, an X-ray recognition model for knee osteoarthritis was constructed. By utilizing the residual fusion module and the feature pyramid (FPN), the accuracy and efficiency issues of knee osteoarthritis image detection were resolved, achieving efficient and accurate diagnosis of knee osteoarthritis.

CN115760770BActive Publication Date: 2025-10-31WUHAN POLYTECHNIC UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211450986.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-18
Publication Date
2025-10-31
Estimated Expiration
2042-11-18

AI Technical Summary

Technical Problem

Current methods for diagnosing knee osteoarthritis rely on doctors' subjective judgment. Manual identification is inefficient and prone to missing information. There is a lack of effective intelligent technologies for efficient and accurate image detection of knee osteoarthritis.

Method used

A model for X-ray recognition of knee osteoarthritis based on YOLOx neural network was constructed. By introducing residual fusion module and feature pyramid FPN, the structure of YOLOx neural network was improved. Combined with multi-combination residual fusion network ResXBlockNet and detection head YoloHead, accurate detection of knee osteoarthritis was achieved.

Benefits of technology

It improves the accuracy of identifying knee osteoarthritis, prevents misdiagnosis, reduces the possibility of doctors misdiagnosing, and enhances detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115760770B_ABST
    Figure CN115760770B_ABST
Patent Text Reader

Abstract

This invention provides a method, device, electronic device, and readable medium for intelligent detection of knee joint images. The method includes image annotation and enhancement processing of knee osteoarthritis X-ray data, and constructing training and validation sets according to a set ratio. A residual fusion module is introduced to construct a knee osteoarthritis X-ray recognition model based on the YOLOx neural network framework. The recognition model replaces the CSPDarknet module in the YOLOx neural network framework with a multi-combination residual fusion network ResXBlockNet, and a residual fusion module is introduced into the ResXBlockNet. Based on the training set, a trained knee osteoarthritis X-ray recognition model is obtained. Newly acquired knee osteoarthritis X-ray images are input into the trained recognition model, and intelligent detection results of the knee joint images are output.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent medical image processing, specifically to an intelligent detection method, apparatus, electronic device, and readable medium for knee joint images. Background Technology

[0002] Knee osteoarthritis (KOA) is a chronic degenerative disease that seriously affects the quality of life of middle-aged and elderly people. It is a multifactorial disease affecting the entire joint, with the exact cause still unclear. Its occurrence is related to age, obesity, inflammation, trauma, and genetic factors. Pathological features include articular cartilage destruction, subchondral bone sclerosis or cystic changes, marginal osteophyte formation, synovial lesions, joint capsule contracture, ligament laxity or contracture, and muscle weakness.

[0003] There are several grading systems for knee osteoarthritis, with the Kellgren-Lawrence (KL) grading system being the most widely used. The Kellgren-Lawrence grading system classifies knee osteoarthritis into five grades from mild to severe based on X-ray findings: Grade 0 (normal knee), Grade I (suspected joint space narrowing, possible osteophytes, but mild), Grade II (clearly present small osteophytes and possible joint space narrowing), Grade III (numerous moderate osteophytes, clear joint space narrowing, some subchondral bone sclerosis, and possible knee osteoarthritis), Grade IV (numerous large osteophytes, severe joint space narrowing, and obvious osteophytes). Subchondral bone sclerosis and obvious knee joint bony deformities are observed. Doctors use the KL classification to quantify the severity of knee osteoarthritis and to communicate with colleagues to develop treatment plans. Current KOA diagnosis relies on subjective judgment by doctors, resulting in low efficiency and potential for inaccurate identification when the eye is fatigued or the number of images is too large. However, due to the difficulty of feature extraction from medical images and the current lack of effective technologies, providing intelligent technologies to support efficient and accurate image detection of knee osteoarthritis through data is a pressing issue that needs to be addressed. Summary of the Invention

[0004] To address the problems in the prior art, this application provides a method, device, electronic device, and readable medium for intelligent detection of knee joint images, which can accurately detect knee joint-related lesions in real time.

[0005] To address the shortcomings of the existing technology, this invention proposes an intelligent detection method for knee joint images, comprising the following steps:

[0006] Step 1: Perform image annotation and enhancement processing on the X-ray data of knee osteoarthritis, and construct training and validation sets according to the proportions;

[0007] Step 2: Introduce a residual fusion module and construct an X-ray recognition model for knee osteoarthritis based on the YOLOx neural network architecture framework. The recognition model replaces the CSPDarknet module in the YOLOx neural network architecture framework with a multi-combination residual fusion network ResXBlockNet, and introduces a residual fusion module into the multi-combination residual fusion network ResXBlockNet.

[0008] Step 3: Based on the training set, train to obtain a trained X-ray recognition model for knee osteoarthritis;

[0009] Step 4: Input the newly acquired X-ray images of knee osteoarthritis into the trained recognition model and output the intelligent detection results of the knee joint images.

[0010] Moreover, step 1 is implemented by acquiring X-ray image data of the knee joint and dividing it into training and validation sets, and then labeling it to obtain labeled image data; the labeled images are then subjected to enhancement processing such as flipping, color gamut transformation, Gaussian blurring, scaling and stitching to obtain the enhanced dataset.

[0011] Furthermore, the X-ray recognition model for knee osteoarthritis includes a multi-combination residual fusion network ResXBlockNet, a feature pyramid FPN, and a detection head YoloHead. The multi-combination residual fusion network ResXBlockNet includes an input layer, a CBL module, and five residual fusion modules ResxB. The input from the input layer is passed to the CBL module, and then sequentially to the five ResxB modules.

[0012] The feature transfer via the Feature Pyramid Network (FPN) in ResXBlockNet is divided into three parts.

[0013] The first part involves directly transmitting the data output from the fifth residual fusion module, ResxB, to the first YoloHead;

[0014] The second part involves performing CBL and upsampling operations on the data output from the second and fifth residual fusion modules ResxB, then connecting them via Concat and inputting them into five cascaded CBL modules, which are then output to the second YoloHead.

[0015] The third part consists of the data output from the first residual fusion module ResxB block and the data after the second part is concat-connected and input into the five cascaded CBL modules. The data at this point is output to the third YoloHead for processing.

[0016] Furthermore, in the ResxB module, the first layer is the CBL module, and the second layer is the multi-scale feature fusion module. The output of the CBL module is input to the multi-scale feature fusion module. The features extracted by the first-layer CBL module are connected to the features extracted after multi-scale fusion by ADD. The feature maps are added together under the condition that the number of channels remains unchanged. Max pooling and channel shuffling operations are performed on the feature maps after ADD connection.

[0017] Moreover, the multi-scale feature fusion module consists of three different CBL modules with a stride of 1×1 and convolution kernel sizes of 1×1, 3×3, and 5×5, respectively. The ADD operation is selected to connect the three CBL modules, keeping the number of channels unchanged while upsampling feature maps of different scales to the same size for fusion.

[0018] On the other hand, the present invention provides an intelligent detection device for knee joint images, which is used to implement the intelligent detection method for knee joint images as described above.

[0019] Moreover, it includes the following modules,

[0020] The acquisition unit is used to acquire X-ray images of the knee osteoarthritis to be detected from the X-ray machine;

[0021] The diagnostic unit is used to input the X-ray image of the knee osteoarthritis to be detected into the knee osteoarthritis X-ray recognition model. The X-ray image to be detected is processed by the multi-combination residual fusion network ResXBlockNet, feature pyramid FPN and detection head YoloHead of the recognition model in sequence to obtain the output result. The output result is the recognition result of the X-ray image of the knee osteoarthritis to be detected, including whether the X-ray image to be detected is knee osteoarthritis and the intelligent detection result of the grade of knee osteoarthritis.

[0022] An interactive unit that outputs model-based intelligent detection results to doctors and patients;

[0023] On the other hand, the present invention provides an electronic device including a memory, a processor and a network interface connected via a system bus, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to realize the intelligent detection method for knee joint images as described above.

[0024] On the other hand, the present invention provides a readable storage medium storing a computer program, which, when executed, implements the intelligent detection method for knee joint images as described above.

[0025] The beneficial effects of the method and system of the present invention are as follows: The present invention improves the YOLOx neural network by introducing a residual fusion module and constructs a recognition model, so that the recognition model solves the problem of model overfitting and enhances the representation ability of the knee osteoarthritis detection model while keeping the original model size almost unchanged, thereby improving the accuracy of knee osteoarthritis recognition.

[0026] This invention provides a method, apparatus, device, and readable storage medium for detecting knee osteoarthritis. The method includes acquiring an X-ray image of the knee osteoarthritis to be detected; inputting the X-ray image into a recognition model; processing the image using the model to obtain an output result, which is the recognition result of the X-ray image of the knee osteoarthritis to be detected. The output result includes whether the X-ray image of the knee osteoarthritis to be detected indicates knee osteoarthritis and the degree of knee osteoarthritis. This application provides intelligent recognition results for whether an X-ray image of the knee osteoarthritis to be detected indicates knee osteoarthritis and the degree of knee osteoarthritis, thereby helping doctors to make effective judgments, prevent misdiagnosis, and reduce related losses.

[0027] The present invention is simple and convenient to implement, highly practical, and solves the problems of low practicality and inconvenience in actual application of related technologies. It can improve user experience and has significant market value. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the steps of a YOLO-based method for identifying knee osteoarthritis provided in an embodiment of the present invention.

[0029] Figure 2 This is a structural block diagram of the recognition model provided in an embodiment of the present invention;

[0030] Figure 3 This is a structural block diagram of the residual fusion module provided in an embodiment of the present invention;

[0031] Figure 4 This is a structural block diagram of the multi-scale feature fusion layer provided in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the channel washing process provided in an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram of the structure of a knee osteoarthritis detection device provided in an embodiment of the present invention;

[0034] Figure 7This is a schematic diagram of the structure of a knee osteoarthritis identification device provided in an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. No limitation is made on the order of the steps; the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0036] like Figure 1 As shown, this embodiment of the invention provides an intelligent detection method for knee joint images, including the following steps:

[0037] 1) Perform image annotation and enhancement processing on X-ray data of knee osteoarthritis, and construct training and validation sets according to proportions;

[0038] Its implementation includes the following process,

[0039] 1.1) Obtain knee osteoarthritis X-ray images based on X-ray machine to obtain knee osteoarthritis X-ray data, and divide it into training set and validation set; label the knee osteoarthritis X-ray image data based on labelImg to obtain labeled image data, that is, both training set data and validation set data are accompanied by corresponding labels;

[0040] Specifically, it is recommended to use the labelImg tool to annotate knee osteoarthritis X-ray images in PascalVOC format according to the KL grading method for knee osteoarthritis.

[0041] 1.2) The labeled images are enhanced by flipping, Gaussian blurring, color gamut transformation, scaling and stitching to obtain the enhanced dataset;

[0042] Specifically, it is recommended to use the Mosaic data augmentation method to enhance the dataset during the training process for the original X-ray images of knee osteoarthritis. Four images are randomly selected from the training data and subjected to operations such as flipping, Gaussian blurring, and color gamut transformation. The images are then randomly scaled and stitched together to form a new image, which is then fed into the training network for feature extraction. This can enhance the target recognition model's ability to distinguish similar features, thereby improving recognition accuracy.

[0043] 2) Introduce a residual fusion module and build a recognition model based on the YOLOx neural network architecture framework;

[0044] This invention proposes to modify and prune the original YOLOx model to obtain a recognition model; then proceed to step 3).

[0045] 3) Train the recognition model based on the training set to obtain the trained recognition model;

[0046] The specific implementation of the recognition model obtained in this embodiment of the invention is as follows:

[0047] Specifically, the recognition model replaces the CSPDarknet module in the YOLOvx neural network framework with the ResxBlockNet (multi-combination residual fusion network) module and introduces a residual fusion module; for example... Figure 2 The recognition model includes a ResBlockNet module (multi-combination residual fusion network), an FPN (feature pyramid), and a YoloHead (detection head).

[0048] The initial features are fused based on Feature Pyramid Network (FPN) to output the feature results;

[0049] The identification result is obtained by predicting the feature results based on YoloHead.

[0050] like Figure 2 As shown, the feature transfer performed by the ResXBlockNet module via the Feature Pyramid (FPN) is divided into three parts.

[0051] The first part involves the data output from the 5th ResxB being directly transmitted to the first YoloHead.

[0052] The second part involves performing CBL and upsampling operations on the data output from the second and fifth ResxB modules, concatenating them, and then inputting them into five cascaded CBL modules before outputting them to the second YoloHead.

[0053] The third part consists of the data output from the first ResxB module and the data from the second part, which are concat-connected and then input into the five cascaded CBL modules. This data is then output to the third YoloHead for processing.

[0054] Specifically, for the introduced ResXBlockNet model, such as Figure 2As shown, the system includes an input layer, a CBL module, and five residual fusion modules (ResxB) to improve recognition accuracy. The input from the input layer is fed into the CBL layer. Based on CBL, the knee joint X-ray image undergoes convolution, batch sample normalization, and activation function processing to obtain the first feature. The input data is then processed according to a preset dimension to extract semantic information at different levels. This information is then sequentially fed into the five ResxB modules, which further subdivide the first feature to obtain corresponding features. The input image size is 640*640*3; after processing by the CBL module, its size is reduced to 320*320*32. The image processed by CBL is then output to the first ResxB module, resulting in a feature size of 160*160*64. The feature map is processed by the first ResxB module (160*160*64) to obtain an 80*80*128 feature map. After processing by the second ResxB module (80*80*128), it is processed by the third ResxB module (40*40*256 feature map). After processing by the third ResxB module (40*40*256), it is processed by the fourth ResxB module (20*20*512 feature map). After processing by the fourth ResxB module (20*20*512), it is processed by the fifth ResxB module (10*10*1024 feature map.

[0055] Residual fusion module (ResxB) such as Figure 3 As shown, the detection model solves the overfitting problem and enhances the representation ability of the knee osteoarthritis X-ray image detection model while keeping the original model size almost unchanged, thus improving the detection speed of the model while ensuring the recognition accuracy of knee osteoarthritis.

[0056] This invention proposes three improvement strategies for the Residual Fusion Module (ResxB): 1. A multi-scale feature fusion module is designed to fuse semantic information at different levels, separating more significant contextual features. 2. Residual connections are performed between the input features and the features after multi-scale feature fusion. 3. A max-pooling layer is introduced to ensure the extraction of more refined knee osteoarthritis features while reducing the computational load of the model. 4. A channel shuffle operation is added at the end of the residual fusion module to reorganize the fused features, improving the interaction between features.

[0057] See Figure 3Specifically, in the embodiment, it is preferred to set the first layer of the ResxB module as a CBL module and the second layer as a multi-scale feature fusion module. The output of the CBL module is input to the multi-scale feature fusion module. The features extracted by the CBL of the first layer are connected by ADD with the features extracted after multi-scale fusion. The feature maps are added together under the condition that the number of channels remains unchanged. Max pooling and channel shuffle operations are performed on the feature maps after ADD connection.

[0058] The multi-scale feature fusion module fuses semantic information at different levels, separates contextual feature information, and obtains the output feature map.

[0059] Based on channel shuffling, the preliminary feature maps are grouped and combined to obtain a preliminary feature matrix. The preliminary feature matrix is ​​then inverted and grouped again to obtain the output feature map. The CBL module consists of three parts: a convolutional layer (conv), batch normalization (BN), and an activation function (LeakyReLU). Through continuous feature transformation, the expressive dimension of the features is increased, enabling the fusion of semantic information at different levels. (Refer to...) Figure 4 The multi-scale feature fusion module enables better communication and more accurate recognition between deep and shallow information. This module consists of three different CBL modules with a stride of 1×1 and kernel sizes of 1×1, 3×3, and 5×5, respectively. The ADD operation is selected to connect the three CBL modules, keeping the number of channels unchanged while upsampling feature maps of different scales to the same size for fusion. This expands the tensor dimension and reduces information loss, improving the overall adaptability of the network to scale, making the output feature information richer, and improving the accuracy of the detection results.

[0060] Specifically, such as Figure 5 The channel shuffling method used in this invention can achieve information fusion between channels without increasing the computational load and parameter count, thereby enhancing the classification effect and better addressing the connection between the feature input and output of the two modules. For example... Figure 5 The channel shuffling module divides the input feature map into 'a' groups, with x input channels. It combines the two to form a new feature matrix (a, x), then performs a reverse transpose operation to transform the matrix into (x, a). Finally, it divides the matrix back into 'a' groups and splices them together to form a new feature map, which is then output to the next module.

[0061] For example Figure 5In the first module, there are three sets of feature maps, each with five channels. The combined feature maps form a new feature matrix (a, x) of (3, 5), a 3x5 matrix. This matrix is ​​then inverted to (5, 3), a 5x3 matrix. Finally, these matrices are regrouped and concatenated to form a new feature map, which is then output to the next module. The feature maps are fused using a Feature Pyramid Network (FPN). The fused features are then processed using a YoloHead detection head to output the recognition result.

[0062] See Figure 2 Specifically, the Feature Pyramid (FPN) module outputs the feature map from the ResXBlockNet (Multi-Combined Residual Fusion Network) module directly to the detection head (YoLoHead). At the same time, it performs CBL and upsampling operations on the output of the second ResxB module and concatenates the output through Concat and five CBL operations. Then, while directly outputting it to the detection head (YoLoHead), it performs CBL and upsampling operations again and fuses the feature map with the features output from the first ResxB through Concat and five CBL operations before outputting the recognition result to the detection head (YoLoHead).

[0063] Furthermore, the recognition model is validated based on the validation set. If the accuracy of the recognition result is greater than the preset threshold, the trained recognition model is obtained, ensuring high accuracy and sufficient generalization ability of the model.

[0064] 4) Input the newly acquired X-ray images of knee osteoarthritis into the trained recognition model and output the intelligent detection results of the knee joint images.

[0065] This step involves acquiring real-time X-ray images of knee osteoarthritis and inputting them into the trained recognition model, outputting recognition results for knee osteoarthritis. Specifically:

[0066] Real-time acquisition of X-ray images of knee osteoarthritis and input into the input layer of the trained recognition model;

[0067] The input data is processed by the ResXBlockNet module and the Feature Pyramid (FPN) module according to preset dimensions for feature extraction and feature fusion. The processing results are then output to the detection head (YoloHead) to output the identification results of knee osteoarthritis based on the KL grading.

[0068] As one possible embodiment, a smart detection device for knee joint images is also provided, such as... Figure 6 As shown, it includes:

[0069] The acquisition unit is used to acquire X-ray images of the knee osteoarthritis to be detected from the X-ray machine;

[0070] The diagnostic unit is used to input the X-ray image of the knee osteoarthritis to be detected into the knee osteoarthritis X-ray recognition model. The X-ray image to be detected is processed by the multi-combination residual fusion network ResXBlockNet, feature pyramid FPN and detection head YoloHead of the recognition model in sequence to obtain the output result. The output result is the recognition result of the X-ray image of the knee osteoarthritis to be detected. The output result includes whether the X-ray image to be detected is knee osteoarthritis and the grade of knee osteoarthritis.

[0071] An interactive unit that outputs model-based intelligent detection results to doctors and patients;

[0072] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device and each unit described above can be referred to the corresponding process in the aforementioned intelligent knee joint image detection embodiment, and will not be repeated here.

[0073] The apparatus provided in the above embodiments can be implemented as a computer program that can run on an osteoarthritis detection device.

[0074] As one possible embodiment, an intelligent detection electronic device for knee joint images is also provided, with reference to... Figure 7 It includes: a memory, a processor, and a network interface connected via a system bus, wherein the memory stores at least one instruction, and the processor loads and executes at least one instruction to implement all or part of the steps of the aforementioned osteoarthritis detection and identification method.

[0075] The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0076] The processor can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or any conventional processor. The processor is the control center of the computer device, connecting various parts of the device through various interfaces and lines. Memory is used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in memory, and by accessing data stored in memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for a function (such as image playback), etc.; the data storage area can store data created based on the use of the device (such as image data), etc. In addition, the memory may include high-speed mobile access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device or other volatile solid-state storage device.

[0077] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements all or part of the steps of the aforementioned intelligent detection method for knee joint images.

[0078] The embodiments of this application can implement all or part of the aforementioned processes, or they can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various methods described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, servers, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0082] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for intelligent detection of knee joint images, characterized in that: Includes the following steps, Step 1: Perform image annotation and enhancement processing on the X-ray data of knee osteoarthritis, and construct training and validation sets according to the proportions; Step 2: Introduce a residual fusion module and construct an X-ray recognition model for knee osteoarthritis based on the YOLOx neural network architecture framework. The recognition model replaces the CSPDarknet module in the YOLOx neural network architecture framework with a multi-combination residual fusion network ResXBlockNet, and introduces a residual fusion module into the multi-combination residual fusion network ResXBlockNet. The described X-ray recognition model for knee osteoarthritis includes a multi-combination residual fusion network ResXBlockNet, a feature pyramid FPN, and a detection head YoloHead. The multi-combination residual fusion network ResXBlockNet includes an input layer, a CBL module, and five residual fusion modules ResxB. The input from the input layer is passed to the CBL module, and then sequentially to the five ResxB modules. The feature transfer via the feature pyramid FPN in ResXBNet is divided into three parts. The first part involves directly transmitting the data output from the fifth residual fusion module, ResxB, to the first YoloHead; The second part involves performing CBL and upsampling operations on the data output from the second and fifth residual fusion modules ResxB, then connecting them via Concat and inputting them into five cascaded CBL modules, which are then output to the second YoloHead. The third part consists of the data output from the first residual fusion module ResxB block and the data after the second part is concat-connected and input into the five cascaded CBL modules. The data at this point is output to the third YoloHead for processing. In the ResxB module, the first layer is the CBL module, and the second layer is the multi-scale feature fusion module. The output of the CBL module is input to the multi-scale feature fusion module. The features extracted by the first-layer CBL module are connected to the features extracted after multi-scale fusion by ADD. The feature maps are added while the number of channels remains unchanged. Max pooling and channel shuffling operations are performed on the feature maps after ADD connection. Step 3: Based on the training set, train to obtain a trained X-ray recognition model for knee osteoarthritis; Step 4: Input the newly acquired X-ray images of knee osteoarthritis into the trained recognition model and output the intelligent detection results of the knee joint images.

2. The intelligent detection method for knee joint images according to claim 1, characterized in that: Step 1 is implemented by acquiring X-ray image data of the knee joint and dividing it into training and validation sets, and then labeling it to obtain labeled image data; the labeled images are then subjected to enhancement processing such as flipping, color gamut transformation, Gaussian blurring, scaling and stitching to obtain the enhanced dataset.

3. The intelligent detection method for knee joint images according to claim 1, characterized in that: The multi-scale feature fusion module consists of three different CBL modules with a stride of 1×1 and kernel sizes of 1×1, 3×3, and 5×5, respectively. The ADD operation is selected to connect the three CBL modules, keeping the number of channels unchanged while upsampling the feature maps of different scales to the same size for fusion.

4. A knee joint image intelligent detection device, characterized in that: This method is used to implement an intelligent detection method for knee joint images as described in any one of claims 1-3.

5. The intelligent detection device for knee joint images according to claim 4, characterized in that: It includes the following modules: an acquisition unit, used to acquire X-ray images of the knee osteoarthritis to be detected from an X-ray machine; The diagnostic unit is used to input the X-ray image of the knee osteoarthritis to be detected into the knee osteoarthritis X-ray recognition model. The X-ray image to be detected is processed by the multi-combination residual fusion network ResXBlockNet, feature pyramid FPN and detection head YoloHead of the recognition model in sequence to obtain the output result. The output result is the recognition result of the X-ray image of the knee osteoarthritis to be detected, including whether the X-ray image to be detected is knee osteoarthritis and the intelligent detection result of the grade of knee osteoarthritis. The interactive unit is used to output model-based intelligent detection results to doctors and patients.

6. An electronic device, characterized in that: The system includes a memory, a processor, and a network interface connected via a system bus. The memory stores at least one instruction, which is loaded and executed by the processor to implement the intelligent detection method for knee joint images as described in any one of claims 1-3.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed, implements the intelligent detection method for knee joint images as described in any one of claims 1-3.

Citation Information

Patent Citations

  • Automobile drivable area planning method based on multi-task neural network

    CN112418236A

  • Method and apparatus for extracting multi-modal POI feature

    WO2022198854A1