Bone joint image recognition method, device, electronic device and readable medium

Abstract

The present disclosure relates to a method, device, electronic device and readable medium for identifying bone joint images. The method comprises: performing resolution preprocessing and pixel value preprocessing on acquired medical images of bone joints, calling a pre-trained bone joint positioning model to identify bone joint images in medical images, calling a pre-trained bone joint suture segmentation model to segment bone joint suture images from bone joint images, stacking bone joint suture images with bone joint images to obtain dual-channel images, and obtaining osteoarthritis grading results by calling a pre-trained bone joint image recognition model to recognize the dual-channel images, thereby facilitating the detection of early symptoms of osteoarthritis, improving the accuracy of osteoarthritis recognition, avoiding complete reliance on human subjective judgment of osteoarthritis in medical images, and avoiding omissions that may be caused by human subjective judgment.

Description

Technical Field

[0001] The present disclosure relates to the field of medical image processing, and in particular, to a method, device, electronic device and readable medium for identifying bone and joint images. Background Art

[0002] Osteoarthritis (OA) is one of the leading causes of disability worldwide and one of the most common chronic diseases, accounting for 40-60% of patients with musculoskeletal degenerative diseases. In my country, about 50% of the elderly over 60 years old have osteoarthritis in their bone and joint X-ray images, and 30% to 50% of them have clinical symptoms. In addition to aging, the incidence of OA caused by trauma and obesity is also gradually increasing in young people.

[0003] At present, there is still a lack of reliable standards for the early diagnosis of OA. It is mainly based on the patient's symptoms, signs and imaging examinations, and there is no specific laboratory test. X-ray images are the imaging standard for diagnosing knee OA. The scoring system includes Kellgren and Lawrence (KL) grading, Osteoarthritis Research Society International (OARSI) grading, etc. The KL grading and OARSI grading are subject to subjective influences, and the accuracy of early grading of OA is not high. Therefore, the current diagnosis of OA mainly occurs in the moderate to severe (late) stage of the disease. The existing diagnostic method of OA relies entirely on the subjective judgment of doctors, and the efficiency of manual recognition is low. When the human eye is tired or the number of images is too large, some information is easily missed, resulting in inaccurate recognition. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, electronic device and readable medium for identifying bone and joint images, so as to solve the problem of low recognition efficiency and accuracy caused by the existing reliance on the subjective judgment of doctors.

[0005] In a first aspect, the present disclosure provides a method for bone joint images, comprising: acquiring a medical image of a bone joint; preprocessing the medical image; the preprocessing comprising resolution preprocessing and pixel value preprocessing; calling a pre-trained bone joint positioning model to identify the bone joint image in the medical image; the bone joint image is an image of the area where the bone joint is located in the medical image; the bone joint positioning model is obtained by training based on multiple groups of medical images of bone joints; calling a pre-trained bone joint suture segmentation model to segment a bone joint suture image from the bone joint image; the bone joint suture segmentation model is obtained by training based on the multiple groups of bone joint images; stacking the bone joint suture image with the bone joint image to obtain a dual-channel image; calling a pre-trained bone joint image recognition model to recognize the dual-channel image to obtain an image recognition result of the medical image.

[0006] Optionally, the resolution preprocessing includes: obtaining the resolution and pixel pitch of the medical image; obtaining the size of the medical image according to the resolution and the pixel pitch; and adjusting the medical image to a predetermined target size based on the size of the medical image.

[0007] Optionally, the step of adjusting the medical image to a predetermined target size includes: when the size of the medical image is larger than the target size, cropping the medical image to the target size; when the size of the medical image is smaller than the target size, setting the pixel value of the area between the edge of the medical image and the target size to a fixed value.

[0008] Optionally, the pixel value preprocessing includes: obtaining initial pixel values ​​of all pixels on the medical image; arranging the initial pixel values ​​from small to large to obtain a pixel value queue; obtaining a minimum pixel value and a maximum pixel value based on the pixel value queue; the minimum pixel value is the median value of a first proportional position in the pixel value queue, and the maximum pixel value is the median value of a second proportional position in the pixel value queue; obtaining a replacement value for each initial pixel value based on the initial pixel value, the minimum pixel value and the maximum pixel value; and replacing the initial pixel values ​​of all pixels with the replacement values ​​to obtain a normalized medical image.

[0009] Optionally, the calculation formula of the replacement value includes:

[0010]

[0011] Among them, i is the i-th pixel, I i0 is the initial pixel value, I min is the minimum pixel value, I max is the maximum pixel value.

[0012] Optionally, the medical image of the bone joint is a medical image of the knee joint; the bone joint positioning model is a knee joint positioning model; the step of calling a pre-trained bone joint positioning model to identify the bone joint image in the medical image includes: identifying the coordinates of the concave points of the raised part in the suture of the knee joint in the medical image of the knee joint; extracting the image within a predetermined range above, below, left and right of the coordinates as the knee joint image.

[0013] Optionally, after the step of preprocessing the medical image, the method further includes: scaling the resolution of the medical image to obtain a scaled image, and the scaled image is used for the bone joint positioning model to identify the bone joint image in the medical image.

[0014] Optionally, the bone joint positioning model is a knee joint positioning model, which is obtained by training based on training samples, and the training samples include multiple groups of medical images of knee joints; each group of medical images of knee joints includes medical images of left knee joints and medical images of right knee joints; the training method of the knee joint positioning model includes: scaling the resolution of the multiple groups of medical images of knee joints to obtain multiple groups of scaled medical images of knee joints; for each group of medical images of knee joints in the multiple groups of scaled medical images of knee joints, recording the first coordinates of the concave point of the convex part in the suture of the knee joint in the medical image of the left knee joint, and the medical image of the right knee joint. The second coordinates of the concave point of the raised part in the suture of the knee joint in the image; creating a first image in which all pixel values ​​are the first pixel values, and setting the pixel value of the first coordinate in the first image to the second pixel value; creating a second image in which all pixel values ​​are the first pixel values, and setting the pixel value of the second coordinate in the second image to the second pixel value; the resolution of the first image and the second image is the same as the resolution of the multiple groups of medical images of the knee joint after scaling; using the first image and the second image as labels of the training samples, so that when training the knee joint positioning model, the first image and the second image are used to respectively detect the knee joint image of the left knee joint and the knee joint image of the right knee joint.

[0015] In a second aspect, the present disclosure provides a device for bone joint images, comprising: an acquisition module for acquiring medical images of bone joints; a processing module for preprocessing the medical images; the preprocessing includes resolution preprocessing and pixel value preprocessing; a recognition module for calling a pre-trained bone joint positioning model to identify the bone joint image in the medical image; the bone joint image is an image of the area where the bone joint is located in the medical image; the bone joint positioning model is obtained by training based on multiple groups of medical images of bone joints; a segmentation module for calling a pre-trained bone joint suture segmentation model to segment the bone joint suture image from the bone joint image; the bone joint suture segmentation model is obtained by training based on the multiple groups of bone joint images; the processing module is also used to stack the bone joint suture image with the bone joint image to obtain a dual-channel image; the processing module is also used to call a pre-trained bone joint image recognition model to recognize the dual-channel image to obtain an image recognition result of the medical image.

[0016] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the aforementioned bone and joint image recognition method.

[0017] In a fourth aspect, the present disclosure provides a computer device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the aforementioned bone joint image recognition method.

[0018] Through the above technical scheme, the obtained medical images of bone joints are subjected to resolution preprocessing and pixel value preprocessing, the pre-trained bone joint positioning model is called to identify the bone joint images in the medical images, the pre-trained bone joint suture segmentation model is called to segment the bone joint suture images from the bone joint images, the bone joint suture images are stacked with the bone joint images to obtain a dual-channel image, and the osteoarthritis grading results are obtained by calling the pre-trained bone joint image recognition model to recognize the dual-channel image, which is conducive to detecting the early symptoms of osteoarthritis, improving the accuracy of osteoarthritis recognition, avoiding complete reliance on human subjective judgment of osteoarthritis in medical images, realizing automatic recognition and avoiding omissions that may be caused by human subjective judgment, thereby improving recognition efficiency and accuracy.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 It is a schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure.

[0022] Figure 2 The figure is a flow chart of a method for recognizing bone joint images according to an exemplary embodiment of the present disclosure.

[0023] Figure 3 is a flowchart of resolution preprocessing shown in an exemplary embodiment of the present disclosure.

[0024] Figure 4 4 is a flowchart of pixel value preprocessing shown in an exemplary embodiment of the present disclosure.

[0025] Figure 5 is an X-ray plain film image of a knee joint shown in an exemplary embodiment of the present disclosure.

[0026] Figure 6 is an image of a knee joint shown in an exemplary embodiment of the present disclosure.

[0027] Figure 7It is a flow chart of a training method for a knee joint positioning model shown in an exemplary embodiment of the present disclosure.

[0028] Figure 8 is a bone suture annotation diagram shown in an exemplary embodiment of the present disclosure.

[0029] Fig. 9 It is a block diagram of a bone joint image recognition device shown in an exemplary embodiment of the present disclosure.

[0030] Fig.10 It is a block diagram of an electronic device shown in an exemplary embodiment of the present disclosure.

[0031] Fig.11 is a block diagram of another electronic device according to an exemplary embodiment of the present disclosure.

[0032] Description of Reference Numerals

[0033] 120-terminal; 140-server; 20-device; 201-acquisition module; 203-processing module; 205-identification module; 207-segmentation module; 400-electronic device; 401-processor; 402-memory 403; -multimedia component; 404-input / output (I / O) interface; 405-communication component; 500-electronic device; 522-processor; 532-memory; 526-power supply component; 550-communication component; 558-input / output (I / O) interface. DETAILED DESCRIPTION

[0034] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0035] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0036] Before introducing the bone joint image recognition method, device, electronic device and readable medium provided by the present disclosure, the scenarios involved in the present disclosure are first introduced. The bone joint image recognition method provided by the present disclosure can be executed by a terminal or a server. It can also be executed by a system composed of a terminal and a server. For example, Figure 1 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown. The computer system includes a terminal 120 and a server 140 .

[0037] The terminal 120 and the server 140 are connected to each other via a wired or wireless network.

[0038] The terminal 120 may include at least one of a smart phone, a laptop computer, a desktop computer, a tablet computer, a smart speaker, and a smart robot.

[0039] The terminal 120 includes a display; the display is used to display the bone joint positioning results, the bone joint suture segmentation results and the osteoarthritis grading results.

[0040] The terminal 120 includes a first memory and a first processor. A first program is stored in the first memory; the first program is called and executed by the first processor to implement the bone joint image recognition method provided in the present disclosure. The first memory may include but is not limited to the following: Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electric Erasable Programmable Read-Only Memory (EEPROM).

[0041] The first processor may be composed of one or more integrated circuit chips. Optionally, the first processor may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). Optionally, the first processor may identify the bone joint image in the medical image by calling a pre-trained bone joint positioning model, or call a pre-trained bone joint suture segmentation model to segment the bone joint suture image from the bone joint image, or call a pre-trained bone joint image recognition model to recognize a dual-channel image to obtain an osteoarthritis grading result. Exemplarily, the various models called in the terminal may be obtained by training the terminal; or, may be obtained by training the server, and the terminal obtains it from the server.

[0042] Server 140 includes a second memory and a second processor. A second program is stored in the second memory, and the second program is called by the second processor to implement the recognition method of bone joint images provided by the present disclosure. Exemplarily, a pre-trained bone joint positioning model, a bone joint suture segmentation model and a bone joint image recognition model are stored in the second memory, and the pre-trained bone joint positioning model, the bone joint suture segmentation model and the bone joint image recognition model are called by the second processor to implement the recognition method of bone joint images. Optionally, the second memory may include but is not limited to the following: RAM, ROM, PROM, EPROM, EEPROM. Optionally, the second processor may be a general-purpose processor, such as a CPU or NP.

[0043] The server in each embodiment of the present disclosure may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal in each embodiment of the present disclosure may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure.

[0044] See also Figure 2 , Figure 2 A flowchart of a method for identifying a bone joint image provided by an exemplary embodiment of the present disclosure. The method is executed by a computer device, for example, Figure 1 The computer system shown is executed by a terminal or a server. Figure 2 The training method of the corpus classification model shown includes the following steps:

[0045] In step S101, a medical image of a bone joint is acquired.

[0046] Specifically, medical images of human bones and joints are acquired through relevant medical equipment. The medical images may be, but are not limited to, magnetic resonance imaging images, CT images, DR images, or X-ray images. In the disclosed embodiment, X-ray images are used as an example for illustration.

[0047] In step S102, the medical image is preprocessed.

[0048] It should be noted that the preprocessing of the acquired X-ray plain film image may include two steps: resolution preprocessing and pixel value preprocessing. The resolution preprocessing and pixel value preprocessing will be described in detail below.

[0049] See also Figure 3 , Figure 3 A flowchart of resolution preprocessing is provided for an exemplary embodiment of the present disclosure, as shown in FIG. Figure 3 As shown, step S102 may include the following steps.

[0050] In step S1021, the resolution and pixel pitch of the medical image are acquired.

[0051] For example, the resolution (x, y) and the pixel spacing (a, b) of the X-ray film image are obtained, where the pixel spacing refers to the distance between two pixels.

[0052] In step S1022 , the size of the medical image is acquired according to the resolution and pixel pitch.

[0053] The size of the medical image refers to the physical size of the X-ray image. The physical size of the X-ray image can be determined based on the resolution (x, y) and the pixel spacing (a, b). The length is x*a and the width is y*b. The length and width of the physical size can be rounded during specific calculations. The rounded length and width are expressed as ([x*a], [y*b]), where [] indicates taking an approximate integer.

[0054] In step S1023 , the medical image is adjusted to a predetermined target size based on the size of the medical image.

[0055] After obtaining the physical size of the X-ray film image, the physical size of the X-ray film image is adjusted to a predetermined target size. The predetermined target size can be obtained based on human experience or other feasible methods. For example, the resolution of the predetermined target size can be but is not limited to 448*448. In an optional embodiment, the resolution of the predetermined target size is an integer multiple of 32, so that the bone joint positioning model can downsample the X-ray film image multiple times, so that reasonable feature selection is made during the sampling process, and the accuracy of the positioning of the bone joint positioning model is ensured.

[0056] Adjusting the medical image to a predetermined target size includes: when the size of the medical image is larger than the target size, cropping the medical image to the target size; when the size of the medical image is smaller than the target size, setting the pixel value of the area between the edge of the medical image and the target size to a fixed value.

[0057] For example, in this embodiment, the resolution of the predetermined target size is 448*448 for explanation. For example, the resolution of the physical size of the X-ray film image is 501*501, and the portion with a resolution greater than 448 is cropped approximately in equal amounts on the left, right, top, and bottom sides of the X-ray film image, where one side is cropped (501-448) / / 2=26 rows of pixels, and the other side is cropped 501-26-448=27 rows of pixels, where / / represents the integer part of division. For another example, if the resolution of the physical size of the X-ray film image is 400*400, then one of the upper and lower sides and one of the left and right sides of the X-ray film image is supplemented with (448-400)=48 rows of pixel values, that is, these 48 rows of pixel values ​​are set to fixed values. In an optional implementation, the fixed value may be 0.

[0058] See also Figure 4 , Figure 4 A flowchart of pixel value preprocessing is provided for an exemplary embodiment of the present disclosure, as shown in FIG. Figure 4 As shown, step S102 may include the following steps.

[0059] In step S1025, initial pixel values ​​of all pixels on the medical image are obtained.

[0060] Specifically, the initial pixel value I0 of all pixel points on the X-ray film image is obtained.

[0061] In step S1026, the initial pixel values ​​are arranged from small to large to obtain a pixel value queue.

[0062] Arrange all initial pixel values ​​I0 on the X-ray film image from small to large to obtain a pixel value queue.

[0063] In step S1027, the minimum pixel value and the maximum pixel value are obtained based on the pixel value queue.

[0064] Specifically, the minimum pixel value is the median value of the first proportional position in the pixel value queue, and the maximum pixel value is the median value of the second proportional position in the pixel value queue; in an optional embodiment, the first proportional position is 2%, that is, the minimum pixel value is the pixel value at 2% in the pixel value queue, if there are two pixel values ​​at 2%, the median value of these two pixel values ​​is taken as the minimum pixel value; in an optional embodiment, the second proportional position is 98%, that is, the maximum pixel value is the pixel value at 98% in the pixel value queue, if there are two pixel values ​​at 98%, the median value of these two pixel values ​​is taken as the maximum pixel value.

[0065] In step S1028, a replacement value for each initial pixel value is obtained based on the initial pixel value, the minimum pixel value and the maximum pixel value.

[0066] The calculation formula for the replacement value of the initial pixel value includes:

[0067]

[0068] Where i is the i-th pixel point, I1 represents the replacement value of the first initial pixel value, I2 represents the replacement value of the second initial pixel value, and I i represents the replacement value of the i-th initial pixel value, and so on; I i0 is the initial pixel value, I 10 Represents the first initial pixel value, I 20 Represents the second initial pixel value, I i0 represents the i-th initial pixel value, and so on; I min is the minimum pixel value, I max is the maximum pixel value.

[0069] In step S1029, the initial pixel values ​​of all pixels are replaced with replacement values ​​to obtain a normalized medical image.

[0070] Replace the initial pixel values ​​of all pixels with the replacement value, for example, replace I with I1 10 , replace I with I2 20 , use I i Replace I i0 , and so on, which will not be elaborated in the present disclosure, a normalized X-ray film image is obtained after all the pixels are replaced.

[0071] It should be noted that there is no order of execution between steps S1021-S1023 and steps S1025-S1029, that is, steps S1021-S1023 can be executed first and then steps S1025-S1029, or steps S1025-S1029 can be executed first and then steps S1021-S1023.

[0072] It should be noted that after preprocessing the preprocessed X-ray film image, the X-ray film image can also be scaled. For example, the original X-ray film image with a resolution of 448*448 can be scaled to a resolution of 112*112. The scaling process can not only reduce the data processing amount of the computer equipment, but also meet the recognition accuracy requirements of the bone joint positioning model.

[0073] In step S103, a pre-trained bone joint positioning model is called to recognize bone joint images in medical images.

[0074] Bone joint images are images of the areas where bone joints are located in medical images.

[0075] In an optional embodiment, the bone joint positioning model is a knee joint positioning model, and the X-ray plain film image is as follows: Figure 5 The X-ray plain film image of the knee joint shown in the figure, the method of calling the pre-trained knee joint positioning model to identify the knee joint image in the X-ray plain film image includes: identifying the coordinates of the concave point 11 of the raised part 10 in the suture of the knee joint in the medical image of the knee joint, and then extracting the image within the predetermined range above, below, left and right of the coordinate as the knee joint image. The predetermined range can be obtained based on human experience or based on other feasible methods. For example, the predetermined range can be but not limited to 32 pixels, 64 pixels, etc. If the predetermined range is 64 pixels, then the resolution of the extracted knee joint image is 128*128. If the resolution of the knee joint image extracted according to the predetermined range exceeds the resolution of the original X-ray plain film image, then the excess part can be zero-filled, that is, the pixel value of the excess part is set to 0, and the image after the zero-filling operation is used as the knee joint image, such as Figure 6 As shown, Figure 6 An image of a knee joint is provided as an exemplary embodiment of the present disclosure.

[0076] In addition, the left knee joint in the X-ray film image is the right knee joint of a human and does not require any processing; the right knee joint in the X-ray film image is the left knee joint of a human and the X-ray film image of the right knee joint needs to be mirrored to ensure that the outer side of the knee joint is on the left side of the knee joint image and the inner side is on the right side of the knee joint image.

[0077] It should be noted that the knee joint positioning model is obtained based on training samples, which include multiple groups of X-ray images of knee joints, and each group of X-ray images of knee joints includes X-ray images of left knee joints and X-ray images of right knee joints.

[0078] The training method of the knee joint positioning model will be explained in detail below, please refer to Figure 7 , Figure 7 A flowchart of a training method for a knee joint positioning model provided as an exemplary embodiment of the present disclosure.

[0079] In step S1031, the resolution of the multiple groups of medical images of the knee joints is scaled to obtain the multiple groups of scaled medical images of the knee joints.

[0080] In this embodiment, the medical image of the knee joint is taken as an X-ray film image for illustration, and the X-ray film image is scaled to obtain multiple groups of scaled medical images of the knee joint. For example, the X-ray film image with an original resolution of 448*448 is scaled to a resolution of 112*112. This step is consistent with the scaling processing content mentioned above, and the details can also be referred to above.

[0081] In step S1032, for each group of medical images of knee joints in the multiple groups of scaled medical images of knee joints, the first coordinates of the concave point of the convex part in the knee joint suture in the medical image of the left knee joint and the second coordinates of the concave point of the convex part in the knee joint suture in the medical image of the right knee joint are recorded.

[0082] In step S1033, a first image whose pixel values ​​are all the first pixel values ​​is created, and the pixel value of the first coordinate in the first image is set to the second pixel value; a second image whose pixel values ​​are all the first pixel values ​​is created, and the pixel value of the second coordinate in the second image is set to the second pixel value.

[0083] It should be noted that the resolution of the first image and the second image is the same as the resolution of the scaled multiple sets of medical images of the knee joint.

[0084] In an optional implementation, the first pixel value is 0 and the second pixel value is 1; in another optional implementation, the first pixel value is 0 and the second pixel value is 255. The first pixel value and the second pixel value may also be other feasible values, which are not limited in the present disclosure.

[0085] In step S1034, the first image and the second image are used as labels of training samples, so that when training the knee joint positioning model, the first image and the second image are used to respectively detect the knee joint image of the left knee joint and the knee joint image of the right knee joint.

[0086] For example, a network structure similar to U-Net can be selected as a convolutional neural network for the knee joint positioning model, which includes a downsampling layer (maximum pooling layer) and an upsampling layer (deconvolution layer), and is connected in the middle by stacking (Concatenate). Each downsampling layer and upsampling layer is followed by 2 convolution blocks, and the stacking connection is followed by 1 convolution block, and each convolution block includes two-dimensional convolution (2D Conv), batch normalization (Batch normalization) and nonlinear activation (ReLU).

[0087] In step S104, a pre-trained bone joint and bone suture segmentation model is called to segment a bone joint and bone suture image from the bone joint image.

[0088] The bone joint suture image is an image of the area where the bone joint suture is located in the bone joint image.

[0089] In this embodiment, an X-ray image of a knee joint is used as an example for explanation, and a knee joint suture segmentation model is used as an example for explanation. The knee joint suture image is an image of the region where the knee joint suture is located in the knee joint image.

[0090] The knee joint suture segmentation model is trained based on multiple sets of knee joint images.

[0091] Exemplarily, the knee joint suture segmentation model can adopt a network structure similar to U-Net. The input resolution of the network structure can be 128*128. The input image of the knee joint suture segmentation model is the knee joint image after the above-mentioned preprocessing or scaling. The knee joint suture segmentation model is annotated in a manner that a third image with the same resolution as the knee joint image after the above-mentioned preprocessing or scaling is created, for example, 128*128; in one embodiment, the pixel values ​​of the third image can be set to 0, and the pixel values ​​of the pixels corresponding to the knee joint sutures on the third image can be set to 1; in another embodiment, the pixel values ​​of the third image can be set to 0, and the pixel values ​​of the pixels corresponding to the knee joint sutures on the third image can be set to 255; then the annotated third image is obtained, and the annotated third image is used as the suture annotation map, such as Figure 8 As shown, Figure 8 A suture annotation map is provided for an exemplary embodiment of the present disclosure. The suture annotation map is then used to train a knee joint suture segmentation model to obtain a trained knee joint suture segmentation model, and a segmented knee joint suture image is obtained by inputting a knee joint image into the trained knee joint suture segmentation model.

[0092] In step S105, the bone joint suture image and the bone joint image are stacked to obtain a dual-channel image.

[0093] Specifically, the knee joint image and the knee joint suture image are stacked by using relevant image processing software to obtain a dual-channel image.

[0094] In step S106, a pre-trained bone and joint image recognition model is called to recognize the dual-channel image to obtain an image recognition result of the medical image.

[0095] In this embodiment, the bone and joint image recognition model may be an osteoarthritis grading model, the osteoarthritis grading model may be a knee osteoarthritis grading model, and the image recognition result may be an osteoarthritis grading result corresponding to the medical image.

[0096] The following description will be made using the knee osteoarthritis grading model as an example, and the osteoarthritis grading as the knee joint KL grading as an example.

[0097] The aforementioned dual-channel image is used as the input image and input into the pre-trained knee osteoarthritis grading model. Exemplarily, a network structure similar to Resnet18 can be selected as the convolutional neural network of the knee osteoarthritis grading model, and the network structure includes 5 downsampling layers (maximum pooling layers). Each downsampling layer is followed by 2 convolution blocks, each convolution block includes 2-dimensional convolution, batch normalization and nonlinear activation, and the convolution block also includes a residual structure. The knee joint KL grading features obtained by the network structure are sent to the fully connected layer for classification to obtain the knee joint KL grading results.

[0098] In summary, the bone joint image recognition method provided by the present invention performs resolution preprocessing and pixel value preprocessing on the acquired medical images of bone joints, calls a pre-trained bone joint positioning model to recognize the bone joint images in the medical images, calls a pre-trained bone joint suture segmentation model to segment the bone joint suture images from the bone joint images, stacks the bone joint suture images with the bone joint images to obtain a dual-channel image, and obtains an osteoarthritis grading result by calling a pre-trained bone joint image recognition model to recognize the dual-channel image, which is beneficial to detect the early symptoms of osteoarthritis, improves the accuracy of osteoarthritis recognition, avoids relying entirely on human subjective judgment of osteoarthritis in medical images, and avoids omissions that may be caused by human subjective judgment.

[0099] Fig. 9 FIG. 1 is a block diagram of a bone joint image recognition device shown in an exemplary embodiment of the present disclosure. Fig. 9 The device 20 includes an acquisition module 201, a processing module 203, an identification module 205 and a segmentation module 207.

[0100] The acquisition module 201 is used to acquire medical images of bone joints;

[0101] The processing module 203 is used to preprocess the medical image; the preprocessing includes resolution preprocessing and pixel value preprocessing;

[0102] The recognition module 205 is used to call a pre-trained bone joint positioning model to identify a bone joint image in the medical image; the bone joint image is an image of the area where the bone joint is located in the medical image; the bone joint positioning model is obtained by training based on multiple groups of medical images of bone joints;

[0103] The segmentation module 207 is used to call a pre-trained bone joint and suture segmentation model to segment a bone joint and suture image from the bone joint image; the bone joint and suture segmentation model is obtained by training based on the multiple groups of bone joint images;

[0104] The processing module 203 is further used to stack the bone joint suture image and the bone joint image to obtain a dual-channel image;

[0105] The processing module 203 is also used to call a pre-trained bone and joint image recognition model to recognize the dual-channel image and obtain an image recognition result of the medical image.

[0106] Optionally, the processing module 203 includes:

[0107] A resolution preprocessing submodule, used to obtain the resolution and pixel spacing of the medical image;

[0108] The resolution preprocessing submodule is also used to obtain the size of the medical image according to the resolution and the pixel spacing;

[0109] The resolution preprocessing submodule is further used to adjust the medical image to a predetermined target size based on the size of the medical image.

[0110] Optionally, the resolution preprocessing submodule is further used for:

[0111] When the size of the medical image is larger than the target size, cropping the medical image to the target size;

[0112] When the size of the medical image is smaller than the target size, the pixel value of the area between the edge of the medical image and the target size is set to a fixed value.

[0113] Optionally, the processing module 203 further includes:

[0114] A pixel value preprocessing submodule, used to obtain initial pixel values ​​of all pixels on the medical image;

[0115] The pixel value preprocessing submodule is also used to arrange the initial pixel values ​​from small to large to obtain a pixel value queue;

[0116] The pixel value preprocessing submodule is further used to obtain a minimum pixel value and a maximum pixel value based on the pixel value queue; the minimum pixel value is a median value of a first proportional position in the pixel value queue, and the maximum pixel value is a median value of a second proportional position in the pixel value queue;

[0117] The pixel value preprocessing submodule is further used to obtain a replacement value for each of the initial pixel values ​​based on the initial pixel value, the minimum pixel value and the maximum pixel value;

[0118] The pixel value preprocessing submodule is also used to replace the initial pixel values ​​of all pixels with replacement values ​​to obtain a normalized medical image.

[0119] Optionally, the calculation formula of the replacement value includes:

[0120]

[0121] Among them, i is the i-th pixel, I i0 is the initial pixel value, I min is the minimum pixel value, I max is the maximum pixel value.

[0122] Optionally, the identification module 205 is used to:

[0123] Identify the coordinates of the concave points of the convex part in the suture of the knee joint in the medical image of the knee joint;

[0124] An image within a predetermined range of the coordinates above, below, left, and right is extracted as a knee joint image.

[0125] Optionally, the processing module 203 is also used to scale the resolution of the medical image after the step of preprocessing the medical image to obtain a scaled image, and the scaled image is used for the bone joint positioning model to identify the bone joint image in the medical image.

[0126] Optionally, the bone joint positioning model is a knee joint positioning model, and the knee joint positioning model is obtained by training based on training samples, and the training samples include multiple groups of medical images of knee joints; each group of medical images of knee joints includes a medical image of a left knee joint and a medical image of a right knee joint; the processing module 203 is also used to scale the resolution of the multiple groups of medical images of knee joints to obtain the scaled multiple groups of medical images of knee joints;

[0127] For each group of medical images of knee joints in the multiple groups of medical images of knee joints after scaling, record the first coordinates of the concave point of the convex part in the suture of the knee joint in the medical image of the left knee joint, and the second coordinates of the concave point of the convex part in the suture of the knee joint in the medical image of the right knee joint;

[0128] Create a first image whose pixel values ​​are all first pixel values, and set the pixel value of the first coordinate in the first image to the second pixel value;

[0129] Creating a second image whose pixel values ​​are all the first pixel values, and setting the pixel value of the second coordinate in the second image to the second pixel value; the resolution of the first image and the second image is the same as the resolution of the scaled multiple groups of knee joint medical images;

[0130] The first image and the second image are used as labels of the training samples, so that when training the knee joint positioning model, the first image and the second image are used to respectively detect the knee joint image of the left knee joint and the knee joint image of the right knee joint.

[0131] Fig.10 FIG. 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. Fig.10 As shown, the electronic device 400 may include: a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output (I / O) interface 404, and a communication component 405.

[0132] The processor 401 is used to control the overall operation of the electronic device 400 to complete all or part of the steps in the above-mentioned bone joint image recognition method. The memory 402 is used to store various types of data to support the operation of the electronic device 400, and these data may include instructions for any application or method used to operate on the electronic device 400, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 402 or sent through the communication component 405. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 404 provides an interface between the processor 401 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 405 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0133] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned bone joint image recognition method.

[0134] In another exemplary embodiment, a computer-readable medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for identifying bone joint images are implemented. For example, the computer-readable medium can be the above-mentioned memory 402 including program instructions, and the above-mentioned program instructions can be executed by the processor 401 of the electronic device 400 to complete the above-mentioned method for identifying bone joint images.

[0135] Fig.11 is a block diagram of another electronic device 500 according to an exemplary embodiment. For example, the electronic device 500 may be provided as a server. Fig.11 The electronic device 500 includes a processor 522, which may be one or more, and a memory 532 for storing a computer program executable by the processor 522. The computer program stored in the memory 532 may include one or more modules each corresponding to a set of instructions. In addition, the processor 522 may be configured to execute the computer program to perform the above-mentioned bone joint image recognition method.

[0136] In addition, the electronic device 500 may further include a power supply component 526 and a communication component 550. The power supply component 526 may be configured to perform power management of the electronic device 500, and the communication component 550 may be configured to implement communication of the electronic device 500, for example, wired or wireless communication. In addition, the electronic device 500 may further include an input / output (I / O) interface 558. The electronic device 500 may operate based on an operating system stored in the memory 532, such as Windows Server 2008. TM , Mac OSX TM , Unix TM , Linux TM etc.

[0137] In another exemplary embodiment, a computer-readable medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for identifying bone joint images are implemented. For example, the non-temporary computer-readable medium can be the above-mentioned memory 532 including program instructions, and the above-mentioned program instructions can be executed by the processor 522 of the electronic device 500 to complete the above-mentioned method for identifying bone joint images.

[0138] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned bone joint image recognition method when executed by the programmable device.

[0139] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0140] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0141] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for identifying bone joint images, characterized in that: include: Obtain medical images of bone joints; Preprocessing the medical image; the preprocessing includes resolution preprocessing and pixel value preprocessing; Calling a pre-trained bone joint positioning model to identify the bone joint image in the medical image; The bone joint image is an image of the area where the bone joint is located in the medical image; the bone joint positioning model is obtained by training based on multiple groups of bone joint medical images; wherein the bone joint medical image is a medical image of a knee joint; the bone joint positioning model is a knee joint positioning model; the knee joint positioning model is obtained by training based on training samples, and the training samples include multiple groups of knee joint medical images; each group of knee joint medical images includes a left knee joint medical image and a right knee joint medical image; Calling a pre-trained bone joint and suture segmentation model to segment a bone joint and suture image from the bone joint image; the bone joint and suture segmentation model is obtained by training based on the multiple groups of bone joint images; Stacking the bone joint suture image and the bone joint image to obtain a dual-channel image; A pre-trained bone and joint image recognition model is called to recognize the dual-channel image to obtain an image recognition result of the medical image; the bone and joint image recognition model is an osteoarthritis grading model, and the image recognition result is used to characterize the osteoarthritis grading result corresponding to the medical image.

2. The method according to claim 1, characterized in that: The resolution preprocessing includes: Acquiring the resolution and pixel spacing of the medical image; Acquire the size of the medical image according to the resolution and the pixel pitch; Based on the size of the medical image, the medical image is adjusted to a predetermined target size.

3. The method according to claim 2, characterized in that The step of adjusting the medical image to a predetermined target size comprises: When the size of the medical image is larger than the target size, cropping the medical image to the target size; When the size of the medical image is smaller than the target size, the pixel value of the area between the edge of the medical image and the target size is set to a fixed value.

4. The method according to claim 1, characterized in that The pixel value preprocessing comprises: Obtaining initial pixel values ​​of all pixels on the medical image; Arrange the initial pixel values ​​from small to large to obtain a pixel value queue; A minimum pixel value and a maximum pixel value are obtained based on the pixel value queue; the minimum pixel value is a median value of a first proportional position in the pixel value queue, and the maximum pixel value is a median value of a second proportional position in the pixel value queue; Obtaining a replacement value for each of the initial pixel values ​​based on the initial pixel value, the minimum pixel value, and the maximum pixel value; The initial pixel values ​​of all pixels are replaced by replacement values ​​to obtain a normalized medical image.

5. The method according to claim 4, characterized in that The calculation formula of the replacement value includes: in, is the i-th pixel, is the initial pixel value, is the minimum pixel value, is the maximum pixel value.

6. The method according to claim 1, characterized in that The step of calling a pre-trained bone joint positioning model to identify the bone joint image in the medical image comprises: Identify the coordinates of the concave points of the convex part in the suture of the knee joint in the medical image of the knee joint; An image within a predetermined range of the coordinates above, below, left, and right is extracted as a knee joint image.

7. The method according to claim 1, characterized in that After the step of preprocessing the medical image, the method further includes: The resolution of the medical image is scaled to obtain a scaled image, and the scaled image is used by the bone joint positioning model to identify the bone joint image in the medical image.

8. The method according to claim 1, characterized in that The training method of the knee joint positioning model comprises: Scaling the resolution of the plurality of sets of medical images of knee joints to obtain the scaled plurality of sets of medical images of knee joints; For each group of medical images of knee joints in the multiple groups of medical images of knee joints after scaling, record the first coordinates of the concave point of the convex part in the suture of the knee joint in the medical image of the left knee joint, and the second coordinates of the concave point of the convex part in the suture of the knee joint in the medical image of the right knee joint; Create a first image whose pixel values ​​are all first pixel values, and set the pixel value of the first coordinate in the first image to the second pixel value; Creating a second image whose pixel values ​​are all the first pixel values, and setting the pixel value of the second coordinate in the second image to the second pixel value; the resolution of the first image and the second image is the same as the resolution of the scaled multiple groups of knee joint medical images; The first image and the second image are used as labels of the training samples, so that when training the knee joint positioning model, the first image and the second image are used to respectively detect the knee joint image of the left knee joint and the knee joint image of the right knee joint.

9. A bone joint image recognition device, characterized in that: include: An acquisition module, used for acquiring medical images of bone joints; A processing module, used for preprocessing the medical image; The preprocessing includes resolution preprocessing and pixel value preprocessing; A recognition module, used for calling a pre-trained bone joint positioning model to recognize a bone joint image in the medical image; The bone joint image is an image of the area where the bone joint is located in the medical image; The bone joint positioning model is obtained by training based on multiple groups of medical images of bone joints; A segmentation module, used for calling a pre-trained bone joint suture segmentation model to segment a bone joint suture image from the bone joint image; the bone joint suture segmentation model is obtained by training based on the multiple groups of bone joint images; wherein the medical image of the bone joint is a medical image of the knee joint; the bone joint positioning model is a knee joint positioning model; the knee joint positioning model is obtained by training based on training samples, and the training samples include multiple groups of knee joint medical images; each group of knee joint medical images includes a left knee joint medical image and a right knee joint medical image; The processing module is further used to stack the bone joint suture image and the bone joint image to obtain a dual-channel image; The processing module is also used to call a pre-trained bone and joint image recognition model to identify the dual-channel image and obtain an image recognition result of the medical image; the bone and joint image recognition model is an osteoarthritis grading model, and the image recognition result is used to characterize the osteoarthritis grading result corresponding to the medical image.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.

Images