Osteoarthritis diagnosis method and system based on artificial intelligence

Abstract

The invention discloses an osteoarthritis diagnosis method and system based on artificial intelligence. The osteoarthritis diagnosis method based on artificial intelligence comprises the steps that the osteoarthritis diagnosis system receives a diagnosis target image of a shot bone joint and inputs the diagnosis target image; a step in which the osteoarthritis diagnosis system outputs determination information on a reference for at least one of osteophyte, articular space, sclerosis, and bone deformation, which is a reference for osteoarthritis diagnosis, with respect to the inputted diagnosis target image using a deep learning model; and a step in which the osteoarthritis diagnosis system generates an osteoarthritis diagnosis result of the diagnosis target image on the basis of the determination information of each reference that has been output.

Description

Technical Field

[0001] The present invention relates to a method and a system for effectively diagnosing osteoarthritis based on artificial intelligence. Background Art

[0002] Osteoarthritis refers to a disease that damages bone and joint tissues (eg, knee joints). As one of the most common forms of arthritis, it is a disease that often occurs in the elderly population.

[0003] It is generally caused by damage to the cartilage or surrounding tissue at the bone ends, leading to pain, stiffness, and loss of function.

[0004] This type of osteoarthritis needs to be diagnosed and graded according to its degree of progression, and different treatments or prescriptions are given. The KL (Kellgren-Lawrence grade) is generally used.

[0005] KL grade is a method of diagnosis using conventional X-rays, and its grades can be divided into grade 0, grade 1, grade 2, grade 3 and grade 4.

[0006] Grade 0 may mean a state in which there are no obvious X-ray changes in the bone joints, that is, a healthy state, and Grade 1 may mean a state in which the joint space (joint space) is suspected to be narrow and there is a possibility of osteophyte lip deformation. Grade 2 may mean a state in which osteophytes are obvious and the joint space is narrowed, and Grade 3 may mean severe multiple osteophytes, obvious narrowing of the joint space, and partial sclerosis. Grade 4 may mean a large number of multiple osteophytes and significant narrowing of the joint space, severe sclerosis, and bone deformation.

[0007] The larger the KL level, the more serious the progression of osteoarthritis. Treatment or prescription will also vary according to the KL level, so it is extremely important to accurately determine the degree of the current osteoarthritis.

[0008] Figure 1 A diagram depicting how osteoarthritis was previously diagnosed.

[0009] Figure 1 To illustrate the bone joints according to their grades, x-rays of osteoarthritis grades 0 to 4 are shown as examples, starting from the left.

[0010] In the past, osteoarthritis was diagnosed by doctors (or diagnostic experts) visually confirming Figure 1 The patient's X-ray photograph shown is compared with the osteoarthritis diagnostic grade judgment criteria described above to subjectively determine the patient's grade.

[0011] Recently, attempts are being made to perform this diagnosis based on artificial intelligence using deep learning models, i.e., artificial intelligence.

[0012] However, this attempt is nothing more than a simple AI-based diagnosis that was previously performed by doctors. That is, the training data is only the bone and joint X-ray images and the corresponding bone and joint diagnosis levels. The deep learning model trained with this training data cannot know which of the criteria used to determine the diagnosis level (for example, joint space, osteophytes, degree of sclerosis, bone deformation, etc.) is used to mark the corresponding diagnosis level. Therefore, not only is its accuracy relatively low, but it must be trained with a relatively large amount of training data to ensure accuracy to a certain extent. This has the problem of requiring a lot of resources to build training data.

[0013] Therefore, an artificial intelligence-based osteoarthritis diagnosis method and system are required, which can solve such problems while improving the accuracy and explaining the diagnosis results.

[0014] [Prior art literature]

[0015] [Patent Literature]

[0016] - Application number of Korean patent application (10-2021-0144474, "Automatic measurement system of radiological indicators for musculoskeletal system diseases using artificial intelligence") Summary of the invention

[0017] Technical issues

[0018] The technical problem to be achieved by the present invention is to provide a technical concept that can diagnose osteoarthritis relatively accurately and effectively based on artificial intelligence.

[0019] In particular, a technical concept is provided for performing diagnosis using a deep learning model that has been trained to output judgment information for each item in the judgment criteria for diagnosing osteoarthritis, thereby achieving relatively accurate diagnosis and deriving explainable diagnostic results.

[0020] Technical Solution

[0021] The artificial intelligence-based osteoarthritis diagnosis method according to the technical concept of the present invention includes: a step in which the osteoarthritis diagnosis system receives an input of a diagnostic target image of a bone joint; a step in which the osteoarthritis diagnosis system uses a deep learning model to output, for each of the input diagnostic target images, judgment information of at least one of osteophytes, joint spaces, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis; and a step in which the osteoarthritis diagnosis system generates an osteoarthritis diagnosis result for the diagnostic target image based on the output judgment information of each benchmark.

[0022] The osteoarthritis diagnostic system utilizes a deep learning model to output, for an input diagnostic target image, respectively judgment information of at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis. The steps may include: the osteoarthritis diagnostic system judging the femoral axis based on the diagnostic target image; the osteoarthritis diagnostic system judging the tibial axis based on the diagnostic target image; and the osteoarthritis diagnostic system outputting judgment information of each benchmark including bone deformation information based on the judged femoral axis and tibial axis.

[0023] The step of the osteoarthritis diagnostic system determining the femoral axis based on the diagnostic target image may include: the osteoarthritis diagnostic system determining the epiphyseal point at a predetermined position of the bone cortex of the femoral epiphysis in the diagnostic target image; the osteoarthritis diagnostic system determining two reference points corresponding to the lateralmost and medialmost positions of the distal end of the femur; and the osteoarthritis diagnostic system determining the information of the femoral axis based on the midpoint of a first line segment that is perpendicular to the determined epiphyseal point and located on the femur and the midpoint of a second line segment connecting the two reference points.

[0024] The osteoarthritis diagnostic system uses a deep learning model to output, for an input diagnostic target image, judgment information of at least one of osteophytes, joint spaces, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis. The steps may include: the osteoarthritis diagnostic system divides the diagnostic target image into multiple joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia; the osteoarthritis diagnostic system determines osteophyte information corresponding to the osteophytes according to the multiple joint areas; and the osteoarthritis diagnostic system outputs judgment information of each benchmark including the determined osteophyte information.

[0025] The osteoarthritis diagnostic system uses a deep learning model to output, for an input diagnostic target image, respectively, judgment information of at least one of osteophytes, joint gaps, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis. The steps may include: the osteoarthritis diagnostic system determines, based on the diagnostic target image, at least one protrusion point corresponding to the maximum protrusion point at the lower end of the distal femur; determines the distance from the protrusion point to a predetermined point of the tibia; and outputs judgment information of each benchmark including information on the joint gap based on the determined distance.

[0026] The osteoarthritis diagnostic system uses a deep learning model to output, for an input diagnostic target image, judgment information of at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis. The steps may include: the osteoarthritis diagnostic system judges the hardening information corresponding to the hardened area at the distal end of the femur or tibia based on the diagnostic target image; and the osteoarthritis diagnostic system outputs the judgment information of each benchmark including the hardening information.

[0027] According to another aspect of the present invention, the osteoarthritis diagnosis method includes: a step in which the osteoarthritis diagnosis system receives an input of a diagnostic target image of a bone joint; a step in which the osteoarthritis diagnosis system uses a deep learning model to determine the femoral axis of the diagnostic target image; a step in which the osteoarthritis diagnosis system determines the tibial axis of the diagnostic target image; and a step in which the osteoarthritis diagnosis system determines bone deformation information based on the determined femoral axis and tibial axis.

[0028] According to another aspect of the present invention, an osteoarthritis diagnosis method may include: a step in which an osteoarthritis diagnosis system receives an input of a diagnostic target image of a bone joint; a step in which the osteoarthritis diagnosis system uses a deep learning model to divide the diagnostic target image into a plurality of joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia; and a step in which the osteoarthritis diagnosis system determines osteophyte information corresponding to osteophytes according to the plurality of joint areas.

[0029] The method can be implemented in a data processing device by means of a computer program recorded in a computer-readable recording medium.

[0030] According to one embodiment of the present invention, an artificial intelligence-based osteoarthritis diagnostic system includes: a processor; a storage device storing a program; wherein the processor drives the program to accept an input of a diagnostic target image of a bone joint, and uses a deep learning model to output, for the input diagnostic target image, at least one of osteophytes, joint spaces, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis, respectively, and generates an osteoarthritis diagnostic result for the diagnostic target image based on the output judgment information of each benchmark.

[0031] The processor can drive the program to determine the femoral axis based on the diagnostic target image, and the osteoarthritis diagnostic system determines the tibial axis based on the diagnostic target image, and based on the determined femoral axis and tibial axis, outputs the judgment information of each benchmark including bone deformation information.

[0032] The processor can drive the program to determine the epiphyseal point at a predetermined position of the cortical bone of the femoral epiphysis in the diagnostic target image, determine two reference points corresponding to the lateralmost and medialmost positions of the distal end of the femur, and judge the information of the femoral axis based on the midpoint of a first line segment that is perpendicular to the determined epiphyseal point and located on the femur and the midpoint of a second line segment connecting the two reference points.

[0033] The processor can drive the program to divide the multiple joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia based on the diagnostic target image, and determine the osteophyte information corresponding to the osteophyte according to the multiple joint areas, and output the judgment information of the various benchmarks including the determined osteophyte information.

[0034] The processor can drive the program to determine at least one protrusion point corresponding to the maximum protrusion point of the lower end of the distal femur based on the diagnostic target image, judge the distance from the protrusion point to the predetermined point of the tibia, and output the judgment information of each benchmark including the information of the joint gap based on the judged distance.

[0035] The processor may drive the program to determine, based on the diagnosis target image, hardening information corresponding to a hardening region at a distal end of a femur or a tibia, and output determination information of each reference including the hardening information.

[0036] According to another aspect of the present invention, an artificial intelligence-based osteoarthritis diagnostic system includes: a processor; a storage device storing a program; wherein the processor drives the program to receive a diagnostic target image of a bone joint, use a deep learning model to determine the femoral axis of the diagnostic target image, determine the tibial axis of the diagnostic target image, and determine bone deformation information based on the determined femoral axis and tibial axis.

[0037] According to another aspect of the present invention, an artificial intelligence-based osteoarthritis diagnostic system may include: a processor; a storage device storing a program; wherein the processor can drive the program to accept a diagnostic target image input of a bone joint, and use a deep learning model to divide the diagnostic target image into multiple joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia, and determine osteophyte information corresponding to the osteophytes according to the multiple joint areas.

[0038] Effects of the Invention

[0039] According to the technical concept of the present invention, artificial intelligence, namely a deep learning model, can be used to output independent judgment criteria for osteoarthritis (for example, osteophytes, joint spaces, bone deformation, degree of sclerosis, etc.), and these can be integrated to perform diagnosis. Therefore, compared with the method of simply outputting the diagnostic level of the diagnostic target image, it has the effect of achieving relatively more accurate judgment.

[0040] In addition, since judgment can be made for each of the independent judgment criteria for osteoarthritis, there is an effect that an explainable diagnosis result can be derived.

[0041] Furthermore, rather than simply obtaining a diagnostic grade, the degree of progression can be confirmed according to the judgment criteria, which has the effect of enabling more personalized treatment or prescription. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more fully understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.

[0043] Figure 1 A diagram depicting how osteoarthritis was previously diagnosed.

[0044] Figure 2 A diagram for describing the structure of an artificial intelligence-based osteoarthritis diagnosis system according to the technical concept of the present invention.

[0045] Figure 3 1 is a diagram for describing a brief logical structure of an artificial intelligence-based osteoarthritis diagnosis system according to an embodiment of the present invention.

[0046] Figure 4 1 is a diagram for describing a brief physical structure of an artificial intelligence-based osteoarthritis diagnosis system according to an embodiment of the present invention.

[0047] Figure 5a , Figure 5b and Figure 6 A diagram for describing training data for bone deformation judgment information according to an embodiment of the present invention.

[0048] Figure 7 and Figure 8 A diagram for describing training data for osteophyte determination information according to an embodiment of the present invention.

[0049] Fig. 9 A diagram for describing training data for joint gap determination information according to an embodiment of the present invention.

[0050] Fig.10 FIG. 1 is a diagram for describing training data for sclerosis determination information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention may be subjected to various transformations and may have various embodiments, and specific embodiments will be exemplarily shown in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, and it should be understood that it includes all transformations, equivalents and substitutes included in the concept and technical scope of the present invention. In describing the present invention, when it is judged that the specific description of the relevant known technology may confuse the gist of the present invention, the detailed description will be omitted.

[0052] The terms "first", "second", etc. can be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from other components.

[0053] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present invention. As long as the context does not clearly indicate otherwise, a singular expression includes a plural expression.

[0054] In this specification, terms such as "including" or "having" are used to specify the existence of features, numbers, steps, actions, components, parts or their combinations recorded in the specification, and shall not be understood as excluding the existence or possibility of adding one or more other features, numbers, steps, actions, parts, parts or their combinations in advance.

[0055] In addition, in this specification, when it is mentioned that a certain component "transmits" data to another component, it means that the component may transmit the data directly to the other component, or may transmit the data to the other component through at least one other component. On the contrary, when it is mentioned that a certain component "directly transmits" data to another component, it means that the data is directly transmitted from the component to the other component without passing through other components.

[0056] The present invention will be described in detail below with reference to the accompanying drawings, focusing on the embodiments of the present invention. The same reference numerals in the drawings represent the same components.

[0057] Figure 2 A diagram for describing the structure of an artificial intelligence-based osteoarthritis diagnosis system according to the technical concept of the present invention.

[0058] Reference Figure 2 In order to realize the artificial intelligence-based hallux valgus diagnosis method according to the technical concept of the present invention, an artificial intelligence-based osteoarthritis diagnosis system (hereinafter referred to as the osteoarthritis diagnosis system) 100 can be realized.

[0059] The osteoarthritis diagnosis system 100 may receive an input of a diagnosis target image (eg, an X-ray image) corresponding to a bone joint that is a diagnosis target, and perform osteoarthritis diagnosis on the input diagnosis target image.

[0060] The so-called performing osteoarthritis diagnosis may mean determining the diagnostic level of osteoarthritis. In addition, according to the technical features of the present invention, it may mean outputting independent judgment criteria that become the judgment criteria for the diagnostic level of osteoarthritis, such as judgment information for each of bone deformation, osteophyte, joint space, and sclerosis.

[0061] Such determination information based on independent determination criteria also means meaningful information indicating whether osteoarthritis has progressed and / or the degree of progress for each determination criterion.

[0062] For example, the judgment information of each item in the independent judgment criterion may refer to a grade graded in a predetermined manner, or may refer to basic information used to judge such a grade, such as distance (for example, the distance between joint spaces), area (for example, the area of ​​osteophytes or the area of ​​areas with signs of sclerosis), angle (for example, the angle formed by the femoral axis and the tibial axis), etc.

[0063] Regardless of the situation, according to the technical concept of the present invention, the deep learning model according to the embodiment of the present invention can be trained to output the judgment information of the independent judgment benchmark. If the diagnostic target image is input into the osteoarthritis diagnosis system 100 including the deep learning model, the osteoarthritis diagnosis system 100 can output the judgment information of the independent judgment benchmark. Outputting the judgment information of the independent judgment benchmark itself can also be used to diagnose osteoarthritis. However, according to one embodiment, the osteoarthritis diagnosis system 100 can finally determine the diagnosis level of osteoarthritis based on the judgment information of the independent judgment benchmark and output it, so the process including the determination of the diagnosis level can be defined as the diagnosis of osteoarthritis.

[0064] The osteoarthritis diagnosis system 100 can be implemented by a data processing device for implementing the functions defined in this specification, or by a server connected to a network or by an independent diagnostic device. However, those skilled in the art of the present invention can easily deduce that it can also be implemented by any form of device having a data processing capability for executing the functions defined in this specification as needed, such as a computer, a laptop, a set-top box, a tablet computer, etc. used by a user.

[0065] In addition, Figure 2 The osteoarthritis diagnosis system 100 is shown to be implemented by one physical device. However, as required, the osteoarthritis diagnosis system 100 may also be implemented by organically combining multiple devices.

[0066] When receiving a diagnosis target image input, the osteoarthritis diagnosis system 100 may determine the judgment information of each item in the independent judgment criteria for diagnosing osteoarthritis through the deep learning model included in the osteoarthritis diagnosis system 100 .

[0067] In order to determine the judgment information of each item in the independent judgment criterion, the corresponding deep learning model can be trained and prepared according to each of the independent judgment criteria, but it is not necessarily limited to this.

[0068] In any case, the osteoarthritis diagnosis system 100 can determine the judgment information of each item in the independent judgment criterion by means of the deep learning model. Therefore, the osteoarthritis diagnosis system 100 can determine the osteoarthritis diagnosis level of the diagnosis target image based on the determined judgment information.

[0069] The process of determining the osteoarthritis diagnosis level of the diagnostic target image based on the determined judgment information can be performed by a predetermined established algorithm. For example, the deep learning model can be trained so that the numerical values ​​graded according to the independent judgment criteria are output as judgment information. At this time, the osteoarthritis diagnosis system 100 can use the established formula to calculate the judgment information of each independent judgment criterion, that is, the graded numerical value, and finally determine the diagnosis level.

[0070] At this time, the algorithm may be determined in the following manner: the judgment information of each independent judgment criterion is regarded as a parameter, a predetermined weight is assigned to each parameter, and the weight is optimized.

[0071] In order to achieve such optimization, a plurality of X-ray images for which diagnostic grade judgments have been made in the past may be prepared, and the following mathematical formula may be determined: each X-ray image is input into the osteoarthritis diagnosis system 100, and judgment information of an independent judgment criterion of each X-ray image is obtained, and then the judgment information of the independent judgment criterion of each X-ray image obtained is input, that is, the diagnostic grade that has been judged may be output. This method is widely used in conventional machine learning methods (e.g., regression analysis, etc.), and thus will not be described in detail.

[0072] As a result, the technical concept of the present invention is to train the deep learning model included in the osteoarthritis diagnosis system 100 to not simply output the osteoarthritis diagnosis level after accepting the diagnostic target image input, but to output the judgment information of each independent judgment criterion that becomes the osteoarthritis diagnosis criterion, and ultimately achieve the diagnosis of osteoarthritis based on the judgment information of such independent judgment criterion, thereby producing an effect of achieving a relatively accurate and explainable diagnosis, and because the degree of symptom progression is judged according to each independent judgment criterion, it has the effect of achieving more thorough treatment or prescription.

[0073] In addition, a technical concept for more accurately measuring judgment information of an independent judgment criterion is also disclosed.

[0074] The structure of the osteoarthritis diagnosis system 100 used herein will be referred to Figure 3 and Figure 4 Give a description.

[0075] Figure 3 1 is a diagram for describing a simplified logical structure of a system according to an embodiment of the present invention. Figure 4 FIG. 1 is a diagram for describing a simplified physical structure of a system according to an embodiment of the present invention.

[0076] Reference Figure 3 and Figure 4 The osteoarthritis diagnosis system 100 according to the technical concept of the present invention may include a control module 110, a deep learning model 120, and a diagnosis module 130. According to an embodiment, the deep learning model 120 may also include a first deep learning model 121, a second deep learning model 122, a third deep learning model 123, and a fourth deep learning model 124.

[0077] The osteoarthritis diagnosis system 100 may refer to a logical structure with hardware resources and / or software required to implement the technical concept of the present invention, and does not necessarily refer to a physical component or a device. That is, the osteoarthritis diagnosis system 100 may refer to a logical combination of hardware and / or software equipped to implement the technical concept of the present invention. If necessary, it may also be implemented by a set of logical structures that are arranged in separate devices and implement the technical concept of the present invention by performing their respective functions. In addition, the osteoarthritis diagnosis system 100 may also refer to a set of structures that are independently implemented according to each function or role for implementing the technical concept of the present invention. For example, each of the control module 110, the deep learning model 120 and / or the diagnosis module 130 may be located in different physical devices or in the same physical device. In addition, according to the embodiment, the combination of software and / or hardware constituting each of the control module 110, the deep learning model 120 and / or the diagnosis module 130 may also be located in different physical devices, and the structures located in different physical devices may also be organically combined with each other to implement each of the modules.

[0078] In addition, in this specification, the so-called module may refer to the functional or structural combination of hardware for executing the technical concept of the present invention and software for driving the hardware. For example, a person skilled in the art of the present invention can easily deduce that the module may refer to a logical unit of a given code and a hardware resource for executing the given code, and does not necessarily mean a physically connected code or a type of hardware.

[0079] On the other hand, the osteoarthritis diagnosis system 100 may have a physical structure as follows: Figure 4The osteoarthritis diagnosis system 100 may include: a memory (storage device) 120 - 1 storing a program for realizing the technical concept of the present invention; and a processor 110 - 1 for running the program stored in the memory 120 .

[0080] Those skilled in the art can easily deduce that the processor 110-1 can be named as a CPU, a mobile processor, etc. according to the implementation form of the osteoarthritis diagnosis system 100. Figure 2 As described in , the osteoarthritis diagnostic system 100 can also be implemented by organically combining multiple physical devices. In this case, technicians in the technical field of the present invention can easily deduce that the processor 110-1 can be equipped with at least one physical device to implement the system 100 of the present invention.

[0081] The memory 120-1 may be implemented by any form of storage device that stores the program and is accessible to the processor to drive the program. In addition, according to the hardware implementation form, the memory 120-1 may also be implemented by multiple storage devices rather than a single storage device. In addition, the memory 120-1 includes not only a main memory device but also a temporary memory device. In addition, it may also be implemented by a volatile memory or a non-volatile memory, and may be defined as including the meaning of all forms of information storage devices that are implemented in a manner that stores the program and can be driven by the processor.

[0082] According to the embodiment, the osteoarthritis diagnosis system 100 may refer to a system implemented in a manner that can diagnose osteoarthritis according to the technical concept of the present invention. For example, it may be implemented in various ways such as a web server, a computer, etc., and may be defined as including any form of data processing device that can execute the functions defined in this specification.

[0083] In addition, according to the embodiment of the osteoarthritis diagnosis system 100, various peripheral devices (peripheral device 1 to peripheral device N) 130-1, 131-1 may also be provided. For example, a person skilled in the art of the present invention may easily deduce that the osteoarthritis diagnosis system 100 may also include a keyboard, a display, a graphics card, a communication device, etc. as peripheral devices.

[0084] Those skilled in the art can easily deduce that, in the following description, a predetermined module performing a certain function means that the processor 110 - 1 drives the program configured in the memory 120 - 1 to perform the function.

[0085] The control module 110 can control functions and / or resources of other structures included in the osteoarthritis diagnosis system 100 (for example, the deep learning model 120 , the diagnosis module 130 , etc.).

[0086] The control module 110 can both perform training of the deep learning model 120 and use the output results of the deep learning model 120 to perform a series of data processing required for diagnosis.

[0087] According to an embodiment, the control module 110 may also train the deep learning model 120. To this end, the control module 110 may accept training data input for training the deep learning model 120 and execute a series of processes for training the deep learning model 120.

[0088] Of course, the control module 110 may also perform diagnosis simply through the trained deep learning model 120 .

[0089] The control module 110 can accept the input of the diagnostic target image of the bone joint. The input of the diagnostic target image can be performed through the terminal used by the user. The control module 110 can be set in the terminal, or it can be implemented in the form of a web server that can communicate with the terminal. As mentioned above, various embodiments can be implemented.

[0090] The control module 110 can control the deep learning model 120 so that the deep learning model 120 outputs judgment information of at least one of osteophytes, joint space, sclerosis, and bone deformation as the diagnosis basis of osteoarthritis for the diagnostic target image.

[0091] The deep learning model 120 may also be a model integrated in a manner of outputting the judgment information of each criterion in the diagnostic criterion as described above. Figure 3 As shown, the deep learning model corresponding to each diagnostic benchmark can also be configured independently.

[0092] To this end, the deep learning model 120 may include a first deep learning model 121, a second deep learning model 122, a third deep learning model 123 and / or a fourth deep learning model 124. Of course, according to an embodiment, multiple of the diagnostic criteria (e.g., osteophytes, sclerosis) may be determined by any deep learning model, and the remaining diagnostic criteria may be determined by any deep learning model respectively.

[0093] Those skilled in the art of the present invention can easily deduce that various embodiments can be implemented.

[0094] The diagnosis module 130 may generate an osteoarthritis diagnosis result of the diagnosis target image based on the judgment information of each benchmark output by the deep learning model 120 .

[0095] The diagnosis result may be, for example, any one of levels 0 to 4 disclosed in the KL diagnosis scale.

[0096] As described above, the diagnosis module 130 can be implemented in the following manner: if the judgment information input of each benchmark output by the deep learning model 120 is accepted, the diagnosis result is output. To this end, as described above, the diagnosis module 130 can be trained by established machine learning and regression analysis. The algorithm or mathematical formula that can accept the judgment information input of each benchmark and output the diagnosis result in response thereto can be determined in various ways.

[0097] Below, in this specification, the following scenario is described by way of example: the first deep learning model 121 performs judgment on the diagnostic benchmark of bone deformation, i.e., derives, the second deep learning model 122 performs judgment on the diagnostic benchmark of osteophytes, the third deep learning model 123 performs judgment on the diagnostic benchmark of joint space, and the fourth deep learning model 124 performs judgment on the diagnostic benchmark of sclerosis.

[0098] pass Figure 5a , Figure 5b and Figure 6 , describes a specific example of training data prepared for the first deep learning model 121 to judge bone deformation and information on the trained first deep learning model 121 performing judgments on various benchmarks of bone deformation.

[0099] Figure 5a , Figure 5b and Figure 6 A diagram for describing training data for bone deformation judgment information according to an embodiment of the present invention.

[0100] like Figure 5a , Figure 5b and Figure 6 As shown, in order to train the first deep learning model 121, training data with regions corresponding to each of the femur and tibia labeled on bone joint X-ray images can be prepared.

[0101] like Figure 5a As shown, the marking may not include the entire femur, but may include the distal end of the femur (a certain area starting from the lowest end of the thigh), or may only mark a certain length of the upper femur area.

[0102] In addition, the same is true for the tibia. Instead of including the entire tibia, the distal end (a certain area starting from the uppermost end of the tibia) may be included, or only a certain length of the lower tibia area may be marked.

[0103] like Figure 5a As shown, the first deep learning model 121 trained by such training data can determine the femur region and the tibia region on the X-ray image. Such annotation can be achieved by marking the femur region and the tibia region with an annotation tool, and the data type can be a polygon.

[0104] In addition, the first deep learning model 121 can be trained based on the input diagnostic target image so as to respectively determine the femoral axis and the tibial axis. That is, the first deep learning model 121 can be trained to determine the femoral axis and the tibial axis in the diagnostic target image.

[0105] The femoral axis and the tibial axis may respectively refer to anatomical axes, and the degree of bone deformation may be determined based on the angle formed by the femoral axis and the tibial axis.

[0106] In order to determine the femoral axis and the tibial axis respectively, the first deep learning model 121 may require additional training data.

[0107] Such training data may include Figure 5a The marking tool for performing marking determines and marks the data of the epiphysis point 10 at a predetermined position of the cortical bone of the femoral epiphysis in the X-ray image. The epiphysis may refer to the area where the lines at both ends of the femur are parallel or almost parallel, and the marking tool may mark any point in the cortical bone of the epiphysis as the epiphysis point 10.

[0108] In addition, for the tibia, the marking tool can determine and mark the epiphysis point 11 at a predetermined position of the cortical bone of the tibial epiphysis in the X-ray image.

[0109] Moreover, if Figure 5b As shown, the marking tool can determine and mark two reference points 20, 21 corresponding to the outermost and innermost positions of the distal femur respectively. Of course, the same is true for the tibia, and two reference points 22, 23 corresponding to the outermost and innermost positions of the distal tibia can be determined and marked respectively.

[0110] After training the training data with the epiphyseal points 10, 11 and reference points 20, 21, 22, 23 marked on each of the femurs and tibias, the first deep learning model 121 can determine and output the locations of the epiphyseal points and reference points if a diagnostic target image is input.

[0111] After the diagnostic target image is inputted into the first deep learning model 121, if the epiphyseal point and the reference point are determined, the femoral axis and the tibial axis can be determined.

[0112] According to an embodiment of the present invention, Figure 6 As shown, the femoral axis can be determined based on the first midpoint 30 of the first line segment 42 which is perpendicular to the determined epiphyseal point 10 and located on the femur and the second midpoint 31 of the second line segment 40 connecting the two reference points 20, 21. That is, the line 50 connecting the first midpoint 30 and the second midpoint 31 can be the femoral axis.

[0113] In addition, the same is true for the tibial axis, e.g. Figure 6As shown, it can be determined based on the first midpoint 33 of the first line segment 43 which is perpendicular to the determined epiphyseal point 11 and located on the tibia and the second midpoint 32 of the second line segment 41 connecting the two reference points 22, 23. That is, the line 51 connecting the first midpoint 33 and the second midpoint 32 can be the tibial axis.

[0114] Therefore, the bone deformation determination information may include information on the angle formed by the determined femoral axis and tibial axis. The bone deformation determination information may be the angle itself, or when the angle is classified into predetermined levels according to the degree of the angle, it may be a numerical value indicating the level.

[0115] Moreover, the femoral axis and the tibial axis are information that can be automatically determined after determining the epiphyseal points 10, 11 and the reference points 20, 21, 22, 23 as mentioned above. Therefore, the first deep learning model 121 can be trained to output the epiphyseal points 10, 11 and the reference points 20, 21, 22, 23 after inputting the diagnostic target image, and the bone deformation judgment information can also be ultimately calculated by the control module 110.

[0116] In addition, reference points 20, 21, 22, 23 of each of the femur and tibia may also be used to determine the joint space as described later.

[0117] As a result, according to the technical concept of the present invention, the osteoarthritis diagnosis system 100 can automatically output judgment information that can judge the degree of bone deformation, thereby not only simply judging the diagnosis level of osteoarthritis, but also independently judging the degree of bone deformation.

[0118] On the other hand, refer to Figure 7 and Figure 8 , an example describing the judgment information of each benchmark of osteophyte.

[0119] Figure 7 and Figure 8 A diagram for describing training data for osteophyte determination information according to an embodiment of the present invention.

[0120] First refer to Figure 7 According to the technical concept of the present invention, the second deep learning model 122 can obtain osteophyte judgment information by converting the area corresponding to the bone joint, such as Figure 7 The division shown is into multiple joint areas corresponding to the lateral side of the femur (FEM Lat, 60, 65), the medial side of the femur (FEM Med, 61, 64), the lateral side of the tibia (TIB Lat, 62, 67), or the medial side of the tibia (TIB Med, 63, 66).

[0121] For this purpose, the marking tool can mark the areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, and the medial side of the tibia in the X-ray image as training data. This marking can be performed in the following manner, that is, using the established marking tool, using the calibration frame to draw on the X-ray image.

[0122] Moreover, if Figure 8 As shown, the marking tool can mark the osteophyte information indicating where the osteophyte is located according to multiple joint regions. For example, on one side of the knee (e.g., Figure 8 On the left side), use a marking tool to mark the osteophyte 60-1 area included in the lateral femoral area 60, use a marking tool to mark the osteophyte 61-1 area included in the medial femoral area 61, use a marking tool to mark the osteophyte 62-1 area included in the lateral tibial area 62, and use a marking tool to mark the osteophyte 63-1 area included in the medial tibial area 63.

[0123] Alternatively, the other knee (e.g. Figure 8 On the right side of the tibia, use the marking tool to mark the osteophyte 65-1 area included in the lateral femoral area 65, use the marking tool to mark the osteophyte 64-1 area included in the medial femoral area 64, use the marking tool to mark the osteophyte 67-1 area included in the lateral tibial area 67, and use the marking tool to mark the osteophyte 66-1 area included in the medial tibial area 66.

[0124] The second deep learning model 122 that has learned the training data labeled in this manner can determine the osteophyte information of the area corresponding to the osteophyte for each joint area if a diagnostic target image is input.

[0125] According to one embodiment, the training data used by the second deep learning model 122 is not only information for simply determining where osteophytes are located by joint region, but may also further include established osteophyte diagnosis information based on the determined osteophytes. The osteophyte diagnosis information may also be independently labeled by joint region.

[0126] That is, the training data may be labeled with the areas corresponding to the osteophytes in the joint areas as described above and the osteophyte diagnosis information indicating the severity of the final osteophyte (e.g., 0, 1, 2, 3). Thus, the second deep learning model 122 may be trained to output not only the location of the osteophytes by joint area, but also the osteophyte diagnosis information.

[0127] According to an embodiment, the osteophyte diagnosis information may also be determined by the control module 110 based on the osteophyte information. That is, after determining the osteophyte information according to the joint area, that is, the area where the osteophyte occurs, the control module 110 may output the osteophyte diagnosis information by measuring the size of the osteophyte area.

[0128] In the present invention, exemplary osteophyte diagnostic information, for example, when it is 0, it may mean that no osteophyte is found; when it is 1, it may mean that there is a degree of possibility of osteophyte occurrence; when it is 2, it may mean the degree to which osteophyte has actually occurred; when it is 3, it means that osteophyte has occurred and the size is above a given level.

[0129] As a result, according to the technical concept of the present invention, the osteoarthritis diagnosis system 100 does not simply determine the diagnosis level of osteoarthritis, but has the effect of outputting the degree of determination of osteophytes in more detail.

[0130] Furthermore, when the bone joint area is divided into a plurality of joint areas and the diagnostic information of osteophytes is determined according to each area, it has the effect of being able to more accurately determine the progression of osteophytes according to specific locations.

[0131] On the other hand, referring to Fig. 9 , describing the joint space judgment information judged by the third deep learning model 123 included in the osteoarthritis diagnosis system 100.

[0132] Fig. 9 A diagram for describing training data for joint gap judgment information according to an embodiment of the present invention.

[0133] Reference Fig. 9 , used to train the third deep learning model 123 to determine the training data of the joint clearance determination information, such as Fig. 9 As shown, at least one protrusion point 70, 71 corresponding to the maximum protrusion point of the lower end of the distal femur can be determined and marked by a marking tool.

[0134] Therefore, the third deep learning model 123 may be trained to determine salient points from a diagnosis target image if a diagnosis target image is input.

[0135] Thus, the distance from the protruding points 70, 71 to the predetermined points 72, 73 of the tibia can be determined. The predetermined points 72, 73 can be the points corresponding to the tibia that are first encountered when drawing a line from the protruding points 70, 71 to the vertical lower part (or in a direction parallel to the femoral axis).

[0136] After determining the salient points 70 and 71 output by the third deep learning model 123 , as described above, the control module 110 may determine the predetermined points 72 and 73 and calculate the distances from the salient points 70 and 71 to the predetermined points 72 and 73 .

[0137] Therefore, each of the determined distances may also be determination information of the joint gap, and the joint gap may be classified into predetermined levels based on the distances. In this case, the numerical value indicating the level may also be determination information of the joint gap.

[0138] In addition, according to the embodiment, not only the joint gap but also the area of ​​the joint space may be used as the judgment information of the joint gap. The joint space may refer to the information of the area between the femur and the tibia, which can be used in Figure 5b The information of the joint space is determined by using the reference points 20, 21, 22, 23 described in the joint space determination. That is, after the reference points 20, 21, 22, 23 are determined, the area with the reference points 20, 21, 22, 23 as vertices and surrounded by the lower edge of the femur and the upper edge of the tibia can be determined as the joint space, so the information of the joint space can be additionally included in the joint space determination information.

[0139] After the first deep learning model 121 determines the reference points 20 , 21 , 22 , and 23 , the control module 110 can of course determine the information of the joint space as described above.

[0140] On the other hand, refer to Fig.10 , the judgment information of each sclerosis criterion is described.

[0141] Fig.10 FIG. 1 is a diagram for describing training data for sclerosis determination information according to an embodiment of the present invention.

[0142] Reference Fig.10 There will be white areas on the X-ray images at the lower end of the femur and the upper end of the tibia, which can be confirmed as sclerosis.

[0143] Therefore, if Fig.10 As shown, the labeling tool can label the sclerosis occurrence area on the X-ray image. After the labeled training data is learned by the fourth deep learning model 124, if the diagnostic target image is input, the fourth deep learning model 124 can determine and output the sclerosis occurrence area 80, that is, the sclerosis information.

[0144] Therefore, the sclerosis occurrence area 80 itself may be the sclerosis information, and numerical information classified into predetermined levels based on the area of ​​the area may also be the sclerosis information.

[0145] When such sclerosis information is classified into predetermined levels, the marking tool can also mark the area and level of sclerosis occurrence.

[0146] Alternatively, after the fourth deep learning model 124 determines the sclerosis occurrence region 80 , the control module 110 may also calculate the area of ​​the sclerosis occurrence region 80 to automatically classify and determine the sclerosis information.

[0147] As described above, after the control module 110 determines the judgment information of each benchmark itself or the information used to determine the judgment information of each benchmark through the deep learning model 120, it can finally determine the judgment information of each benchmark.

[0148] Then, the control module 110 may control the diagnosis module 130 to finally determine the osteoarthritis diagnosis level.

[0149] In addition, as described above, the osteoarthritis diagnosis system 100 may be a system that does not output the osteoarthritis diagnosis level but only determines at least one of the judgment information of each benchmark. In this case, the specific degree of progression of osteoarthritis can also be diagnosed, which will be a meaningful diagnosis.

[0150] The artificial intelligence-based osteoarthritis diagnosis method according to an embodiment of the present invention can be implemented in a computer-readable recording medium as a computer-readable code. Computer-readable recording media include all kinds of recording devices for data storage that can be read by means of a computer system. Examples of computer-readable recording media include ROM (read-only memory), RAM (random access memory), CD-ROM (read-only optical disk drive), magnetic tape, hard disk, floppy disk, optical data storage device, etc. In addition, the computer-readable recording medium is distributed in a computer system connected to a network, and the computer-readable code is stored and run in a distributed manner. Moreover, the functional program, code, and code segment used to implement the present invention can be easily derived by a programmer in the technical field to which the present invention belongs.

[0151] The present invention is described with reference to the embodiments illustrated in the accompanying drawings, but these are merely examples, and those skilled in the art will understand that other various variant embodiments and equivalent embodiments can be implemented accordingly. Therefore, the true technical protection scope of the present invention should be determined according to the technical ideas of the attached claims.

Claims

1. An artificial intelligence-based osteoarthritis diagnosis method, comprising: The osteoarthritis diagnosis system receives a diagnostic target image input of a bone joint; The osteoarthritis diagnosis system uses a deep learning model to output, for an input diagnostic target image, at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis; and The osteoarthritis diagnosis system generates an osteoarthritis diagnosis result of the diagnosis target image based on the output judgment information of each reference.

2. The artificial intelligence-based osteoarthritis diagnosis method according to claim 1, wherein: The osteoarthritis diagnosis system uses a deep learning model to output, for an input diagnostic target image, at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis, including the following steps: The osteoarthritis diagnosis system determines the femoral axis based on the diagnosis target image; The osteoarthritis diagnosis system determines the tibial axis based on the diagnosis target image; and The osteoarthritis diagnosis system outputs the determination information of the respective references including bone deformation information based on the determined femoral axis and tibial axis.

3. The artificial intelligence-based osteoarthritis diagnosis method according to claim 2, wherein: The step of determining the femoral axis based on the diagnostic target image by the osteoarthritis diagnostic system includes: The osteoarthritis diagnosis system determines the epiphysis point at a predetermined position of the bone cortex of the femoral epiphysis in the diagnosis target image; The osteoarthritis diagnosis system determines two reference points corresponding to the outermost and innermost positions of the distal femur; and The osteoarthritis diagnosis system includes a step of determining information of the femoral axis based on a midpoint of a first line segment that is perpendicular to the determined epiphyseal point and located on the femur and a midpoint of a second line segment that connects the two reference points.

4. The artificial intelligence-based osteoarthritis diagnosis method according to claim 1, wherein: The osteoarthritis diagnosis system uses a deep learning model to output, for an input diagnostic target image, at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis, including the following steps: The osteoarthritis diagnosis system divides a plurality of joint regions corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia based on the diagnosis target image; The osteoarthritis diagnosis system determines osteophyte information corresponding to the osteophytes according to the plurality of joint regions; and The osteoarthritis diagnosis system outputs the determination information of the respective references including the determined osteophyte information.

5. The artificial intelligence-based osteoarthritis diagnosis method according to claim 1, wherein: The osteoarthritis diagnosis system uses a deep learning model to output, for an input diagnostic target image, at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis, including the following steps: The osteoarthritis diagnosis system determines at least one protrusion point corresponding to the maximum protrusion point of the lower end of the distal femur based on the diagnosis target image; The step of determining the distance from the protruding point to a predetermined point of the tibia; and A step of outputting the determination information of the respective references including the information of the joint space based on the determined distance.

6. The artificial intelligence-based osteoarthritis diagnosis method according to claim 1, wherein: The osteoarthritis diagnosis system uses a deep learning model to output, for an input diagnostic target image, at least one of osteophytes, joint space, sclerosis, and bone deformation as a diagnostic benchmark for osteoarthritis, including the following steps: The osteoarthritis diagnosis system determines hardening information corresponding to the hardened area at the distal end of the femur or tibia based on the diagnosis target image; The osteoarthritis diagnosis system outputs the determination information of the respective criteria including the sclerosis information.

7. An artificial intelligence-based osteoarthritis diagnosis method, comprising: The osteoarthritis diagnosis system receives a diagnostic target image input of a bone joint; The osteoarthritis diagnosis system uses a deep learning model to determine the femoral axis of the diagnosis target image; The osteoarthritis diagnosis system determines the tibial axis of the diagnosis target image; and The osteoarthritis diagnosis system comprises a step of determining bone deformation information based on the determined femoral axis and tibial axis.

8. An artificial intelligence-based osteoarthritis diagnosis method, comprising: The osteoarthritis diagnosis system receives a diagnostic target image input of a bone joint; The osteoarthritis diagnosis system uses a deep learning model to divide the diagnosis target image into a plurality of joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia; and The osteoarthritis diagnosis system includes a step of determining osteophyte information corresponding to the osteophytes according to the plurality of joint regions.

9. A computer program recorded on a computer-readable recording medium, installed in a data processing device, for executing the method according to any one of claims 1 to 8.

10. An osteoarthritis diagnosis system based on artificial intelligence, comprising: processor; A storage device storing a program; In which, the processor drives the program to accept the input of the diagnostic target image of the bone joint, and uses the deep learning model to output the judgment information of at least one of the osteophytes, joint spaces, sclerosis, and bone deformation that serve as the diagnostic benchmark for osteoarthritis for the input diagnostic target image, and generates the osteoarthritis diagnosis result of the diagnostic target image based on the output judgment information of each benchmark.

11. The artificial intelligence-based osteoarthritis diagnosis system according to claim 10, wherein: The processor drives the program to determine the femoral axis based on the diagnostic target image, and the osteoarthritis diagnostic system determines the tibial axis based on the diagnostic target image, and outputs determination information of the respective benchmarks including bone deformation information based on the determined femoral axis and tibial axis.

12. The artificial intelligence-based osteoarthritis diagnosis system according to claim 11, wherein: The processor drives the program to determine the epiphyseal point at a predetermined position of the cortical bone of the femoral epiphysis in the diagnostic target image, determine two reference points corresponding to the lateralmost and medialmost positions of the distal end of the femur, and judge the information of the femoral axis based on the midpoint of a first line segment that is perpendicular to the determined epiphyseal point and located on the femur and the midpoint of a second line segment connecting the two reference points.

13. The artificial intelligence-based osteoarthritis diagnosis system according to claim 10, wherein: The processor drives the program to divide the multiple joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia based on the diagnostic target image, determine the osteophyte information corresponding to the osteophyte according to the multiple joint areas, and output the judgment information of the various benchmarks including the determined osteophyte information.

14. The artificial intelligence-based osteoarthritis diagnosis system according to claim 10, wherein: The processor drives the program to determine, based on the diagnostic target image, at least one protrusion point corresponding to the maximum protrusion point of the lower end of the distal femur, determine the distance from the protrusion point to a predetermined point of the tibia, and output the judgment information of each benchmark including the information of the joint gap based on the determined distance.

15. The artificial intelligence-based osteoarthritis diagnosis system according to claim 10, wherein: The processor drives the program to determine hardening information corresponding to a hardening area at a distal end of a femur or a tibia based on the diagnosis target image, and output determination information of each reference including the hardening information.

16. An osteoarthritis diagnosis system based on artificial intelligence, comprising: processor; A storage device storing a program; Among them, the processor drives the program to accept the input of the diagnostic target image of the bone joint, use the deep learning model to determine the femoral axis of the diagnostic target image, determine the tibial axis of the diagnostic target image, and determine the bone deformation information based on the determined femoral axis and tibial axis.

17. An osteoarthritis diagnosis system based on artificial intelligence, comprising: processor; A storage device storing a program; Among them, the processor drives the program to accept the input of the diagnostic target image of the bone joint, and uses the deep learning model to divide the diagnostic target image into multiple joint areas corresponding to the lateral side of the femur, the medial side of the femur, the lateral side of the tibia, or the medial side of the tibia, and determine the osteophyte information corresponding to the osteophyte according to the multiple joint areas.

Images