Service provision system and method for predicting and diagnosing joint conditions

Abstract

A system and method for providing a service capable of predicting and diagnosing a joint condition are provided. The method for providing a service capable of predicting and diagnosing a joint condition, which is performed by a server, includes the steps of: extracting image information for each frame from photographed video data of a target object; inputting the image information for each frame into a pose estimation model among a plurality of convolution layers of a convolutional neural network (CNN) to extract predicted joint data of the target object; and generating result data for determining whether or not there is an abnormality in the joint of the target object by comparing and analyzing the predicted joint data extracted through the pose estimation model based on standard data. The step of extracting the predicted joint data includes the steps of: recognizing the target object as an object by separating the target object from a background from the image information for each frame; detecting a boundary area centered on the object and drawing a bounding box for each frame. extracting predicted coordinate information for each frame for a body part of the object located within the bounding box; extracting an object movement direction of the object located within the bounding box; and labeling the predicted coordinate information based on the object movement direction to extract the predicted joint data.

Description

[Technical field]

[0001] The present invention relates to a service providing system and method capable of predicting and diagnosing the condition of a joint, and in particular to a service providing system and method capable of predicting dislocation of an animal's patella, hip joint, etc., and determining the presence or absence of abnormality without meeting the user, using video obtained from a user. [Background technology]

[0002] Recently, as the population structure ages and single-person households increase, humans are becoming increasingly self-centered and mentally desolate. As a result, the number of people who view companion animals as family members or partners is increasing, and the market for companion animals is also steadily growing.

[0003] When companion animals start to show abnormal symptoms that are not normally seen, in many cases people have had to take them to a veterinary hospital for treatment or solve the problem based on information they have obtained from people around them or via the internet or telephone.

[0004] For example, patellar luxation, which is common in aging dogs, is a disease in which the patella at the top of the knee joint becomes displaced. It can occur due to trauma or a congenital poor joint shape, or it can occur as the dog ages and abnormalities appear in a previously strong knee joint.

[0005] Patellar luxation is known to occur especially frequently in certain small dogs such as toy poodles, Pomeranians, Yorkshire terriers, and Maltese. In severe cases, the dislocated joint swells and ligaments rupture, causing severe pain, leading to companion dogs dragging their legs or walking with their legs raised. In cases where an individual is vulnerable to patellar luxation or has a high probability of developing the disease, intensive management in advance can delay the onset of the disease. In other words, patellar luxation can be prevented by training the individual not to do things that strain the joints, providing physical therapy, feeding food that is good for the joints, maintaining an appropriate weight, creating an appropriate living environment that avoids trauma, exercising, massaging, etc., and corrective surgery can also be performed in advance at an early stage. Therefore, in order to prevent patellar luxation and improve the quality of life of companion dogs, a means is needed to predict and diagnose patellar luxation early.

[0006] However, the information obtained from people around them or through the internet or telephone is often incorrect, making it difficult to receive treatment. Even if patients visit hospitals directly, they often have to wait long periods of time, and due to the hospital's heavy workload, customers are often not provided with proper service.

[0007] The matters described above as the background art are merely intended to enhance understanding of the background of the present invention, and should not be construed as an admission that they constitute prior art already known to those of ordinary skill in the art. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Republic of Korea Patent Publication No. 2003-0032088 (Published on April 26, 2003) Summary of the Invention [Problem to be solved by the invention]

[0009] The problem to be solved by the present invention is to provide a service provision system and method capable of predicting and diagnosing the condition of a joint.

[0010] The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems, a method for providing a service capable of predicting and diagnosing a joint condition according to an embodiment of the present invention is a method for providing a service capable of predicting and diagnosing a joint condition, which is performed by a server, and includes the steps of: extracting image information for each frame from photographed video data of an object; inputting the image information for each frame into a pose estimation model among a plurality of convolution layers of a convolutional neural network (CNN) to extract predicted joint data of the object; and generating result data for determining whether or not there is an abnormality in the joint of the object by comparing and analyzing the predicted joint data extracted through the pose estimation model based on standard data; and extracting the predicted joint data includes the steps of: recognizing the object as an object by separating the object from a background from the image information for each frame; detecting a boundary area centered on the object and drawing a bounding box for each frame. extracting predicted coordinate information for each frame for a body part of the object located within the bounding box; extracting an object movement direction of the object located within the bounding box; and labeling the predicted coordinate information based on the object movement direction to extract the predicted joint data.

[0012] In one embodiment of the present invention, the step of extracting the predicted joint data may include: generating a derived variable using the predicted coordinate information; and inputting the derived variable into the posture estimation model based on the standard data to extract the predicted joint data.

[0013] In one embodiment of the present invention, the step of generating the result data may include matching the predicted coordinate information and the standard coordinate information for each frame based on the standard data, and generating the result data indicating that there is an abnormality in the joint of the target object if the predicted coordinate information and the standard coordinate information are equal to or greater than a preset criterion.

[0014] In one embodiment of the present invention, the derived variables may include at least one of the height of the step between the right and left feet of the object, the number of frames corresponding to a change in the movement of the object, the angle of at least one of the object's feet, and the movement of the object's neck and tail.

[0015] In one embodiment of the present invention, the step of generating the result data may include analyzing a difference in gait height between the right foot and the left foot of the object, and generating the result data indicating that there is an abnormality in the joint of the object if the difference in height is greater than an average value set based on the standard data.

[0016] In one embodiment of the present invention, the step of generating result data may include analyzing a difference in the number of frames corresponding to a change in the movement of the object, and generating the result data indicating that an abnormality exists in the joint of the object if the number of frames exceeds an average value set based on the standard data.

[0017] In one embodiment of the present invention, the step of generating result data may include analyzing angles of at least one or more feet of the object, analyzing a difference in foot angles corresponding to a change in movement, and generating the result data indicating that an abnormality exists in a joint of the object if an angle difference occurs that is greater than an average value set based on the standard data.

[0018] In one embodiment of the present invention, the step of generating the result data may include analyzing movements of the neck and tail of the object, analyzing a difference in coordinate values ​​corresponding to a change in movement, and generating the result data indicating that the joints of the object are abnormal if a Mean Squared Error (MSE) greater than an average value set based on the standard data occurs.

[0019] In one embodiment of the present invention, the method includes the step of generating the standard data, and the step of generating the standard data may include the steps of: extracting an image for each frame from a video; extracting and labeling standard coordinate information for a body part of an object included in the extracted image; generating standard derived variables using the standard coordinate information; setting standard parameters based on the standard derived variables and inputting them into the posture estimation model; and verifying the standard data generated by iteratively learning standard joint data acquired through the posture estimation model and standard result data corresponding to the standard joint data.

[0020] In an embodiment of the present invention, the method may further include labeling the predicted joint data extracted through the pose estimation model for each object by frame.

[0021] In one embodiment of the present invention, the captured video data may be a video file captured in a consistent direction while the subject walks for 10 seconds or more.

[0022] In accordance with another embodiment of the present invention, there is provided a service providing system capable of predicting and diagnosing a condition of a joint, the system including: a user terminal that acquires photographed image data from a target object to be examined; and a management server that extracts frame-by-frame image information from the photographed image data of the target object, inputs the frame-by-frame image information into a pose estimation model among a plurality of convolution layers of a convolutional neural network (CNN) to extract predicted joint data of the target object, and compares and analyzes the predicted joint data extracted through the pose estimation model based on standard data to determine whether or not there is an abnormality in the joint of the target object, thereby generating result data for diagnosis; the management server separates the target object from a background in the frame-by-frame image information, recognizes the target object as an object, detects a boundary area centered on the object, and generates a bounding box for each frame. a bounding box (hereinafter referred to as a bounding box) for extracting predicted coordinate information for each frame of a body part of the object located within the bounding box; extracting an object movement direction of the object located within the bounding box; and labeling the predicted coordinate information based on the object movement direction to extract the predicted joint data.

[0023] In one embodiment of the present invention, a service linking terminal for sharing medical management data generated in response to the result data may be included.

[0024] In one embodiment of the present invention, the system may further include an administrator terminal for learning the standard data and generating the result data corresponding to the captured image data.

[0025] A program according to an embodiment of the present invention is stored in a computer-readable recording medium that can be read by the computer so as to be combined with a computer, which is hardware, and to perform the service providing method for predicting and diagnosing the condition of the joint.

[0026] Further details of the invention are contained in the detailed description and the drawings. Effect of the Invention

[0027] According to the present invention, it is possible to predict patellar luxation in animals and determine the presence or absence of abnormalities without meeting the user in person by using video obtained from the user, thereby increasing convenience and reliability while respecting the diversity of users.

[0028] According to the present invention, when an abnormality occurs in the subject of examination, particularly in the joints of a companion animal, the current condition of the companion animal can be quickly and accurately determined using a portable terminal device, thereby providing a sense of confidence to the guardian of the companion animal.

[0029] According to the present invention, the current condition of a companion animal can be accurately diagnosed in real time using a portable terminal, thereby improving the convenience and reliability of the user.

[0030] According to the present invention, hospital information regarding the current condition of a companion animal is also provided, so that the companion animal's illness can be treated early at a hospital that can treat the illness, thereby protecting the health of the companion animal.

[0031] According to the present invention, notification information about a companion animal is continuously provided to a guardian, thereby encouraging the guardian to return to the veterinary clinic and preventing the guardian from abandoning the pet.

[0032] According to the present invention, by sharing the medical examination results of companion animals with the linked service, the condition of the companion animal can be more accurately understood and responded to, thereby giving a sense of confidence to the guardian of the companion animal.

[0033] The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief description of the drawings]

[0034] [Figure 1] 1 is a conceptual diagram for explaining a service providing system capable of predicting and diagnosing a joint condition according to one embodiment of the present invention. FIG. [Diagram 2] 2 is a diagram for explaining a detailed configuration of a service providing system capable of predicting and diagnosing the condition of a joint shown in FIG. 1. [Diagram 3] 1 is a diagram illustrating a method for providing a service capable of predicting and diagnosing a joint condition according to an embodiment of the present invention. [Figure 4] 4 is a diagram for explaining a step of generating standard data shown in FIG. 3; [Diagram 5] 5 is a diagram for explaining basic shooting information shown in FIG. 4; [Figure 6] 5 is a diagram for explaining a standard coordinate information extraction and labeling step shown in FIG. 4; [Figure 7] 4 is a diagram for explaining a step of generating result data shown in FIG. 3; [Figure 8] 8 is a diagram for explaining the difference in height of normal gait shown in FIG. 7. [Figure 9] 8 is a diagram for explaining the abnormal gait height difference shown in FIG. 7. [Figure 10] 8 is a diagram for explaining the number of normal frames shown in FIG. 7; [Figure 11] 8 is a diagram for explaining the number of abnormal frames shown in FIG. 7; [Figure 12] 8 is a diagram for explaining a normal foot angle shown in FIG. 7. [Figure 13] 8 is a diagram for explaining the abnormal foot angle shown in FIG. 7. [Figure 14] 8 is a diagram for explaining normal and abnormal states of neck and tail movement based on the coordinate information of the front legs shown in FIG. 7. [Figure 15] FIG. 8 is a diagram for explaining normal and abnormal states of neck and tail movement based on the coordinate information of the hind legs shown in FIG. 7. [Figure 16]8 is a diagram for explaining normal and abnormal states of neck and tail movement based on coordinate information of front and back legs shown in FIG. 7. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] The advantages and features of the present invention, as well as the methods for achieving them, will become clearer with reference to the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various different forms, and the embodiments are provided to complete the disclosure of the present invention and to fully inform those skilled in the art of the present invention of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0036] The terms used in the present specification are for the purpose of describing the embodiments and are not intended to limit the present invention. In the present specification, the singular includes the plural unless otherwise specified in the text. The terms "comprises" and / or "comprising" used in the specification do not exclude the presence or addition of one or more other elements in addition to the elements mentioned. The same reference numerals refer to the same elements throughout the specification, and "and / or" includes each and every combination of one or more of the elements mentioned. Although "first", "second", etc. may be used to describe various elements, it is understood that these elements are not limited by these terms. These terms are used merely to distinguish one element from another element. Therefore, it is understood that the first element mentioned below may be the second element within the technical concept of the present invention.

[0037] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the sense commonly understood by a person skilled in the art to which the present invention belongs. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless otherwise clearly defined.

[0038] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0039] FIG. 1 is a conceptual diagram illustrating a service providing system capable of predicting and diagnosing a joint condition according to one embodiment of the present invention, and FIG. 2 is a diagram illustrating a detailed configuration of the service providing system capable of predicting and diagnosing a joint condition illustrated in FIG. 1.

[0040] 1 and 2, a service providing system 1 capable of predicting and diagnosing a joint condition according to an embodiment of the present invention may include a user terminal 10, a management server 20, a service linking terminal 30, and an administrator terminal 40. In this case, the administrator terminal 40 may be omitted.

[0041] Here, the user terminal 10, the management server 20, the service linking terminal 30 and the manager terminal 40 can transmit and receive data in real time in synchronization using a wireless communication network. The wireless communication network may support various long distance communication methods, such as Wireless LAN (WLAN), Digital Living Network Alliance (DLNA), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO), Wideband CDMA (WCDMA), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTEA), Wireless Mobile Broadband Service (WMBS), Bluetooth Low Energy (BLE), and other standards. Various communication methods such as, but not limited to, IEEE 802.11b (IEEE 802.11b), ...

[0042] First, in this embodiment, the service providing system 1 capable of predicting and diagnosing the condition of joints is disclosed as a system for taking a picture of the walking style of a subject, for example, a companion animal, particularly a puppy, and judging whether or not there is an abnormality in the joints, but the present invention is not limited to this. For example, it is possible to predict and measure the joint condition of a person and judge whether or not there is an abnormality, as well as joint-related diseases of various animals including vertebrates such as mammals, birds, reptiles, amphibians, and fish, and invertebrates such as arthropods and mollusks, living with a guardian, companion, or owner (hereinafter referred to as a guardian).

[0043] The user terminal 10 is a portable terminal carried by a guardian of a companion animal 11, and can operate using an application program (or application) in the present disclosure, and such application can be downloaded from an external server or a management server 20 via wireless communication. For example, the user terminal 10 can include various terminals such as a smart phone, a PDA (Personal Digital Assistant), a tablet, a wearable device (including, for example, a smartwatch, a smart glass, a head mounted display (HMD), etc.), and various Internet of Things (IoT) terminals, but is not limited thereto.

[0044] Such a user terminal 10 can recognize the non-running movements of the companion animal 11 and obtain photographed image data that focuses on the walking pattern.

[0045] For example, the captured video data may be a video file of at least 10 seconds captured in a consistent direction for at least 10 seconds of companion animal 11, which is the subject for which prediction and diagnosis of the joint condition is required, walking.

[0046] That is, the user terminal 10 can capture the movement of the companion animal 11 moving from left to right in the same direction, from the front, side, or back.

[0047] Here, the captured image data may include information on photos and / or videos of the state in which the companion animal 11 needs to be managed. For example, the captured image data may include, but is not limited to, a video of the movement of the companion animal 11.

[0048] According to an embodiment, the user terminal 10 can automatically adjust the brightness and clarity of the captured image data in consideration of the surrounding environment, shaking, speed of movement, fur color, and the like.

[0049] In addition, the user terminal 10 can receive and output result data corresponding to the captured image data.

[0050] At this time, the result data may be data including the result of judging whether or not there is an abnormality in the joints of the companion animal 11 using the photographed image data based on the standard data.

[0051] Also, the standard data may be data generated by matching the standard condition information of the companion animal 11 with the standard result data. The standard condition information may include basic information, basic photographing information, and standard coordinate information. The basic information may include guardian information, abandonment information, hospital record information, unique identification number, dog breed, cat breed, sex, age, weight, whether neutered, etc., the guardian information may include contact information, etc., and the hospital record information may include vaccination information, medical information, whether allergies are present, etc. At this time, depending on the embodiment, the hospital record information may include cosmetic information. The basic photographing information may include video information, and the standard coordinate information may be coordinate information displayed on a body part of the companion animal 11 that can determine the presence or absence of joint abnormality corresponding to the video information.

[0052] The standard result data may be data obtained by determining the presence or absence of an abnormality in a joint using basic imaging information.

[0053] According to an embodiment, the standard result data may include the presence or absence of joint abnormality, the progression stage of each disease, etc. In this case, the standard result data may be judged by the naked eye, but is not limited thereto.

[0054] In addition, the user terminal 10 can transmit and receive medical management data, which can include, but is not limited to, recommendation information that can be recommended according to the current state or disease state of the companion animal 11, reservation management information, veterinary clinic link information, and clinic record information.

[0055] For example, when the user terminal 10 outputs medical treatment management data, it can display hospital information such as the location, contact information, and available appointment dates of the hospital corresponding to the recommendation information generated based on the veterinary hospital link information along with a road view, map, or calendar, display hospital record information including medical treatment details and preventive information, and display reservation management information including a reservation completion signal received in response to a reservation request signal from the user terminal 10 on a screen.

[0056] In addition, the user terminal 10 may include a means for visually and audibly displaying the current operating status of the user terminal 10, such as a display that can output symbols, letters, numbers, etc. on a screen depending on the operating status, a lamp that outputs by changing colors or flashing, or a speaker that outputs audio.

[0057] For example, when outputting result data after completing a measurement on the companion animal 11, the display unit can display the result data in O / X, flash the screen in red or green, display guidance text such as "Normal" or "We recommend you visit a vet", or display dictionary information about the result data.

[0058] In addition, the user terminal 10 can store a number of application programs (or applications) operated by the user terminal 10, and data and commands for the operation of the user terminal 10. At least some of these applications can be downloaded from an external server through wireless communication.

[0059] In other words, the user terminal 10 can predict and diagnose the condition of the joints of the companion animal 11 without meeting the owner at any time and place by using a portable terminal. Accordingly, the health condition of the companion animal 11 can be understood and diseases can be treated at an early stage to protect the health of the companion animal 11, thereby increasing convenience and reliability while respecting the diversity of the caregivers.

[0060] The management server 20 may include a data communication unit 200 , a database unit 220 , a monitoring unit 240 and a management control unit 260 .

[0061] When the data communication unit 200 receives the photographed image data from the user terminal 10, it can transmit the result data to the user terminal 10.

[0062] According to the embodiment, when the data communication unit 200 transmits standard data to the user terminal 10, it can receive result data from the user terminal 10.

[0063] In addition, the data communication unit 200 can transmit and receive medical management data between the user terminal 10 and the management server 20 .

[0064] According to an embodiment, the data communication unit 200 can transmit and receive medical management data between the user terminal 10 and the service link terminal 30 .

[0065] The database unit 220 can store data transmitted to and received from the user terminal 10 and the service linking terminal 30 through the wireless communication network. At this time, the standard data can be updated in real time corresponding to the result data and stored.

[0066] The database unit 220 can store data supporting various functions of the management server 20. The database unit 220 can store a number of application programs (or applications) operated by the management server 20, and data and commands for the operation of the management server 20. At least some of these applications can be downloaded from an external server via wireless communication.

[0067] Meanwhile, the standard state information, the photographed image data, the result data, and the standard data used in this embodiment and stored in the database unit 220 may be embodied in the form of a mapping table corresponding to each other, but is not limited thereto.

[0068] The monitoring unit 240 can monitor the operating status of the user terminal 10, the operating status of the management server 20, and data transmitted and received between the user terminal 10 and the management server 20 through a screen. That is, by checking the usage status of the user terminal 10 in real time, it is possible to make parental use more convenient and to give parental confidence.

[0069] When the management controller 260 receives the captured image data from the user terminal 10, it can generate result data based on the standard data.

[0070] Specifically, the management control unit 260 pre-processes the video included in the captured video data, extracts image information for each frame, and inputs the image information for each frame to a pose estimation model among multiple convolution layers of a Convolutional Neural Network (CNN) to extract predicted joint data. The management control unit 260 can compare and analyze the predicted joint data to determine whether or not there is an abnormality in the joints, and generate diagnosis result data.

[0071] For example, when the photographed video data is information on the movement of the companion animal 11 to understand the joint state, the management control unit 260 can extract frame-by-frame image information for joint analysis from the video, which is included in the photographed video data and whose brightness and definition are automatically corrected. The companion animal 11 can be recognized as an object by separating the companion animal 11 from the frame-by-frame image information. After detecting a boundary area centered on the recognized object and setting a bounding box for each frame, predicted coordinate information for each frame of the body part of the object located within the set bounding box can be extracted, and the predicted coordinate information can be labeled based on the object movement direction to extract predicted joint data. At this time, it is preferable that the object movement direction of the object located within the bounding box moves in one direction.

[0072] In this embodiment, the body parts may include, but are not limited to, body parts of the companion animal 11 such as legs, neck, tail, etc.

[0073] In addition, the management control unit 260 may generate a derived variable using the extracted predicted coordinate information, input the generated derived variable to a posture estimation model, and extract predicted joint data. At this time, the extracted predicted joint data may be labeled for each frame based on standard data.

[0074] Specifically, the management control unit 260 generates derived variables for at least one of the following variables: the height of the step between the object's right and left feet, the number of frames corresponding to a change in the object's movement, the angle of at least one of the object's feet, and the movement of the object's neck and tail, and generates predicted joint data using the generated derived variables. Thereafter, the management control unit 260 analyzes the predicted joint data corresponding to the captured image data based on the standard data to generate result data.

[0075] For example, the management control unit 260 may analyze the difference in gait height between the object's right and left feet, and if the difference in height is greater than an average value set based on standard data, may generate result data indicating that the companion animal 11 has an abnormality in its joints.

[0076] In addition, the management control unit 260 can analyze the difference in the number of frames corresponding to changes in the object's movement, and if the number of frames occurs above the average value set based on the standard data, it can generate result data indicating that there is an abnormality in the joints of the companion animal 11.

[0077] In addition, the management control unit 260 analyzes the angle of at least one or more legs of the object, analyzes the difference in the leg angles corresponding to changes in movement, and if an angle difference occurs that is greater than an average value set based on standard data, it can generate result data indicating that there is an abnormality in the joints of the companion animal 11.

[0078] In addition, the management control unit 260 may analyze the movement of the object's neck and tail, analyze the difference in coordinate values ​​corresponding to the change in movement, and if a Mean Squared Error (MSE) occurs that is greater than the average value set based on the standard data, generate result data indicating that there is an abnormality in the joints of the companion animal 11. In this case, the MSE may indicate the mean square error between the y coordinate values ​​(height) of the neck or tail of companion animals 11 with joint abnormalities and companion animals 11 without abnormalities among companion animals 11 of similar size.

[0079] According to an embodiment, the management control unit 260 can match the predicted coordinate information and the standard coordinate information frame by frame based on the standard data, and if the difference is above a preset standard, generate result data indicating that there is an abnormality in the joints of the companion animal 11.

[0080] According to an embodiment, the management control unit 260 can generate derived variables for the height of the step between the object's right and left feet, the number of frames corresponding to changes in the object's movement, the angle of at least one of the object's feet, and the movement of the object's neck and tail, and generate result data corresponding to the captured image data based on the standard data.

[0081] Meanwhile, the management controller 260 can filter the captured image data, and in this case, the filtering can be performed by filtering the degree of shaking of the image using a Laplace filter to obtain an image, but is not limited thereto.

[0082] According to an embodiment, when the management control unit 260 transmits the standard data to the user terminal 10 , the management control unit 260 can receive result data corresponding to the captured image data of the companion animal 11 .

[0083] In addition, the management control unit 260 can generate standard data by matching standard state information and standard result data using deep learning. Although the present embodiment has been described as using deep learning, the present invention is not limited to this, and machine learning techniques such as Random Forest and Support Vector Machine can be used. At this time, the management control unit 260 can update the standard data in real time in response to the result data.

[0084] Specifically, the management control unit 260 can generate standard data by iteratively learning standard state information and standard result data based on a Convolutional Neural Network (CNN) algorithm and verifying suitability.

[0085] For example, the management control unit 260 can generate standard result data by acquiring basic information from a plurality of companion animals 11 and analyzing the acquired basic information. In this case, the basic information can be information acquired from puppies in facilities such as abandoned dog centers and shelters, but is not limited thereto and can also be information acquired from cats in abandoned cat centers.

[0086] In addition, the management control unit 260 can manage medical management data transmitted and received between the user terminal 10 and the service linking terminal 30 based on the result data. Here, the medical management data can include recommendation information that can be recommended depending on the current state or disease state of the companion animal 11, reservation management information, veterinary clinic linking information, and clinic record information.

[0087] For example, when medical treatment is required based on the result data, the management control unit 260 may provide recommendation information generated based on the hospital information to the user terminal 10. At this time, the recommendation information may be information recommending hospital information corresponding to the result data using veterinary clinic link information generated based on the hospital information provided from the service link terminal 30.

[0088] In addition, the management control unit 260 can transmit and receive reservation management information between the user terminal 10 and the service linking terminal 30 .

[0089] For example, the management control unit 260 can transmit a reservation request signal received from the user terminal 10 to the service linking terminal 30, and can transmit a reservation completion signal generated in response to the reservation request signal from the service linking terminal 30 to the user terminal 10.

[0090] In addition, the management control unit 260 can transmit the hospital record information to the user terminal 10. At this time, the management control unit 260 can receive the hospital record information from the service link terminal 30.

[0091] For example, after the medical treatment of the companion animal 11 is completed, the management control unit 260 may transmit hospital record information including medical treatment details information to the user terminal 10, and in general, may transmit hospital record information including preventive information or cosmetic information to the user terminal 10. In addition, the management control unit 260 may transmit notification information regarding the companion animal 11 to the user terminal 10. At this time, the notification information is information generated by the service link terminal 30, and may be, but is not limited to, notification information regarding the medical treatment or cosmetic treatment of the companion animal 11.

[0092] Depending on the embodiment, the management controller 260 may share the hospital record information with other servers.

[0093] According to an embodiment, the management control unit 260 can insert and transmit advertisement information together with data transmitted and received between the user terminal 10, the service link terminal 30, and / or the administrator terminal 40. Accordingly, separate advertising revenue can be generated to support facilities such as abandoned dog centers, abandoned cat centers, and shelters.

[0094] The management server 20 with such a structure can automatically extract diagnostic sites from the photographed image data acquired through the user terminal 10 based on the verified standard data by repeatedly learning the standard result data labeled in response to the standard condition information acquired from a plurality of companion animals 11, compare and analyze the extracted images, analyze a plurality of diseases, and generate result data judging the presence or absence of abnormalities in the joints. As a result, it is possible to solve problems such as unnecessary hospital visits and neglect that may occur when the condition of a companion animal 11 is judged only by the naked eye image.

[0095] In addition, the management server 20 can provide recommendation information to the user terminal 10 according to the current state or disease state of the companion animal 11, thereby enabling the companion animal 11 to be managed quickly and accurately.

[0096] The management server 20 may be implemented as a hardware circuit (e.g., a CMOS-based logic circuit), firmware, software, or a combination thereof. For example, the management server 20 may be implemented using transistors, logic gates, and electronic circuits in the form of various electrical structures.

[0097] The service link terminal 30 may be a plurality of veterinary clinics that manage and treat the health of the companion animals 11, and may use the result data to more quickly treat the companion animals 11.

[0098] The service link terminal 30 can share the hospital record information with the user terminal 10, the management server 20 and a separate server.

[0099] The service linking terminal 30 can provide hospital information and notification information to the user terminal 10 and / or the management server 20.

[0100] According to an embodiment, the service linking terminal 30 may include a separate facility such as a stray dog ​​center, a stray cat center, or a shelter.

[0101] The manager terminal 40 is a terminal owned by a separate manager, and can transmit and receive data in real time by synchronizing with the user terminal 10, the management server 20, and the service link terminal 30 using a wireless communication network. At this time, the manager terminal 40 can transmit and receive data using an application program.

[0102] The administrator terminal 40 can learn the standard data received from the management server 20, analyze the photographed image data received from the user terminal 10, and generate result data corresponding to the photographed image data.

[0103] According to an embodiment, when the administrator terminal 40 receives the photographed image data from the user terminal 10, the administrator terminal 40 can compare and analyze the photographed image data based on the standard data to generate result data.

[0104] According to the embodiment, when result data is generated in the user terminal 10, the administrator terminal 40 can receive the result data from the user terminal 10 and transmit it to the management server 20. Also, when result data is generated in the management server 20, the administrator terminal 40 can receive the result data from the management server 20 and transmit it to the user terminal 10.

[0105] According to an embodiment, the manager terminal 40 can transmit and receive medical management data corresponding to the current state or disease state of the companion animal 11 based on the result data with at least one of the user terminal 10, the management server 20, and the service link terminal 30.

[0106] The administrator terminal 40 may include various portable electronic communication devices that support communication with the user terminal 10, the management server 20, and the service linking terminal 30. For example, as separate smart devices, various terminals such as a smart phone, a PDA (Personal Digital Assistant), a tablet, a wearable device (including, for example, a smartwatch, a smart glass, and an HMD (Head Mounted Display)) and various Internet of Things (IoT) terminals may be included, but are not limited thereto.

[0107] The operation of the service providing system capable of predicting and diagnosing a joint condition according to an embodiment of the present invention having the above-mentioned structure is as follows. Fig. 3 is a diagram for explaining a service providing method capable of predicting and diagnosing a joint condition according to an embodiment of the present invention, Fig. 4 is a diagram for explaining a step of generating standard data shown in Fig. 3, Fig. 5 is a diagram for explaining basic photographing information shown in Fig. 4, Fig. 6 is a diagram for explaining a step of extracting and labeling standard coordinate information shown in Fig. 4, Fig. 7 is a diagram for explaining a step of generating result data shown in Fig. 3, Fig. 8 is a diagram for explaining a difference in height of normal gait shown in Fig. 7, Fig. 9 is a diagram for explaining a difference in height of abnormal gait shown in Fig. 7, Fig. 10 is a diagram for explaining a step of extracting and labeling standard coordinate information shown in Fig. 7, Fig. 11 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 12 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 13 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 14 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 15 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 16 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 17 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 18 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 7, Fig. 19 is a diagram for explaining a step of extracting and labeling normal coordinate information shown in Fig. 8, Fig. 10 is a diagram for explaining a step of extracting and label FIG. 11 is a diagram for explaining the abnormal frame number shown in FIG. 7, FIG. 12 is a diagram for explaining the normal leg angle shown in FIG. 7, FIG. 13 is a diagram for explaining the abnormal leg angle shown in FIG. 7, FIG. 14 is a diagram for explaining normal and abnormal states of neck and tail movement through coordinate information of the front legs shown in FIG. 7, FIG. 15 is a diagram for explaining normal and abnormal states of neck and tail movement through coordinate information of the hind legs shown in FIG. 7, and FIG. 16 is a diagram for explaining normal and abnormal states of neck and tail movement through coordinate information of the front and hind legs shown in FIG. 7.

[0108] First, although the companion animal 11 is disclosed as being limited to a puppy in the embodiment of the present invention, the present invention is not limited to this.

[0109] As shown in FIG. 3, the management server 20 can generate standard data (S10).

[0110] Specifically, the management server 20 can acquire basic information from a plurality of companion animals 11 (S100), as shown in Fig. 4. Here, the basic information can include, but is not limited to, guardian information, abandonment information, hospital record information, unique identification number, dog breed, cat breed, sex, age, weight, neutered or not, etc.

[0111] According to an embodiment, the management server 20 can input basic information of the companion animal 11 using a separate mobile terminal.

[0112] Next, the management server 20 can obtain basic photographing information that captures the movement of the companion animal 11 based on the basic information (S110).

[0113] According to an embodiment, the management server 20 can obtain basic photographic information of the companion animal 11 using a separate photographic tool or scanner.

[0114] Next, the management server 20 can recognize the companion animal 11 included in the basic photographing information as an object and set a bounding box (S120).

[0115] Specifically, the companion animal 11 included in the basic photographing information is separated from the background and recognized as an object, and then a boundary area is detected around the recognized object to set a bounding box for each frame. In other words, by excluding individuals other than the companion animal 11 included in the basic photographing information, false detections that may occur due to individuals can be prevented and the movement of the companion animal 11 can be determined more accurately.

[0116] For example, referring to FIG. 5, the management server 20 can set a bounding box A around a recognized object.

[0117] Next, the management server 20 can set the movement direction of the object located within the bounding box A (S130).

[0118] For example, referring to FIG. 5, the management server 20 can set the moving direction of an object from left to right based on a video file.

[0119] That is, by setting the movement of the object recognized in one direction, the joint state of the object can be determined more accurately.

[0120] Next, the management server 20 can extract and label standard coordinate information B for the body part of the object located within the bounding box A (S140).

[0121] For example, referring to FIG. 6, the management server 20 can extract and label standard coordinate information B for the recognized body part of the object (see FIG. 6(a)), and store the corresponding standard coordinate information B in XML format (see FIG. 6(b)).

[0122] In this embodiment, the video file may be a video file that is recorded over 10 seconds of companion animal 11 walking in a consistent direction.

[0123] According to the embodiment, the management server 20 can extract standard coordinate information B for the body part of the object for each frame (see FIG. 6(c)).

[0124] Next, the management server 20 can generate standard derived information using the extracted standard coordinate information B (S150).

[0125] At this time, the extracted standard coordinate information B is extracted as coordinates in XML format and converted into a file for input to a pose estimation model among multiple convolution layers of a Convolutional Neural Network (CNN), and then a coordinate outlier verification step and a preprocessing step for correction and standardization may be performed.

[0126] Specifically, the management server 20 can generate standard data by generating standard derived variables for at least one of the following variables: the height of the step between the object's right and left feet, the number of frames corresponding to a change in the object's movement, the angle of at least one of the object's feet, and the movement of the object's neck and tail.

[0127] Next, standard parameters are set based on the standard derived variables, and then the standard parameters are input into a posture estimation model to obtain standard joint data. Standard result data corresponding to the obtained standard joint data is generated (S160). Then, the standard result data is iteratively learned, and the generated standard data is verified to generate standard data (S170, S180).

[0128] That is, the management server 20 can match the standard state information with the standard result data and perform iterative learning based on the CNN algorithm to verify compatibility and generate standard data.

[0129] Next, when the guardian requests a diagnosis of the current state or disease state of the companion animal 11, the user terminal 10 can obtain captured image data of the movement of the companion animal 11 (S12).

[0130] According to the embodiment, image data of the movement of the companion animal 11 can be obtained through a separate photographing device.

[0131] In this case, the captured image data may be a video file of at least 10 seconds captured in a consistent direction for at least 10 seconds of the companion animal 11, which is the subject for which the joint condition needs to be predicted and diagnosed.

[0132] Next, the management server 20 can generate result data corresponding to the captured image data based on the standard data (S14).

[0133] Specifically, referring to FIG. 7, the management server 20 pre-processes the moving image included in the photographed image data and extracts corrected images for each frame (S200).

[0134] Specifically, the management server 20 can extract frame-by-frame image information for joint analysis from the video, which is included in the captured video data and has been automatically corrected for brightness, clarity, and the like.

[0135] Next, the management server 20 can obtain predicted coordinate information of the companion animal 11 using the image information for each frame (S210).

[0136] Specifically, the management server 20 separates the companion animal 11 included in the image information for each frame from the background and recognizes it as an object, then detects a boundary area centered on the recognized object, sets a bounding box for each frame, and extracts predicted coordinate information for each frame for the body parts of the object located within the set bounding box.

[0137] According to an embodiment, the management server 20 may label predicted coordinate information based on the object movement direction.

[0138] In this embodiment, the body parts may include body parts of the companion animal 11 such as legs, neck, tail, etc., but are not limited thereto.

[0139] Next, the management server 20 can generate a derived variable using the extracted predicted coordinate information, and input the generated derived variable into a posture estimation model to extract predicted joint data (S220).

[0140] Specifically, the management server 20 can extract predicted joint data by generating derived variables for at least one of the following variables: the height of the step between the object's right and left feet, the number of frames corresponding to a change in the object's movement, the angle of at least one or more of the object's feet, and the movement of the object's neck and tail.

[0141] Specifically, the management server 20 analyzes the difference in gait height between the object's right and left feet, and if the difference in height is greater than an average value set based on standard data (S230), it can generate result data indicating that there is an abnormality in the joints of the companion animal 11 (S270).

[0142] For example, as shown in Figure 8, when a photograph is taken of the companion animal 11 from the front (see Figure 8(a)), when a photograph is taken of the companion animal 11 from the side (see Figure 8(b)), or when a photograph is taken of the companion animal 11 from the back (see Figure 8(c)), if there is little difference in height between the left and right legs based on the corresponding predicted coordinate information, it can be determined that the condition is normal and result data indicating that there is no abnormality in the joints of the companion animal 11 can be generated.

[0143] In contrast, as shown in Figure 9, if a photograph is taken from the front of companion animal 11 (see Figure 9(a)), from the side (see Figure 9(b)), or from behind (see Figure 9(c)), if a large difference in height occurs between the left and right legs based on the corresponding predicted coordinate information, it can be determined that this is an abnormal state and result data can be generated indicating that there is an abnormality in the joints of companion animal 11.

[0144] Next, the management server 20 analyzes the difference in the number of frames of the object's gait, and if the number of frames occurs above the average value set based on the standard data (S240), it can generate result data indicating that there is an abnormality in the joints of the companion animal 11 (S270).

[0145] For example, as shown in Figure 10, when a photograph is taken from the front of companion animal 11 (see Figure 10(a)), when a photograph is taken from the side of companion animal 11 (see Figure 10(b)), or when a photograph is taken from behind companion animal 11 (see Figure 10(c)), if there is little difference in the number of frames through the corresponding predicted coordinate information, it can be determined that the condition is normal and result data can be generated that indicates that there is no abnormality in the joints of companion animal 11.

[0146] In contrast, as shown in Figure 11, when companion animal 11 is photographed from the front (see Figure 11(a)), when companion animal 11 is photographed from the side (see Figure 11(b)), or when companion animal 11 is photographed from the back (see Figure 11(c)), if there is a large difference in the number of frames through the corresponding predicted coordinate information, it can be determined that there is an abnormal state and result data can be generated that indicates that there is an abnormality in the joints of companion animal 11.

[0147] Next, the management server 20 analyzes the difference in leg angle corresponding to the change in the object's movement, and if the leg angle difference is greater than the average value set based on the standard data (S250), it can generate result data indicating that there is an abnormality in the joint of the companion animal 11 (S270).

[0148] For example, as shown in Figure 12, when a photograph is taken of the left leg of companion animal 11 in front (see Figure 12(a)) or the right leg of companion animal 11 in front (see Figure 12(b)), if the difference in leg angle is an average of 1 to 3° based on the corresponding predicted coordinate information, it can be determined that the state is normal and result data indicating that there is no abnormality in the joints of companion animal 11 can be generated.

[0149] In contrast, as shown in Figure 13, when a photograph is taken of the left leg of companion animal 11 in front (see Figure 13(a)) or the right leg of companion animal 11 in front (see Figure 13(b)), if the difference in leg angle is an average of 13 to 16° based on the corresponding predicted coordinate information, it can be determined that there is an abnormal state and result data can be generated indicating that there is an abnormality in the joint of companion animal 11.

[0150] Next, the management server 20 analyzes the movement of the object's neck and tail, and analyzes the difference in coordinate values ​​corresponding to the change in movement. If an MSE (Mean Squared Error) occurs that is greater than the average value set based on the standard data (S260), it can generate result data indicating that there is an abnormality in the joints of the companion animal 11 (S270).

[0151] For example, when photographing the front paws of companion animal 11, if the MSE value is 20 or more based on the corresponding predicted coordinate information, it is judged to be abnormal (see Figure 14(a)), and if the MSE value is 15 or less, it is judged to be normal (see Figure 14(b)), and corresponding result data can be generated.

[0152] In addition, when photographing the hind legs of companion animal 11, if the MSE value is 25 or more based on the corresponding predicted coordinate information, it is judged to be abnormal (see Figure 15(a)), and if the MSE value is 10 or less, it is judged to be normal (see Figure 15(b)), and corresponding result data can be generated.

[0153] In addition, when photographing the front and hind legs of companion animal 11, if the MSE value of the tail is 6 or more and the MSE value of the neck is 8 or more based on the corresponding predicted coordinate information, it is judged to be abnormal (see Figure 16(a)), and if the MSE value of the tail is 4 or less and the MSE value of the neck is 4 or less, it is judged to be normal (see Figure 16(b)), and corresponding result data can be generated.

[0154] Next, the user terminal 10 can receive result data corresponding to the photographed image data from the management server 20 (S16).

[0155] Next, the service linking terminal 30 can provide hospital information based on the standard data (S18).

[0156] Here, the step of providing the hospital information may be performed in advance, but is not limited thereto.

[0157] Next, the management server 20 can generate veterinary clinic linkage information based on the clinic information (S20).

[0158] Here, the step of generating the veterinary clinic linkage information may be performed in advance, but is not limited thereto.

[0159] Next, the management server 20 can generate recommendation information corresponding to the result data based on the veterinary clinic link information and transmit the recommendation information to the user terminal 10 (S22).

[0160] Next, the management server 20 can generate reservation management information (S24).

[0161] For example, the management server 20 can receive a reservation request signal generated based on custom-made information from the user terminal 10, and can receive reservation management information corresponding to the reservation request signal from the service link terminal 30 and transmit it to the user terminal 10.

[0162] Next, the service link terminal 30 can generate and share hospital record information including details of medical treatment for the companion animal 11 (S26).

[0163] At this time, the hospital record information can be transmitted to the user terminal 10 and the management server 20 .

[0164] Finally, the management server 20 can update the standard data in real time in response to the result data (S28).

[0165] The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in a software module executed by hardware, or in a combination thereof. The software module may reside in a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Flash Memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer readable recording medium commonly known in the art to which the present invention pertains.

[0166] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art will understand that the present invention may be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the embodiments described above are illustrative in all respects and are not restrictive. [Explanation of symbols]

[0167] 1: Service provision system that can predict and diagnose joint conditions 11: Companion animal 10: User terminal 20: Management Server 30: Service link terminal 40: Administrator terminal

Claims

1. A method for providing a service capable of predicting and diagnosing a joint condition, the method being performed by a server, extracting image information for each frame from the captured image data of the object; inputting the frame-by-frame image information into a pose estimation model among a plurality of convolution layers of a convolutional neural network (CNN) to extract predicted joint data of the object; and, generating result data for determining whether or not there is an abnormality in the joint of the object by comparing and analyzing the predicted joint data extracted through the posture estimation model based on standard data indicating a normal state; The step of extracting predicted joint data comprises: separating the object from a background from the frame-by-frame image information and recognizing the object as an object; Detecting a boundary area around the object and setting a bounding box for each frame; extracting predicted coordinate information for each frame for a body part of the object located within the bounding box; Extracting an object movement direction of the object located within the bounding box; and labeling the predicted coordinate information based on a moving direction of the object, and extracting the predicted joint data, which is information on a coordinate of a joint, from the predicted coordinate information; The step of extracting predicted joint data comprises: generating a derived variable using the predicted coordinate information; and, inputting the derived variables into the pose estimation model; The step of generating result data comprises: and generating the result data indicating that the joint of the object is abnormal when the predicted coordinate information and standard coordinate information related to the coordinates of the object are matched for each frame based on the standard data and the difference is equal to or exceeds a preset criterion; The derived variables include at least one of a step height between the right and left feet of the object, a frame number corresponding to a change in the movement of the object, an angle of at least one of the object's feet, and a movement of the object's neck and tail. A service providing method for predicting and diagnosing a joint condition, comprising:

2. The step of generating result data comprises: Analyzing the difference in gait height between the right and left feet of the object, and generating result data indicating that the object has an abnormality in the joint when the difference in height is greater than an average value set based on the standard data. A service providing method for predicting and diagnosing a joint condition according to claim 1.

3. The step of generating result data comprises: A difference in the number of frames corresponding to a change in the movement of the object is analyzed, and if the number of frames is greater than an average value set based on the standard data, the result data indicating that an abnormality exists in the joint of the object is generated. A service providing method for predicting and diagnosing a joint condition according to claim 1.

4. The step of generating result data comprises: Analyzing the angle of at least one or more feet of the object, and analyzing the difference in the foot angle corresponding to the change in movement, and generating the result data indicating that the joint of the object is abnormal if an angle difference occurs that is greater than an average value set based on the standard data. A service providing method for predicting and diagnosing a joint condition according to claim 1.

5. The step of generating result data comprises: The movement of the neck and tail of the object is analyzed, and the difference in coordinate values ​​corresponding to the change in movement is analyzed. If a mean squared error (MSE) occurs that is greater than the average value set based on the standard data, the result data indicating that the joint of the object is abnormal is generated. A service providing method for predicting and diagnosing a joint condition according to claim 1.

6. generating said standard data; The step of generating the standard data includes: Extracting images frame by frame from the video; extracting and labeling standard coordinate information for body parts of an object included in the extracted image; generating standard derived variables using the standard coordinate information; inputting the standard derived variables into the pose estimation model; and verifying standard data generated by iteratively learning standard joint data, which is joint data acquired through the posture estimation model, and standard result data corresponding to the standard joint data; The standard derived variables include at least one of a step height between the right and left feet of the object, a frame number corresponding to a change in the movement of the object, an angle of at least one of the object's feet, and a movement of the object's neck and tail. A service providing method for predicting and diagnosing a joint condition according to claim 1.

7. labeling the predicted joint data extracted through the pose estimation model for each object by frame; A service providing method for predicting and diagnosing a joint condition according to claim 1.

8. The captured video data is a video file captured in a consistent direction for 10 seconds or more of the subject walking. A service providing method for predicting and diagnosing a joint condition according to claim 1.

9. A service provision system that can predict and diagnose joint conditions, A user terminal that acquires captured image data from an object to be inspected; and a management server for extracting frame-by-frame image information from captured image data of the object, inputting the frame-by-frame image information into a pose estimation model among a plurality of convolution layers of a convolutional neural network (CNN) to extract predicted joint data of the object, and comparing and analyzing the predicted joint data extracted through the pose estimation model based on standard data indicating a normal state to determine whether or not there is an abnormality in the joints of the object and generating result data of the diagnosis; The management server includes: a frame-by-frame image information storing unit that stores a frame-by-frame image of a subject and a background of the subject; a frame-by-frame image information storing unit that stores a frame-by-frame image of the subject and a background of the subject; a frame-by-frame image information storing unit that stores a frame-by-frame image of the subject and a background of the subject; a frame-by-frame image information storing unit that stores a frame-by-frame image of the subject and a background of the subject; To extract the predicted joint data, Generate a derived variable using the predicted coordinate information; inputting the derived variables into the pose estimation model; To generate the result data, and generating the result data indicating that the joint of the object is abnormal when the predicted coordinate information and standard coordinate information related to the coordinates of the object are matched for each frame based on the standard data and the difference is equal to or exceeds a preset criterion; The derived variables include at least one of a step height between the right and left feet of the object, a frame number corresponding to a change in the movement of the object, an angle of at least one of the object's feet, and a movement of the object's neck and tail. A service provision system capable of predicting and diagnosing the condition of joints.

10. a service linking terminal for sharing medical management data generated in response to the result data; A service providing system capable of predicting and diagnosing a joint condition according to claim 9.

11. an administrator terminal for learning the standard data and generating the result data corresponding to the captured image data; A service providing system capable of predicting and diagnosing a joint condition according to claim 9.

12. A method for performing the method according to claim 1, which is stored in a computer-readable recording medium when combined with a computer that is hardware. A computer program comprising:

Images