Health condition assessment method and device utilizing skeleton model

Abstract

A health condition evaluation method using a skeleton model, which is performed by a health management server, includes the steps of: providing a user device with a standard action image in which a trainer performs a standard action; receiving from the user device a user action image for a user action in which the user imitates the standard action; comparing the standard action image with the user action image to calculate a similarity between the standard action and the user action; and evaluating the health condition of the user based on the calculated similarity.

Description

[Technical Field]

[0001] The embodiments relate to a method and apparatus for assessing health status, and more particularly to a method and apparatus for assessing cognitive impairment, physical exhaustion, etc. by comparing a skeleton model of a standard motion model with motion information of a user's skeleton model. [Background technology]

[0002] As the culture of non-face-to-face communication spreads, the need for telemedicine is increasing. In this context, visual confirmation of the user's movements is necessary for clinical assessment of neurological disorders such as cognitive impairment. However, as the number of hospital visits decreases, there is also the problem of a corresponding decrease in examinations and treatments.

[0003] Meanwhile, home training, which involves exercising at home without visiting a fitness center, is becoming popular, coupled with the trend toward non-face-to-face contact, and the related industry is developing. The improvement in the quality of home training services has increased opportunities for people to engage in various types of exercise, providing motivation for continued exercise. However, as mentioned above, no services have been proposed that can replace hospital visits to diagnose neurological disorders.

[0004] To solve these problems, a service is needed that allows people at home to carry out actions to diagnose nervous system-related diseases and transmit the images to a medical institution to check their health status. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Korean Patent Publication No. 10-2020-0073765 (2020.06.24) Summary of the Invention [Problem to be solved by the invention]

[0006] One object of this specification is to evaluate a user's ability to simulate actions by comparing the user's actions with standard actions.

[0007] Another object of the present invention is to evaluate the degree of cognitive impairment or physical exhaustion of a user by comparing the user's movements with standard movements.

[0008] Another object of this specification is to compare the two actions using skeletal models of user and standard actions.

[0009] Another object of the present invention is to enable rapid clinical judgment by displaying the actions of multiple users and their analysis results on a single screen.

[0010] It is yet another object of the present invention to apply weights to major body parts for each standard action to more accurately determine the similarity between two actions.

[0011] The problem to be solved by this specification is not limited to the above, but can be expanded to various matters that can be derived from the embodiments of the invention described below. [Means for solving the problem]

[0012] A health condition evaluation method using a skeleton model according to one embodiment of the present specification is performed by a health management server, and the method includes the steps of: providing a standard action image in which a trainer performs a standard action to a user device; receiving a user action image from the user device, the user action image corresponding to a user action in which the user imitates the standard action; comparing the standard action image with the user action image to calculate a similarity between the standard action and the user action; and evaluating the health condition of the user based on the calculated similarity.

[0013] In one embodiment, the step of calculating the similarity includes the steps of: acquiring a trainer skeleton movement model for a trainer performing a standard movement from the standard movement image; acquiring a user skeleton movement model for a user performing a user movement from the user movement image; and calculating the similarity by comparing corresponding feature points of the trainer skeleton movement model and the user skeleton movement model for each body part, wherein the feature points of each body part may be nodes constituting the skeleton movement model.

[0014] In one embodiment, the step of comparing the feature points for each body part and calculating the similarity may include the steps of: obtaining a weight for each body part for the standard movement from a database; and applying the weight for each body part to the similarity calculation.

[0015] In one embodiment, the step of assessing the health condition assesses the user's level of physical exhaustion or cognitive impairment, and the cognitive impairment level may be assessed based on behavioral characteristic data that appears during cognitive impairment for each standard movement.

[0016] In one embodiment, the degree of physical exhaustion may be determined based on the sum of the movement trajectories at the feature points of each body part of the user skeleton motion model; and the weight of the physical exhaustion evaluation according to the movement amount of each body part that has already been defined.

[0017] In one embodiment, the method may include steps of receiving user action images for each of a plurality of users from a plurality of user devices; displaying the received plurality of user action images on a display device; calculating a similarity between each of a plurality of user actions in the received plurality of user action images and the standard action; and highlighting at least one of the plurality of user action images displayed on the display device based on the calculated similarity.

[0018] In one embodiment, the highlighting step may highlight a predetermined number of user action images among the plurality of user action images, the calculated similarity of which falls within a range lower or higher than a predetermined standard.

[0019] In one embodiment, the highlighting step may include the steps of: comparing skeleton movement models of the plurality of user movements to calculate a similarity between the users; grouping the plurality of user movement images based on the calculated similarity between the users; and displaying the grouped user movement images in a distinguishable manner.

[0020] In one embodiment, the step of comparing skeleton movement models of the plurality of user movements to calculate the similarity between the users may not apply weights for each body part to the standard movements.

[0021] A program stored on a computer-readable recording medium according to an embodiment of the present specification may be configured with instructions for executing the above-described method. [Effects of the Invention]

[0022] This specification can evaluate the user's ability to simulate movements by comparing the user's movements with standard movements.

[0023] The present invention can evaluate the degree of cognitive impairment or physical exhaustion of a user by comparing the user's actions with standard actions.

[0024] This specification uses skeleton models of user actions and standard actions to allow comparison of the two actions.

[0025] This specification allows for quick clinical decisions by displaying the actions of multiple users and their analysis results on a single screen.

[0026] In this specification, weights are applied to the main body parts of each standard action, so that the similarity between two actions can be determined more accurately.

[0027] The effects of this specification are not limited to the above-mentioned matters, but can be extended to various contents that can be derived from the detailed description of the embodiments of the invention below. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 illustrates an example of an operating environment of a system according to an embodiment of the present specification. [Figure 2] 2 is a block diagram illustrating the internal configuration of a computing device 200 according to an embodiment of the present specification. [Figure 3] 1 illustrates a system environment in which a health status assessment method utilizing a skeleton model according to an embodiment of the present specification is performed. [Figure 4] 1 illustrates a scene in which a user assumes a posture according to a standard motion, according to an embodiment of the present specification. [Figure 5] According to an embodiment of the present specification, the results of extracting skeleton models for the trainer 500 and the user 400 from standard action images and user action images are shown. [Figure 6a] 10A and 10B are diagrams illustrating an example of comparing a trainer skeleton movement model with a user skeleton movement model to calculate a similarity in this specification. [Figure 6b] 10A and 10B are diagrams illustrating an example of comparing a trainer skeleton movement model with a user skeleton movement model to calculate a similarity in this specification. [Figure 6c] 10A and 10B are diagrams illustrating an example of comparing a trainer skeleton movement model with a user skeleton movement model to calculate a similarity in this specification. [Figure 7] FIG. 10 is a diagram illustrating weights for each body part according to an embodiment of the present specification. [Figure 8] 10 shows the angles of the body parts (postures) formed by the nodes according to one embodiment of the present specification. [Figure 9] 1 shows a plurality of user action images and their analysis results provided by the health management server 300 according to an embodiment of the present specification. [Figure 10] 10 illustrates an example of highlighting user action images with low similarity to a standard action according to an embodiment of the present specification. [Figure 11] 10 shows an example in which two or more related user action images are grouped and displayed according to an embodiment of the present specification. [Figure 12] 1 is a flowchart of a health status assessment method utilizing a skeleton model according to an embodiment of the present specification. DETAILED DESCRIPTION OF THE INVENTION

[0029] When describing the embodiments of this specification, if it is determined that a detailed description of a known configuration or function may obscure the gist of the embodiments of this specification, the detailed description will be omitted. In addition, in the drawings, parts that are not related to the description of the embodiments of this specification are omitted, and similar parts are designated by similar reference numerals.

[0030] In the embodiments of this specification, when a component is said to be "coupled," "coupled," or "connected" to another component, this includes not only a direct connection, but also an indirect connection where another component is interposed between them. Furthermore, when a component is said to "include" or "have" another component, this does not exclude the other component, and means that the component may further include other components, unless otherwise specified.

[0031] In the embodiments of this specification, terms such as first and second are used only to distinguish one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of the embodiments of this specification, a first component in an embodiment may be referred to as a second component in another embodiment, and similarly, a second component in an embodiment may be referred to as a first component in another embodiment.

[0032] In the embodiments of this specification, components that are distinguished from one another are used to clearly describe the respective features and do not necessarily mean that the components are separate. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed into multiple hardware or software units. Therefore, even if not otherwise specified, such integrated or distributed embodiments are also included within the scope of the embodiments of this specification.

[0033] The term "network" as used herein may refer to a concept that includes both wired and wireless networks. In this case, the term "network" may refer to a communication network that allows data exchange between devices and systems, and between devices, and is not limited to a specific network.

[0034] The embodiments described herein may have aspects that are all hardware, partly hardware and partly software, or all software. As used herein, terms such as "unit," "device," or "system" refer to a computer-related entity, such as hardware, a combination of hardware and software, or software. For example, a unit, module, device, or system may refer to, but is not limited to, a running process, a processor, an object, an executable, a thread of execution, a program, and / or a computer. For example, both an application running on a computer and the computer may be referred to as a unit, module, device, or system.

[0035] In this specification, a device may refer to not only mobile devices such as smartphones, tablet PCs, wearable devices, and head-mounted displays (HMDs), but also fixed devices such as PCs and home appliances with display functions. As another example, a device may be an in-car cluster or an Internet of Things (IoT) device. That is, a device in this specification may refer to a device capable of running an application, and is not limited to a specific type. Hereinafter, for convenience of explanation, a device on which an application runs will be referred to as a device.

[0036] In this specification, the communication method of the network is not limited, and the components may not be connected using the same network method. The network may include not only a communication method that uses a communication network (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, etc.), but also short-range wireless communication between devices. For example, the network may include all communication methods that allow objects to network with each other, and is not limited to wired communication, wireless communication, 3G, 4G, 5G, or other methods.For example, the wired and / or wireless network may be a LAN (Local Area Network), MAN (Metropolitan Area Network), GSM (Global System for Mobile Network), EDGE (Enhanced Data GSM Environment), HSDPA (High Speed ​​Downlink Packet Access), W-CDMA (Wideband Code Division Multiple Access), CDMA (Code Division Multiple Access), TDMA (Time Division Multiple Access), Bluetooth, ZigBee, Wi-Fi, VoIP (Voice over Internet Protocol), LTE Advanced, IEEE 802.16m, WirelessMAN-Advanced, HSPA+, 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE 802.16e), UMB (formerly EV-DO Rev. C), Flash-OFDM, iBurst and MBWA (IEEE 802.20) systems, HIPERMAN, Beam-Division Multiple Access It may refer to a communication network using one or more communication methods selected from the group consisting of (BDMA), Wi-MAX (World Interoperability for Microwave Access), and ultrasonic communication, but is not limited to these.

[0037] The components described in the various embodiments do not necessarily mean essential components, and some may be optional components. Therefore, an embodiment consisting of a subset of the components described in the embodiments is also included in the scope of the embodiments of the present specification. Note that an embodiment including other components in addition to the components described in the various embodiments is also included in the scope of the embodiments of the present specification.

[0038] Hereinafter, embodiments of the present specification will be described in detail with reference to the drawings.

[0039] Fig. 1 is a diagram showing an example of an operating environment of a system according to an embodiment of the present specification. Referring to Fig. 1, a user device 110 and one or more servers 120, 130, and 140 are connected via a network 1. Fig. 1 is an example for explaining the present specification, and the number of user devices and the number of servers are not limited to those shown in Fig. 1.

[0040] The user device 110 may be a fixed or mobile terminal implemented as a computer system. Examples of the user device 110 include a smartphone, a mobile phone, a navigation system, a computer, a notebook, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, a game console, a wearable device, a smart ring, an internet of things (IoT) device, a virtual reality (VR) device, and an augmented reality (AR) device. As an example, in an embodiment, the user device 110 may represent one of various physical computer systems that can communicate with other servers 120 to 140 over the network 1 using a wireless or wired communication method.

[0041] Each server may be implemented as a computer device or multiple computers that communicate with the user device 110 via the network 1 and provide instructions, code, files, content, services, etc. For example, the server may be a system that provides each service to the user device 110 connected via the network 1. As a more specific example, the server may provide the user device 110 with a service (e.g., information provision) targeted by an application as a computer program installed and run on the user device 110. As another example, the server may distribute files for installing and running the application to the user device 110, receive user input information, and provide the corresponding service.

[0042] 2 is a block diagram illustrating the internal configuration of a computing device 200 according to an embodiment of the present specification. Such computing device 200 may be applied to one or more of the user devices 110-1, 110-2, or servers 120-140 described above with reference to FIG. 1, and each device and server may have the same or similar internal configuration by adding or excluding some components.

[0043] Referring to FIG. 2, the computing device 200 may include a memory 210, a processor 220, a communication module 230, and a transceiver 240. The memory 210 is a non-transitory computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), a read-only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. Here, a non-volatile mass storage device such as a ROM, an SSD, a flash memory, or a disk drive may be included in the above-mentioned device or server as a separate permanent storage device separate from the memory 210. The memory 210 may also store an operating system and at least one program code (e.g., code for a browser installed and operated on the user device 110, or an application installed on the user device 110 to provide a particular service). These software components may be loaded from a computer-readable recording medium separate from the memory 210. Such a separate computer-readable recording medium may include a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc.

[0044] In other embodiments, software components may be loaded into memory 210 via communication module 230 rather than via a computer-readable recording medium. For example, at least one program may be loaded into memory 210 based on a computer program (e.g., the aforementioned application) to be installed by a file provided over network 1 by a developer or a file distribution system (e.g., the aforementioned server) that distributes application installation files.

[0045] Processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 220 by memory 210 or by communication module 230. For example, processor 220 may be configured to execute instructions received by program code stored in a storage device, such as memory 210.

[0046] The communication module 230 can provide the functionality for the user device 110 and the servers 120-140 to communicate with each other over the network 1, and can provide the functionality for each of the device 110 and / or the servers 120-140 to communicate with other electronic devices.

[0047] The transceiver 240 may be a means for interfacing with an external input / output device (not shown). For example, the external input device may include a keyboard, a mouse, a microphone, a camera, etc., and the external output device may include a display, a speaker, a haptic feedback device, etc.

[0048] Furthermore, in other embodiments, computing device 200 may include more components than those shown in Figure 2 depending on the characteristics of the device to which it is applied. For example, when computing device 200 is applied to user device 110, it may be implemented to include at least some of the input / output devices described above, or may further include other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, etc. As a more specific example, when the user device is a smartphone, it may be implemented to further include various components typically included in smartphones, such as an acceleration sensor, a gyro sensor, a camera module, various physical buttons, buttons using a touch panel, an input / output port, and a vibrator for vibration.

[0049] FIG. 3 shows a system environment in which a health condition assessment method using a skeleton model according to an embodiment of the present specification is performed. Referring to FIG. 3, a health management server 300 can communicate with one or more user devices 310-330, and the health management server 300 can also communicate with a medical institution server 301. The health management server 300, the user devices 310-330, and the medical institution server 301 in FIG. 3 can be realized as the computing devices described in FIG. 2. Furthermore, although three user devices are shown in FIG. 3, there may be fewer or more user devices, and the present invention is not limited to this.

[0050] In one embodiment, the health management server 300 may provide various standard motion images to the user device, receive and analyze the user motion images from the user device, and then generate health information based on the user's motion and provide it to the user device or the medical institution server. Alternatively, the health management server 300 may first process or analyze the user motion images and provide them to the medical institution server 301, and then receive clinical judgment information from the medical institution server to additionally analyze the user motion images.

[0051] In one embodiment, the health management server 300 can provide a standard movement image in which the trainer performs the standard movement to the user device 310. The following description will be given using one user device 310 as an example, but the same procedure can be performed for other user devices.

[0052] 4 illustrates a scene in which a user assumes a posture according to a standard movement, according to an embodiment of the present specification. Referring to FIG. 4, a user 400 can perform a standard movement using a user device 401 in response to a standard movement image 510 in which a trainer 500 performs the standard movement. Such an action of the user 400 can be captured by a camera 401c of the user device 401 and recorded as an image.

[0053] In one example, the image captured by the camera 401c may be in various forms such as a color image, a depth image, a color depth image, etc. Therefore, the camera 401c may include a depth camera or may be configured as a stereo camera, but is not limited thereto.

[0054] Such an image is referred to herein as a user action image 410. The user action image 410 may or may not be displayed on the display unit of the user device.

[0055] The playback of the standard action images and the capture of the user's actions can be performed on the user device through an application or web service provided by the health management server 300. Therefore, the health management server can store not only the standard action images but also user action images, information before and after the user's actions, and basic information about the user (such as the user's age, sex, underlying diseases, and lifestyle patterns).

[0056] 4 shows a standard motion with hands together, legs spread apart, and knees bent, but various standard motions can be provided for evaluating the user's health condition. The health management server 300 can receive from the user device 310 a user motion image of the user imitating the standard motion.

[0057] The health management server 300 can then compare the standard motion image with the user motion image to calculate the similarity between the standard motion and the user motion. The health management server 300 can then evaluate the user's health status based on the calculated similarity. For example, the health management server 300 can compare the user motion with the standard motion to evaluate the user's athletic ability, cognitive ability, physical strength, etc., but is not limited thereto.

[0058] In one embodiment, the health management server 300 can calculate the similarity by applying a skeleton model of the object in the image. Figure 5 shows the results of extracting skeleton models for the trainer 500 and the user 400 from the standard movement image and the user movement image, respectively, according to one embodiment of the present specification. These skeleton models may or may not be displayed on the display unit of the user device.

[0059] In one example, the health management server 300 can extract skeletal movement models of the trainer and user from standard movement images or user movement images using a learning model that identifies the positions of human joints or other body parts based on human body images. Such skeletal movement models can be composed of multiple nodes (feature points for each body part). The types of body parts to which nodes are assigned and the number of nodes can be varied as needed, but preferably, 24 nodes can be applied to each joint part. However, a fewer number of nodes are shown in the drawings to simplify and clearly illustrate the features of the invention.

[0060] In this specification, the skeleton movement model can be arranged in two-dimensional space or three-dimensional space. This can be determined depending on the type of image being acquired. In other words, since it is difficult to precisely match the viewpoint of the standard movement image and the viewpoint of the user movement image, at least one of the standard movement image and the user movement image must be constructed using a three-dimensional skeleton movement model to match the viewpoint of the skeleton movement model of the other image. In this case, the health management server 300 can switch the viewpoint of the skeleton movement model acquired in three dimensions to the viewpoint of another skeleton movement model acquired in two dimensions and compare the two skeleton movement models.

[0061] Specifically, in one embodiment, the health management server 300 may perform a step of acquiring a trainer skeleton movement model for a trainer performing a standard movement from the standard movement image. It may also perform a step of acquiring a user skeleton movement model for a user performing a user movement from the user movement image. The health management server 300 may then calculate the similarity by comparing corresponding feature points of the trainer skeleton movement model and the user skeleton movement model for each body part. Here, the feature points of the body parts may be nodes that constitute the skeleton movement model.

[0062] In one example, the similarity is a Euclidean distance and can be determined based on a predetermined formula for the Euclidean distance. That is, the similarity can be measured by comparing the positions of corresponding body part feature points of the user skeleton motion model and the trainer skeleton motion model. In this case, the similarity can be expressed as a percentage.

[0063] As an example, to compare two skeleton models, the distance between corresponding nodes can be calculated by projecting or superimposing the two skeleton models onto a plane.

[0064] 6A and 6B are diagrams illustrating an example of comparing a trainer skeleton movement model and a user skeleton movement model to calculate similarity in this specification. Referring to FIG. 6A, the health management server 300 can calculate the similarity between the acquired trainer skeleton movement model 520 and the user skeleton movement model 420 based on the positions of each node of the two models. As mentioned above, the viewpoint of the user movement image may differ from the viewpoint of the standard movement image. In this case, matching between the two skeleton models is required to compare the positions of the nodes of the skeleton movement models arranged on a two-dimensional plane. While the following describes transforming the user skeleton movement model based on the trainer skeleton movement model, the trainer skeleton movement model can also be transformed to match the user skeleton movement model.

[0065] 6B, the health management server 300 can rotate the user skeleton motion model 420 and convert it to the same time point as the trainer skeleton motion model. To enable such rotation, the user skeleton motion model may need to be configured as a three-dimensional model.

[0066] For example, the physical states of the trainer and the user may differ. In this case, if two skeleton models are compared directly, the results may be highly similar regardless of whether the postures match. To solve this problem, it is necessary to make the sizes of the two skeleton models the same.

[0067] Referring to Figure 6C, the health management server 300 can normalize the size of the user skeleton movement model 420 to correspond to the trainer skeleton movement model 520. Specifically, since Figure 6C corresponds to the case where the user's legs are longer, the distance between the user's ankles and knees can be adjusted to match the trainer's physical condition. This allows the distance between the knee and ankle of each leg to be shortened by d1 and d2.

[0068] The normalization process described in Figures 6A to 6C can be applied to either or both of the two skeleton models, and when applied to both, they can be converted to conform to the ratio or length values ​​of the reference model.

[0069] FIG. 7 is a diagram illustrating weights for each body part according to an embodiment of the present specification. Referring to FIG. 7, in one example, when comparing feature points for each body part to calculate the similarity, the health management server 300 may obtain weights for each body part for the standard movement from a database and apply weights for each body part according to the type of movement to the similarity calculation. That is, depending on the type of standard movement, there may be body parts that are highly important and other parts that are not. Referring to FIG. 7, the standard movement may have high importance in the positions and angles of the neck, waist node, and both knee nodes (identification number 710), while the positions and angles of both arms, wrists, and wrists (identification number 720) may have low importance. Based on these differences in importance, weights for each body part for each standard movement may be reflected in the calculation of the similarity.

[0070] Table 1 shows weight values ​​for body parts for each exemplary standard action, where node n represents each body part. For example, node 3 may be the right elbow and node 5 may be the right shoulder, but this is for illustrative purposes only. Note that the weights 3, 2, and 1 in Table 1 are merely exemplary.

[0071] [Table 1]

[0072] Such weights for each body part are assigned for each standard movement, but can also be subdivided for each cognitive impairment. For example, to evaluate cognitive impairment A for standard movement 4, weights may be assigned to nodes 8 and 9, while to evaluate cognitive impairment B, weights may be assigned to nodes 10 and 12 for standard movement 4. Note that the similarity described herein can be determined not only using the distance between corresponding nodes, but also using the angle formed by three or more nodes. Referring to FIG. 8, the values ​​of the angle a1 between the two legs, the bending angle a2 of the right leg, and the bending angle a3 of the left leg in the user skeleton movement model can be compared with the angles of the corresponding parts of the trainer skeleton movement model.

[0073] Furthermore, since the skeleton movement model has a time element, the calculation of similarity can also take the time element into consideration. For example, if a 2 cm difference is maintained for 1 second and a 0.5 cm difference is maintained for 5 seconds, the latter can be calculated to have a lower similarity value. As an example, the similarity can be determined by applying weights to the product of the distance difference and time or to each of these values. As another example, the similarity can be determined by further reflecting the direction value in addition to the distance difference. The similarity can also be determined by reflecting the angle between the rotation direction of the node in the basic posture and the rotation direction of the actual node.

[0074] In one embodiment, the health management server 300 may evaluate the user's health condition based on the above-described similarity calculation. Here, the evaluation of the health condition may be an evaluation of the user's level of physical exhaustion or cognitive impairment.

[0075] In one example, the level of cognitive impairment can be determined based on the calculated level of similarity. That is, if the similarity to a specific standard posture is lower than a predetermined value, the probability of cognitive impairment corresponding to the specific standard posture can be calculated. Alternatively, node position-movement values ​​according to the cognitive impairment level for each specific standard posture can be defined, and the probability of cognitive impairment and the degree of impairment can be determined by comparing these defined values.

[0076] More specifically, this can be evaluated based on behavioral characteristics data that appear when a person has cognitive impairment for each standard action. That is, if a person is unable to imitate a standard action to a certain level, a probability of cognitive impairment can be given. For more accurate judgment, behavioral characteristics data of people with cognitive impairment for each standard action can also be used.

[0077] For example, the behavioral characteristic data may represent the patient's behavioral characteristics as node data for each standard action and cognitive impairment type, as shown in Table 2. The weights for each body part relative to the standard action may be compiled into a database and stored in the health management server 300 or another external device.

[0078] [Table 2]

[0079] The percentage of similarity may be determined by input by the evaluator, based on predefined criteria, or continuously updated through an artificial intelligence learning model. For example, the health management server 300 may evaluate the user's physical exhaustion level when evaluating the user's health condition. Specifically, the physical exhaustion level is used to identify unnecessary movements or postures that increase the amount of exercise when performing standard movements.

[0080] For example, the degree of physical exhaustion may be determined based on the sum of the movement trajectories of the feature points of each body part of the user skeleton motion model and the weight of the physical exhaustion evaluation according to the movement amount of each body part that has been defined. That is, when a user rotates both arms while imitating a standard movement of sitting with their feet up, the degree of physical exhaustion may be calculated by taking into account the movement trajectories of the nodes of both arms. In another example, a weight may be assigned according to the movement amount of each body part. Specifically, the weight according to the movement amount of each body part may be assigned to the leg lifting action, since the leg lifting action consumes more energy than the arm lifting action. In addition, since the leg lifting action consumes more energy than the thigh thrusting action, the aforementioned weight of physical exhaustion according to each body part may be used to reflect this difference.

[0081] 3, the health management server 300 can receive user action images from multiple user devices. The received user action images may be real-time images or images taken at different times.

[0082] The health management server 300 can display the received user motion images on a display device. Here, the display device can be a user device or a display device of the medical institution server 301. The health management server 300 can provide the images to a user or a medical institution via a web or an app. Hereinafter, a case will be described in which the health management server 300 provides the user motion images and the standard motion image to the medical institution server 301. More specifically, an example will be described in which the results of processing and analyzing each image are provided to the medical institution server together with the image. Based on the provided information, medical personnel can check and diagnose the motion images of the multiple users.

[0083] 9 shows a plurality of user action images and their analysis results provided by health management server 300 according to one embodiment of the present specification. Referring to FIG. 9, standard action image 510 is displayed, and together with this, a plurality of user action images 901 to 905 are displayed on display device 90D.

[0084] The health management server 300 can calculate the similarity between each of the user actions in the received user action images and the standard action. This step was described in detail above. Through the similarity calculation and evaluation, the health management server 300 can display the similarity (accuracy) 910 between each user action and the standard action, the presence, type, and probability of cognitive impairment 920, and the degree of physical exhaustion 930. A specialist can check and compare the actions of multiple patients through these screens.

[0085] FIG. 10 shows an example of highlighting user action images with low similarity to a standard action in one embodiment of the present specification.

[0086] In one example, the health management server 300 may highlight at least one of the plurality of user motion images displayed on the display device based on the calculated similarity. Specifically, as shown in FIG. 10, the health management server 300 may highlight a predetermined number of user motion images (h1) whose calculated similarity falls within a range lower or higher than a predetermined standard. This allows a specialist to quickly identify users who are unable to imitate standard movements. Specifically, the specialist can visually check the overall physical behavior of the highlighted user and make a quick clinical judgment. Without such a guide, it would be inconvenient to have to check the entire playback time of each image.

[0087] FIG. 11 illustrates an example of grouping and displaying two or more related user motion images according to an embodiment of the present specification. In one example, the health management server 300 can compare the skeleton motion models of the multiple user motions to calculate the similarity between the users and group the multiple user motion images based on the calculated similarity between the users. The grouped user motion images can then be displayed separately. That is, user motion images identified in the data as having the same cognitive impairment can be grouped and displayed. While the dotted box in FIG. 11 shows highlights, this is merely an example. The "highlight" may include various visual effects, such as displaying labels, changing shading, or varying the image size. In another example, the health management server 300 may not apply weights for each body part to the standard motion when comparing the skeleton motion models of multiple user motions to calculate the similarity between users. Applying weights for each body part to the standard motion has the advantage of enabling quick and accurate identification of cognitive impairments that are matched with the standard motion. However, in order to discover unconfirmed disorders or user behavior patterns, it may be useful not to apply such weights. For example, by not applying the weights for each body part as described above, a group of users who have a pattern of slightly bending their necks back when imitating standard movement A can be discovered, and a specialist can make a visual evaluation of these and make a clinical judgment. For example, this allows a conclusion that the user has cognitive impairment C to be reached through clinical judgment.

[0088] 12 is a flowchart of a health status evaluation method using a skeleton model according to an embodiment of the present specification. This evaluation method can be performed by the health management server described above. Referring to FIG. 12, the health status evaluation method using a skeleton model may include the steps of: providing a standard action image of a trainer performing a standard action to a user device (S100); receiving a user action image of a user action in which a user imitates the standard action from the user device (S200); comparing the standard action image with the user action image to calculate a similarity between the standard action and the user action (S300); and evaluating the user's health status based on the calculated similarity (S400). Here, the step of calculating the similarity (S300) includes the steps of: acquiring a trainer skeleton movement model for the trainer performing the standard movement from the standard movement image; acquiring a user skeleton movement model for the user performing the user movement from the user movement image; and calculating the similarity by comparing corresponding body part feature points of the trainer skeleton movement model and the user skeleton movement model, wherein the body part feature points may be nodes constituting the skeleton movement model. More specifically, the step of comparing the body part feature points to calculate the similarity may include the steps of obtaining body part weights for the standard movement from a database and applying the body part weights to the similarity calculation. The step of evaluating the health state evaluates the user's level of physical exhaustion or cognitive impairment, and the cognitive impairment level may be evaluated based on behavioral feature data that appears when cognitive impairment occurs for each standard movement.

[0089] The above-described embodiments may be implemented at least in part by a computer program, which may be recorded on a computer-readable recording medium. The computer-readable recording medium on which a program for implementing the embodiments is recorded includes all types of recording devices for storing computer-readable data. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, and optical data storage devices. Furthermore, the computer-readable recording media may be distributed across computer systems connected via a network, so that computer-readable code may be stored and executed in a distributed manner. Furthermore, functional programs, codes, and code segments for implementing the present embodiments are readily understood by those skilled in the art to which this specification pertains.

[0090] The present specification has been described with reference to the embodiments shown in the drawings, but these are merely illustrative, and a person skilled in the art will understand that various modifications and equivalent embodiments are possible. However, such modifications should be considered to be within the technical scope of protection of the present specification. Therefore, the true technical scope of protection of the present specification is determined by the technical ideas of the appended claims. [Industrial Applicability]

[0091] This specification can evaluate the user's movement simulation ability by comparing the user's movement with a standard movement. This specification can evaluate the user's cognitive impairment or physical exhaustion by comparing the user's movement with a standard movement. This specification can compare the two movements using skeleton models of the user's movement and the standard movement.

Claims

1. A health condition evaluation method using a skeleton model, which is performed by a health management server, comprising: providing a standard action image of a trainer performing a standard action to the user device; receiving, from a user device, a user action image corresponding to a user action in which the user imitates the standard action; A step of comparing the standard action image with the user action image to calculate a similarity between the standard action and the user action; and evaluating a health condition of the user based on the calculated similarity; The step of calculating the similarity includes: obtaining a trainer skeleton action model for a trainer performing a standard action from the standard action image; obtaining a user skeleton motion model for a user performing a user motion from the user motion image; and a step of comparing corresponding feature points of body parts of the trainer skeleton movement model and the user skeleton movement model to calculate the similarity, the feature points of body parts being nodes constituting the trainer skeleton movement model and the user skeleton movement model; The step of assessing the health status includes: The method evaluates the user's level of physical exhaustion and cognitive impairment, The cognitive impairment level is evaluated based on behavioral characteristic data that appears during cognitive impairment for each standard movement; The behavioral characteristic data includes: a type of cognitive impairment for each standard action; behavioral characteristics for each type of cognitive impairment; and confirmation target node information for each behavioral characteristic for confirming each behavioral characteristic. A health condition evaluation method utilizing a skeleton model, characterized by:

2. The step of comparing the feature points of each body part and calculating the similarity includes: Obtaining weights for each body part for the standard movement from a database; and applying the weights for each body part to the similarity calculation. A health condition evaluation method utilizing the skeleton model according to claim 1.

3. The degree of physical exhaustion is determined based on the sum of the movement trajectories of the feature points of each body part of the user skeleton motion model; and the weight of the physical exhaustion evaluation according to the movement amount of each body part that has been defined. A health condition evaluation method utilizing the skeleton model according to claim 1.

4. receiving user action images for each of a plurality of users from a plurality of user devices; displaying the received plurality of user action images on a display device; calculating a degree of similarity between each of a plurality of user actions in the received plurality of user action images and the standard action; and and highlighting at least one of the plurality of user action images displayed on the display device based on the calculated similarity. A health condition evaluation method utilizing the skeleton model according to claim 1.

5. The highlighting step includes: A predetermined number of user action images, among the plurality of user action images, whose calculated similarity falls within a range lower or higher than a predetermined standard, are highlighted. A health condition evaluation method utilizing the skeleton model according to claim 4.

6. The highlighting step includes: a step of comparing skeleton motion models of the plurality of user motions to calculate a similarity between the users; Grouping a plurality of user action images based on the calculated similarity between users; and and a step of displaying the grouped user action images in a distinguishable manner. A health condition evaluation method utilizing the skeleton model according to claim 4.

7. The step of comparing the skeleton motion models of the plurality of user motions to calculate the similarity between the users does not apply weights for each body part to the standard motion. A health condition evaluation method utilizing the skeleton model according to claim 6.

8. A computer-readable recording medium for performing the method according to any one of claims 1 to 3 in combination with hardware. A computer program characterized by:

Images