Care support devices, care support programs, care support methods

Abstract

To provide a care support device, a care support program, and a care support method capable of easily and accurately analyzing body motion of a user and supporting a care service.SOLUTION: A care support system is a care support device which supports a care service and in which a user terminal, a server device, an imaging terminal, and a supporter terminal are communicably connected to each other via a communication network. A server device 20 comprises: an evaluation unit which evaluates physical exertion of a user on the basis of first image information containing the physical exertion of the user; an improvement plan information transmission unit which selects a suitable training menu from the physical exertion evaluation and presents the selected menu to a supporter; and a document information generation unit which generates document information on the basis of at least a training menu implementation status of the user.SELECTED DRAWING: Figure 10

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , , ,

[0001] The present disclosure relates to a care support device, a care support program, and a care support method.

Background Art

[0002] Techniques for supporting care services are known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The above-described technique aims to present a suitable rehabilitation menu based on information such as the facilities and personnel information of a care facility, and the care level and dementia level of users receiving care. However, the actual conditions of users vary so much from person to person that it is difficult to present an appropriate rehabilitation menu for each user based on the above-described information.

[0005] Therefore, the present disclosure has been made in view of the above problems, and its object is to provide a technique capable of easily and accurately analyzing the body movements of a user and supporting care operations.

Means for Solving the Problems

[0006] According to the present disclosure, there is provided a care support device for supporting care operations, including an evaluation unit that evaluates the body movement based on first image information including the body movement of a user, an improvement measure information transmission unit that selects a suitable training menu from the evaluation of the body movement and presents it to a supporter, and a document information generation unit that generates document information based on at least the implementation status of the training menu by the user. [Effects of the Invention]

[0007] According to this disclosure, it is possible to provide support for caregiving tasks.

[0008] Further issues and solutions disclosed in this application will be made clear in the section on embodiments of the invention and in the drawings. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall configuration of the care support system according to this embodiment. [Figure 2] This figure shows an example of the hardware configuration of user terminal 10. [Figure 3] This diagram shows an example of the software configuration of user terminal 10. [Figure 4] This figure shows an example of the configuration of body information stored in the body information storage unit 130. [Figure 5] This figure shows an example of the configuration of an evaluation request that the evaluation request transmission unit 112 sends to the server device 20. [Figure 6] This figure shows an example of the configuration of evaluation information received by the evaluation information receiving unit 113 from the server device 20. [Figure 7] This figure shows an example of the configuration of an improvement request that the improvement request transmission unit 116 sends to the server device 20. [Figure 8] This figure shows an example of the structure of improvement plan information. [Figure 9] This figure shows an example of the hardware configuration of the server device 20. [Figure 10] This figure shows an example of the software configuration of the server device 20. [Figure 11] This figure shows an example of the configuration of image information stored in the image data storage unit 231. [Figure 12] This figure shows an example of the configuration of the reference information stored in the reference information storage unit 232. [Figure 13] This figure shows an example of the configuration of evaluation condition information stored in the evaluation condition information storage unit 233. [Figure 14] It is a diagram showing a configuration example of improvement measure information stored in the improvement measure information storage unit 234. [Figure 15] It is a diagram showing the flow of processing executed in the care support device of the present embodiment. [Figure 16] It is a diagram showing an example of an evaluation screen of a video in the weightlifting mode. [Figure 17] It is another diagram showing an example of an evaluation screen of a video in the weightlifting mode. [Figure 18] It is another diagram showing an example of an evaluation screen of a video in the weightlifting mode. [Figure 19] It is another diagram showing an example of an evaluation screen of a video in the weightlifting mode. [Figure 20] It is another diagram showing an example of an evaluation screen of a video in the weightlifting mode.

Mode for Carrying Out the Invention

[0010] The contents of the embodiments of the present invention will be listed and described. The care support device according to the embodiment of the present invention has the following configuration.

[0011] [Item 1] A care support device for supporting care work, An evaluation unit that evaluates the physical movement based on the first image information including the physical movement of the user, An improvement measure information transmission unit that selects a suitable training menu from the evaluation of the physical movement and presents it to the supporter, A document information generation unit that generates document information based on at least the implementation status of the training menu by the user, A care support device, characterized by having the above. [Item 2] An implementation status information acquisition unit that acquires the implementation status information of the training menu based on at least the second image information of the user, Further comprising, The document information generation unit generates the document information based at least on the implementation status information. A care support device as described in item 1, characterized by the above. [Item 3] The aforementioned improvement measure information transmission unit re-selects the training menu based on the first image information and the second image information. A care support device as described in item 1 or 2, characterized by the above. [Item 4] A care support program that assists with caregiving tasks, In the processor, An evaluation step to evaluate the physical movement based on first image information including the user's physical movement, Based on the aforementioned physical movement evaluation, a step of transmitting improvement information is taken to select a suitable training menu and present it to the support staff. A document information generation step that generates document information based at least on the user's implementation status of the training menu, A care support program that enables the implementation of care. [Item 5] A care support method that assists caregiving work, The processor, An evaluation step to evaluate the physical movement based on first image information including the user's physical movement, Based on the aforementioned physical movement evaluation, a step of transmitting improvement information is taken to select a suitable training menu and present it to the support staff. A document information generation step that generates document information based at least on the user's implementation status of the training menu, A care support method characterized by performing the following.

[0012] A care support device according to one embodiment of the present invention evaluates the physical movement of a user of care services (hereinafter referred to as "user") and presents a suitable training menu such as exercise or rehabilitation.

[0013] The care support device of this embodiment may, for example, use images (which may be still images or moving images, but in this embodiment, they are moving images) of a user performing physical exercises to evaluate the movement of the body, including exercises performed with tools. Furthermore, physical exercises performed with the assistance of a caregiver may also be used. The care support device of this embodiment may also identify body parts and evaluate the movement of those body parts based on their relative positional relationships. Alternatively, the care support device of this embodiment may identify tools and parts of tools from images of a user performing physical exercises with tools, and evaluate the movement of the body based on the absolute position of those parts, the relative positional relationships of multiple different parts, and the relative positional relationships between those parts and body parts.

[0014] Figure 1 is a diagram showing an example of the overall configuration of a care support device according to this embodiment. As shown in the figure, the care support device of this embodiment is composed of a user terminal 10 and a server device 20. Furthermore, an imaging terminal 30 may also be included. The user terminal 10, the server device 20 and the imaging terminal 30 are connected to each other so as to be able to communicate via a communication network 40. The communication network 40 is, for example, the Internet or a LAN (Local Area Network), and is constructed using a public telephone network, a dedicated telephone network, a mobile phone network, Ethernet (registered trademark), a wireless communication channel, etc.

[0015] ==User Terminal 10== The user terminal 10 is a computer operated by the user performing physical exercise or their supporter. This supporter may include not only the user's family but also trainers, physical therapists, caregivers, and other individuals who provide guidance, instruction, explanations, and support to the user performing physical exercise. The user terminal 10 can be, for example, a smartphone, tablet computer, or personal computer. The user terminal 10 is equipped with an imaging device such as a camera, which can capture images of the user's body during exercise. In this embodiment, the video images of the user's body during exercise are transmitted from the user terminal 10 to the server device 20. The user terminal 10 may also function as the imaging terminal 30. Furthermore, although only one user terminal 10 is shown in Figure 1, it goes without saying that there may be multiple user terminals 10. The user terminal 10 may be a sensor worn by the user (such as a wearable sensor in the shape of clothing or shoes, or a sensor attached to clothing or a part of the body), and may sense not only physical movement using an accelerometer, but also activity level, conversation volume, sleep duration, pulse rate, UV level, pulse wave interval (PPI), skin temperature, heart rate, etc., and transmit this data to the server device 20 via the communication network 40.

[0016] ==Server device 20== The server device 20 is a computer that evaluates physical movement. The server device 20 can be, for example, a workstation, a personal computer, or a virtual computer logically implemented through cloud computing. The server device 20 receives video footage captured by the user terminal 10 or the imaging terminal 30, analyzes the received video footage, and evaluates physical movement. The server device 20 also makes suggestions regarding measures to improve physical movement. Details of the evaluation of physical movement and the suggestion of improvement measures will be described later.

[0017] ==Imaging terminal 30== The imaging terminal 30 is a computer or camera with communication capabilities that is installed or attached to the location where the user performs physical exercise and is used to image the user. The imaging terminal 30 may be, for example, a smartphone, tablet computer, or personal computer. The imaging terminal 30 is equipped with an imaging device such as a camera, which allows it to image the user's body during physical exercise. In this embodiment, the video images of the user's body during exercise are transmitted from the imaging terminal 30 to the server device 20. The data of the user images stored in the imaging terminal 30 may be directly input to the server device 20 by the user, their supporter, or a business operator using the care support device, or it may be input via the communication network 40.

[0018] ==Supporter Terminal 50== The support terminal 50 is a computer operated by a person (supporter) who is responsible for providing guidance, instruction, explanation, and support to users performing physical exercise, such as a trainer, physical therapist, or caregiver. The support terminal 50 can be, for example, a smartphone, tablet computer, or personal computer.

[0019] In the care support device according to this embodiment, images of the user's physical movements are analyzed. These images may be captured by the user terminal 10, or they may be captured by a separate imaging terminal 30. In the following description, we will explain how each processing unit processes and each storage unit stores images captured by the user terminal 10, but these images may also be captured by the imaging terminal 30.

[0020] In the care support device according to this embodiment, the caregiver terminal 50 may have the same functions as the user terminal 10, and may take images of the user's physical movements on behalf of the user, or request improvement measures. Furthermore, the caregiver may use the functions of the caregiver terminal 50, which are the same as those of the user terminal 10, to present evaluation results and improvement measures to the user. Moreover, all processing that the server device 20 performs on the user terminal 10 may be performed on the caregiver terminal 50, and all processing that the user terminal 10 performs on the server device 20 may be performed on the caregiver terminal 50.

[0021] Figure 2 shows an example of the hardware configuration of the user terminal 10. The user terminal 10 includes a CPU 101, memory 102, storage device 103, communication interface 104, touch panel display 105, and camera 106. The storage device 103 stores various data and programs, such as a hard disk drive, solid-state drive, or flash memory. The communication interface 104 is an interface for connecting to the communication network 40, such as an adapter for connecting to Ethernet®, a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The touch panel display 105 is a device for inputting and outputting data. The user terminal 10 may also be further equipped with input devices such as a keyboard, mouse, buttons, and microphone, and output devices such as a speaker and printer.

[0022] Figure 3 shows an example of the software configuration of the user terminal 10. The user terminal 10 comprises the following functional units: an imaging unit 111, an evaluation request transmission unit 112, an evaluation information reception unit 113, an evaluation display unit 114, a checkpoint display unit 115, an improvement measure request transmission unit 116, an improvement measure information reception unit 117, and an improvement measure information display unit 118, as well as the following storage units: a body information storage unit 130, an image storage unit 131, an evaluation information storage unit 132, and an improvement measure storage unit 133.

[0023] Each of the above-mentioned functional units is realized when the CPU 101 of the user terminal 10 reads a program stored in the storage device 103 into the memory 102 and executes it, and each of the above-mentioned storage units is realized as part of the storage area provided by the memory 102 and storage device 103 of the user terminal 10.

[0024] The imaging unit 111 captures images, including video, of the user during physical movement. By controlling the camera 106, the imaging unit 111 can acquire video footage of the user's physical movement. The user or their assistant can place the user terminal 10 on a flat surface or wall, point the optical axis of the camera 106 towards the area where the user is performing the movement, and instruct the imaging unit 111 to start video recording. The imaging unit 111 then operates the camera 106 to acquire the video footage. The imaging unit 111 stores the acquired video footage in the image storage unit 131.

[0025] The image storage unit 131 stores the images captured by the imaging unit 111. In this embodiment, the images are moving images, but are not limited to that. The image storage unit 131 can store moving images as files, for example.

[0026] The imaging unit 111 captures images of the user's physical movements, which may be done at the time of the initial evaluation or when the user performs the training menu presented by the server device 1, as described later. In either case, the images captured by the imaging unit 111 are stored in the image storage unit 131.

[0027] The body information memory unit 130 stores information about the user's body, physical abilities, and factors that affect training effectiveness (hereinafter referred to as "body information"). Figure 4 shows an example of the composition of body information stored by the body information memory unit 130. As shown in the figure, body information includes, but is not limited to, height, weight, sex, dominant hand, arm length, leg length, hand size, finger length, grip strength, muscle strength, flexibility, shoulder strength, genome, epigenome, gene polymorphisms, gut microbiota, diet, range of motion, medical history, names of diseases currently present, current medical condition, mental and physical state, level of support needed, level of care needed, dementia level, ADL, IDAL, motor function, daily living function, and cognitive function.

[0028] The evaluation request transmission unit 112 sends a request to the server device 20 to perform an evaluation of body movement based on the image captured by the imaging unit 111 (hereinafter referred to as an evaluation request).

[0029] Figure 5 shows an example of the configuration of an evaluation request that the evaluation request transmission unit 112 sends to the server device 20. As shown in the figure, the evaluation request includes a user ID, mode, physical information, and image data. The user ID is information that identifies the user. The mode is information that indicates the exercise to be performed by the user. The mode can be, for example, "muscle training," "range of motion training," "walking rehabilitation," or "improvement of frailty (improvement of specific symptoms or conditions)." The mode is selected from a predetermined set of options. The physical information is physical information stored in the physical information storage unit 130. The image data is moving image data acquired by the imaging unit 111.

[0030] The evaluation information receiving unit 113 receives information regarding the evaluation of physical movement using tools (hereinafter referred to as evaluation information) from the server device 20 in response to an evaluation request. The evaluation information receiving unit 113 registers the received evaluation information in the evaluation information storage unit 132.

[0031] Figure 6 shows an example of the configuration of evaluation information received by the evaluation information receiving unit 113 from the server device 20. As shown in the figure, the evaluation information includes mode, user ID, tool position information, body part position information, posture information, movement information, and checkpoint information.

[0032] The user ID and mode are those included in the evaluation request. The captured images indicate that they are photographs of the user's body performing the exercise indicated by the mode.

[0033] The tool position information indicates the position of each part of the tool in the image (for example, the entire baseball bat, both ends, grip, the point where the ball hits, the center of gravity, and any other arbitrary part; also, the entire barbell, shaft, plates (weight parts), gripped part, center of the shaft, center of gravity, and any other arbitrary part; including the entire tool). The tool position information includes each part of the tool and its position, corresponding to a point in time on the video's time axis. Based on the tool position information, the movement of the tool and its relationship to body parts can be displayed. That is, for example, a shape representing a part (for example, a circle) can be superimposed on the image at the position indicated by the tool position information. Note that the positions of multiple parts may be included for a single point in time. Furthermore, the part connecting two parts (for example, the midpoint between the parts of a barbell gripped by the right and left hands) does not need to be included in the tool position information. In this case, the part connecting these two parts can be represented by drawing a line between a pair of marks (for example, circles) that represent two predetermined parts. The position information of the tool part may be included in each frame that makes up the video, in each keyframe (including frames related to checkpoints, which will be explained later), in every arbitrary number of frames, or in random points in time. If position information is not included in every frame, the shape can be displayed based on the position information of the most recent past point in time.

[0034] Body position information indicates the position of each body part (e.g., head, shoulders, elbows, hips, knees, ankles, etc.) in an image. Body position information includes the body part and its position, corresponding to a point in time on the video's time axis. Based on body position information, the state of the body's skeleton (bones) can be displayed. That is, for example, a shape representing a body part (e.g., a circle) can be superimposed on the image at the position indicated by the body position information. Note that the positions of multiple body parts may be included for a single point in time. Note that body parts connecting two body parts (e.g., the forearm connecting the wrist and elbow, or the thigh connecting the hip and knee) do not need to have position information. In this case, the body parts connecting these two body parts can be represented by drawing a line between a pair of marks (e.g., circles) that represent two predetermined body parts. Position information may be included for each frame that makes up the video, for each keyframe (including frames related to checkpoints, which will be explained later), for any number of frames, or for random points in time. If position information is not included for every frame, the bones can be displayed based on the position information of the most recent past point in time.

[0035] Tool orientation information refers to information about the orientation of the tool being used by the user and the direction in which parts of the tool are facing. This information, linked to a specific point in the video's timeline, includes the part of the tool being evaluated, tool movement values, evaluation rank, and evaluation comments. Tool orientation values ​​represent the orientation of the tool. Examples include the distance from the ground to a certain part of the tool, the distance between two parts of the tool, the angle between parts (the angle formed by a straight line between the first part of the tool and, for example, the part the user is gripping, and a straight line between the second part of the tool and, for example, the part the user is gripping), and the movement of a particular part of the tool. The evaluation rank is a value that expresses the evaluation value in terms of rank. The evaluation rank can be expressed, for example, on a 5-point scale from 1 to 5, or as A, B, C, or B. Evaluation comments are comments related to the evaluation of posture. For example, if the mode is "upright row" and the distances from the ground to the right and left ends of the barbell are different, an evaluation comment such as "Different forces are being applied to the left and right" might be included.

[0036] The tool movement information pertains to the movement of the tool being used by the user. The tool movement information includes the portion of the tool being evaluated, a list of tool orientation values, an evaluation rank, and an evaluation comment, all associated with a period on the video's time axis. The list of tool orientation values ​​is a time-series representation of the tool orientation values ​​within the given period. The evaluation comment is a comment relating to the evaluation of the tool's movement. For example, if the mode is "upright row" and the barbell's up-and-down movement is insufficient, an evaluation comment such as "The barbell is not being lifted sufficiently" might be included.

[0037] Posture information refers to information about the user's body posture. This information includes the body part being evaluated, posture values, evaluation rank, and evaluation comments, all linked to a specific point in time on the video's timeline. Posture values ​​are values ​​that represent posture. Examples include the distance from the ground to a body part, the distance between two body parts, and the angle of a joint (the angle formed by a straight line from the first end to the joint and a straight line from the second end to the joint). The evaluation rank is a value that expresses the evaluation value using a rank system. Evaluation ranks are expressed, for example, on a 5-point scale from 1 to 5, or using A, B, C, etc. Evaluation comments are comments related to the evaluation of posture. For example, if the mode is "lifting" and flexion is insufficient, an evaluation comment such as "hips are not lowered" might be included. (Flexibility such as joint range of motion, numerical values ​​(expressed as angles or distances) → (confirmation) also affects advice.)

[0038] Body movement information is information related to the user's body movements. Body movement information includes the body parts to be evaluated, a list of posture values, an evaluation rank, and an evaluation comment, all associated with a period on the video's time axis. The list of posture values ​​is a time-series representation of posture values ​​within the given period. The evaluation comment is a comment related to the evaluation of the movement. For example, if the mode is "lifting" and knee extension is not smooth, an evaluation comment such as "Knee movement is not smooth" might be included.

[0039] The relational information indicates one or more positional relationship information between the tool and the body. The relational information includes information on the relationship between the position, orientation, and movement of parts of the tool and information on body parts, posture, and movement, corresponding to a point in time on the time axis of the video. For example, the positional relationship between the tool and body can be displayed based on the tool position information and body position information. That is, for example, a figure representing a part (e.g., a circle) can be superimposed on the image at the position indicated by the tool position information, and a figure representing a body part (e.g., a circle) can be displayed at the position indicated by the body position information. Note that the positions of multiple parts and body parts may be included for a single point in time. Note that positional information does not need to be included for points connecting parts and body parts (for example, the center point between the tip of the bat and the part of the body where the user is holding the bat). In this case, the points connecting these two parts and body parts can be represented by drawing lines between pairs of predetermined parts and body part-representing figures (e.g., circles). Relational information may be included for each frame that makes up the video, for each keyframe, for each checkpoint (details will be described later), for any number of frames, or for random points in time. If positional information is not included for each frame, shapes indicating parts or components, and lines connecting them, can be displayed based on the positional information of the most recent past point in time.

[0040] Checkpoint information refers to information that indicates points (hereinafter referred to as checkpoints) in the movement of the equipment used by the user or a series of movements of the user's body where the orientation of the equipment or the posture of the body should be checked. For example, if the mode is "weightlifting," checkpoints may include the point where the barbell reaches its highest position, the point where it reaches its lowest position, and the moment it is lifted. If the mode is "pitching," checkpoints may include the point where the leg is lifted, the point where the lifted leg is lowered and weight is shifted, and the point where the ball is released. The checkpoint information stores information indicating the checkpoint (hereinafter referred to as a checkpoint ID) corresponding to a point in time on the video's time axis. In other words, it is possible to identify the frame (still image) in the video where the checkpoint indicated by the checkpoint ID is displayed.

[0041] The evaluation display unit 114 displays evaluation information. For example, based on tool position, orientation, tool movement information, body position, posture, and body movement information included in the evaluation information, the evaluation display unit 114 can overlay figures representing parts of the tool or body (for example, circles representing the ends of the tool or the center of gravity of the body, and lines connecting them) onto the video, thereby displaying the movement of parts of the tool or body on top of the video. In addition, the evaluation display unit 114 can display graphs showing the time-series changes of things like the position of parts of the tool, the orientation of the tool, the movement of the tool, or the position of body parts, posture, and body movement.

[0042] Furthermore, the evaluation display unit 114 can display an evaluation rank and evaluation comments in conjunction with the video display, based on the tool orientation information and tool movement information included in the evaluation information. For example, the evaluation display unit 114 can display the evaluation rank and evaluation comments included in the tool orientation information when the video playback time reaches around the time included in the tool orientation information (for example, it can be any length such as around 5 seconds). Also, the evaluation display unit 114 can display the evaluation rank and evaluation comments included in the tool movement information when the video playback time reaches within the period included in the tool movement information. In addition, the evaluation display unit 114 can display the posture values ​​included in the posture information. Furthermore, the evaluation display unit 114 can graphically display the time-series changes in tool orientation values ​​based on the list of tool orientation values ​​included in the tool movement information.

[0043] Furthermore, the evaluation display unit 114 can display an evaluation rank and evaluation comments in conjunction with the video display, based on the posture information and movement information included in the evaluation information. For example, the evaluation display unit 114 can display the evaluation rank and evaluation comments included in the posture information when the video playback time reaches around the time included in the posture information (for example, around 5 seconds, or any other length). Also, the evaluation display unit 114 can display the evaluation rank and evaluation comments included in the movement information when the video playback time reaches within the period included in the movement information. In addition, the evaluation display unit 114 can display the posture values ​​included in the posture information. Furthermore, the evaluation display unit 114 can graphically display the time-series changes in posture values ​​based on the list of posture values ​​included in the movement information. Note that the evaluation rank and evaluation comments may be displayed together with the evaluation rank and evaluation comments displayed in conjunction with the video display based on the tool orientation information and tool movement information described in the previous paragraph.

[0044] The checkpoint display unit 115 can extract and display checkpoint images from the video. The checkpoint display unit 115 can read frames corresponding to the time points included in the checkpoint information from the video image data stored in the image storage unit 131 and display them as still images. Alternatively, the checkpoint display unit 115 may, for example, extract and display only the body portion from the read frame.

[0045] The improvement suggestion request transmission unit 116 sends a request (hereinafter referred to as an improvement suggestion request) to the server device 20 to obtain improvement suggestions regarding the handling of tools or physical movement. Figure 7 shows an example of the configuration of an improvement suggestion request. As shown in the figure, the improvement suggestion request includes the user ID, mode, and purpose. The purpose is the purpose for which the user intends to make improvements. For example, the purpose may be "to increase the range of motion of joints," "to increase muscle strength," "to stabilize walking," or "to improve symptoms / conditions." The purpose is selected from a predetermined set of options, and symptoms and conditions may also be selected from a predetermined set of options.

[0046] The improvement measure information receiving unit 117 receives information about improvement measures (hereinafter referred to as improvement measure information) transmitted from the server device 20 in response to improvement measure requests. The improvement measure information receiving unit 117 stores the received improvement measure information in the improvement measure storage unit 133. An example of the configuration of the improvement measure information is shown in Figure 8. As shown in the figure, the improvement measure information includes the purpose, advice, reference information, training menu (training menu, appropriate training intensity, appropriate number of training sessions, recommended implementation posture such as standing or sitting, tailored to the individual's physical information), and the number of sessions. In this embodiment, the advice is assumed to be a string of characters representing the improvement measure, but it may also be content that presents the improvement measure using images or videos. The reference information is the appropriate orientation and movement of the tool (position, orientation, movement, speed, etc. of each part), and the posture of the body (position and angle of each part, etc.). The improvement measure information receiving unit 117 may also receive improvement measure information transmitted by the improvement measure information transmitting unit 216 based on evaluation results and reference values, even without an improvement measure request.

[0047] The improvement measure information display unit 118 displays the improvement measures. The improvement measure information display unit 118 displays the advice included in the improvement measure information. Furthermore, if the improvement measure information includes a suitable position or angle for a part, the improvement measure information display unit 118 may extract an image from the video information of a frame in which the part is at a suitable angle and display it on the user terminal 10.

[0048] Figure 9 shows an example of the hardware configuration of the server device 20. The server device 20 includes a CPU 201, memory 202, storage device 203, communication interface 204, input device 205, and output device 206. The storage device 203 stores various data and programs, such as a hard disk drive, solid-state drive, or flash memory. The communication interface 204 is an interface for connecting to the communication network 40, such as an adapter for connecting to Ethernet®, a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is for inputting data, such as a keyboard, mouse, touch panel, button, or microphone. The output device 206 is for outputting data, such as a display, printer, or speaker.

[0049] Figure 10 shows an example of the software configuration of the server device 20. As shown in the figure, the server device 20 comprises processing units including an evaluation request receiving unit 211, an image analysis unit 212, an evaluation unit 213, an evaluation information transmission unit 214, an improvement measure request receiving unit 215, an improvement measure information transmission unit 216, an implementation status information acquisition unit 217, and a document information generation unit 218, as well as storage units including an image data storage unit 231, a reference information storage unit 232, an evaluation condition information storage unit 233, an improvement measure information storage unit 234, an implementation status information storage unit 235, a template storage unit 236, and a generated document storage unit 237.

[0050] Each of the above-mentioned functional units is realized by the CPU 201 of the server device 20 reading a program stored in the storage device 203 into the memory 202 and executing it, and each of the above-mentioned storage units is realized as part of the storage area provided by the memory 202 and storage device 203 of the server device 20.

[0051] The evaluation request receiving unit 211 receives evaluation requests transmitted from the user terminal 10. The evaluation request receiving unit 211 registers the information including image data included in the received evaluation request (hereinafter referred to as image information) in the image data storage unit 231. Figure 11 is a diagram showing an example of the configuration of image information stored in the image data storage unit 231. As shown in the figure, the image information includes image data associated with a user ID that indicates the user who was photographed. The image data is the one that was included in the evaluation request.

[0052] The reference information storage unit 232 stores information (hereinafter referred to as "reference information") that includes reference values ​​related to the relationship between the tool and the body, such as tool position, tool movement (direction and movement of the tool, etc.), posture (position and angle of the part, etc.), and the relationship between the tool and the body. Figure 12 is a diagram showing an example of the configuration of reference information stored by the reference information storage unit 232. As shown in the figure, the reference information includes, but is not limited to, reference information regarding the absolute position of the tool part when a body movement is performed using the tool, information on how the tool part moved (speed of movement, distance of movement, direction of movement, etc.), reference information regarding the absolute position of the part or the relative position to other parts or other reference objects (hereinafter referred to as "position reference information"), reference information regarding the angle formed by the straight line connecting each of the three parts, including the joint part, and the joint part (hereinafter referred to as "angle reference information"), and information regarding the relationship between the tool part and the body part.

[0053] The equipment position reference information includes the equipment parts and their reference positions, associated with the mode and checkpoint ID. There may be multiple parts. For the position of a part, the vertical position may be, for example, the height from the ground or the distance from either foot. Alternatively, if the mode is "weightlifting," it may be the distance from a body part or a line connecting parts, such as the distance between the line connecting both shoulders and the shaft. The horizontal position of a part may be the distance from a predetermined reference object (for example, a mound plate or a mark on the floor) or the distance from a reference body part such as the shoulder, chest, or feet. The position reference information is assumed to be pre-registered.

[0054] The tool movement reference information includes reference values ​​for information such as the movement speed and distance of tool parts, the direction of movement at a given point in time, and the trajectory of movement over a certain period of time, all associated with the mode and checkpoint ID.

[0055] The position reference information includes the body part and its reference position, associated with the mode and checkpoint ID. There may be multiple body parts. Regarding the position, the vertical position may be, for example, the height from the ground or the distance from either foot. The horizontal position may be the distance from a predetermined reference object (for example, a mound plate or a mark on the floor) or the distance from a reference body part such as the shoulder, chest, or feet. The position reference information is assumed to be pre-registered.

[0056] The angle reference information includes, in association with the mode and checkpoint ID, two body parts (body part 1 and body part 2), one joint body part, a reference angle between the line connecting body part 1 and the joint body part, and the line connecting body part 2 and the joint body part.

[0057] The relational criteria information includes information about criteria that represent the relationship between parts of the tool and parts of the body, associated with the mode and checkpoint ID. The relational criteria information includes information obtained from movement speed, distance traveled, angles, etc., for one or more parts and body components, associated with the mode and checkpoint ID. For example, if the mode is batting, the relational criteria information may include the movement speed of the tip of the bat at the moment of contact with the ball and the angle formed by the bat and the dominant arm holding the bat.

[0058] The evaluation condition information storage unit 233 stores information for performing evaluations (hereinafter referred to as evaluation condition information). Figure 13 shows an example of the configuration of evaluation condition information stored in the evaluation condition information storage unit 233. The evaluation condition information includes categories, conditions, evaluation rank, and comments. The category is the evaluation category. Examples of categories include "muscle strength," "range of motion," and "endurance." The conditions are the position, orientation, or movement (changes in position over time) of each part of the tool in the image, or the position or movement (changes in position over time) of each part of the body. For example, when analyzing weightlifting movements, conditions such as the elbow angle and the speed of arm extension can be set in the evaluation condition information for the checkpoint at the moment the barbell is lifted, as well as conditions such as the movement of the shaft and the speed of up-and-down motion during the period of lifting and lowering the barbell. Also, when analyzing pitching form, conditions such as the elbow angle and arm rotation speed can be set in the evaluation condition information for the checkpoint at the time the ball is released. The evaluation rank is the evaluation value when the above conditions are met. The comments describe the body's posture and movement under the conditions described above.

[0059] The image analysis unit 212 (part / body identification unit) analyzes the image data. The image analysis unit 212 analyzes the image data to extract feature quantities for each part of the tool and each part of the body, and identifies the position of each part and body in the image. The image analysis unit 212 also analyzes the image data to extract feature quantities for each part of the tool and identifies the direction in which each part is facing. A general method of image analysis by the image analysis unit 212 is assumed to be used, and a detailed explanation is omitted here. The image analysis unit 212 may analyze the image data frame by frame or keyframe by keyframe, or it may analyze the image data at checkpoints, or it may analyze it at random timings.

[0060] The image analysis unit 212 also compares the position of each part extracted from the image data with the position reference information stored in the reference information storage unit 232 for each checkpoint ID, and identifies the closest point in time as the checkpoint time.

[0061] In this embodiment, the user's physical movements include those performed by the user with the assistance of a supporter. The image analysis unit 212 should capture the user's body parts or the movement of tools associated with the user's physical movements from images capturing such physical movements. The image analysis unit 212 can perform person detection by analyzing the image data and extracting features, divide the region for each person, and then identify each part and section of the user. Alternatively, the image analysis unit 212 may use instance segmentation to estimate the region for each person on a pixel-by-pixel basis, and then perform posture estimation.

[0062] Furthermore, if both the user and the caregiver are included in the image, the image analysis unit 212 will distinguish between the user and the caregiver and analyze the user's posture. The image analysis unit 212 may, for example, determine the user by identifying the person who appears largest in the video, or by identifying the person closer to the center of the video. Alternatively, the image analysis unit 212 may identify the user based on markers attached to the user's clothing, body surface, hair, etc., or conversely, attach markers to the caregiver and identify the caregiver based on the markers, identifying anyone who is not a caregiver as the user. The image analysis unit 212 may also identify a person holding or wearing a device used for training, etc., as the user. Furthermore, the image analysis unit 212 may identify the user using general facial recognition technology. Finally, the image analysis unit 212 may identify the user by recognizing the characteristics of the user (e.g., an elderly person or a person receiving care). In this case, for example, the image analysis unit 212 can generate a judgment model that has learned the characteristics of walking and movement of elderly people or people with problems in specific body parts, and use this model to determine that people with the characteristics of elderly people or people receiving care are users. Alternatively, the image analysis unit 212 may analyze the posture of all people in the video, present a function to the user terminal or supporter terminal to select a user, accept the user selection from the user or supporter, and identify the user. It should be noted that the above method of determining the user is not limited to cases where the user and supporter are included in the image, but may also be used by the image analysis unit 212 to determine the user when the video includes both the user and people other than the user.

[0063] The image analysis unit 212 can identify the body parts of the supporter without mistaking them for those of the user, even when the user and supporter are included in the image and overlap in the image. For example, the image analysis unit 212 can output multiple candidate joint points for each body part and then group the candidates that belong to the same person as other candidate body parts in post-processing. Alternatively, the image analysis unit 212 may acquire depth using a depth sensor and group joint points belonging to the same person based on the depth information. Furthermore, the image analysis unit 212 may analyze images acquired from multiple imaging terminals, analyze whether there are any inconsistencies in the identification of the same body part in the body parts determined from each image, and integrate the results to suppress false detections. In addition, the image analysis unit 212 may accept manual designation from the user or supporter regarding the detected joint points from the user terminal or supporter terminal and group them together as belonging to the same person. Furthermore, the image analysis unit 212 may group detected body parts as belonging to the same person by using physical characteristics such as joint length between identified joint points, joint range of motion (joint angle constraints), and the degree of curvature of the waist. The image analysis unit 212 may also track each joint using time-series information (the support person should be away from the camera at the start of shooting). Alternatively, using an imaging terminal, the user or support person may be fitted with a marker such as gloves to capture movement of limbs that are prone to false detection, and the detected body parts may be grouped as belonging to the same person based on these markers. Note that the method of grouping detected joint points, body parts, etc., as belonging to the same person as described above is not limited to cases where the user and support person are included in the image, but may also be used by the image analysis unit 212 to group joint points and body parts when the user and other people are captured in a video.

[0064] The evaluation unit 213 evaluates the movement of the tool used by the user based on image data. In this embodiment, the evaluation unit 213 retrieves evaluation condition information from the evaluation condition information storage unit 233, which includes the position of each part of the tool identified from the image data and the conditions that the movement of each part must satisfy. If there is evaluation condition information that satisfies the conditions, it retrieves the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the movement of the tool and count the number of physical movements.

[0065] The evaluation unit 213 evaluates the user's body movements based on image data. In this embodiment, the evaluation unit 213 retrieves evaluation condition information from the evaluation condition information storage unit 233, which includes the conditions that the position and movement of each body part identified from the image data must satisfy. If there is evaluation condition information that satisfies the conditions, it retrieves the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the body movements and count the number of times the body movements occur.

[0066] The evaluation unit 213 evaluates the user's use of tools and body movements based on image data. In this embodiment, the evaluation unit 213 retrieves evaluation condition information from the evaluation condition information storage unit 233, including the positions of each part of the tool and each part of the body identified from the image data, and the conditions that the movement or relationship between the parts and body satisfies. If there is evaluation condition information that satisfies the conditions, it retrieves the evaluation rank and comments contained therein. The evaluation unit 213 may also evaluate the use of the tool and the body movements and count the number of body movements.

[0067] The evaluation information transmission unit 214 transmits evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates tool position information, which includes the time point in the video time axis identified by the image analysis unit 212 and the position of each part of the tool. For the evaluation rank and comments acquired by the evaluation unit 213, if the position of a part of the tool satisfies the conditions, it generates posture information, which includes the time point, part and tool orientation values, evaluation rank and comments. If the movement of the part (change in position over time) satisfies the conditions, it generates tool movement information, which includes a list of the time point, part and tool orientation values, evaluation rank and comments. The evaluation information transmission unit 214 also generates checkpoint information, which includes the time point corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating that checkpoint. The evaluation information transmission unit 214 creates evaluation information, which includes the generated tool position information, tool orientation information, tool movement information and checkpoint information, and transmits it to the user terminal 10. The evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

[0068] The evaluation information transmission unit 214 transmits evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates position information including the time point in the video time axis identified by the image analysis unit 212 and the position of each part. For the evaluation rank and comments acquired by the evaluation unit 213, if the position of the part satisfies the conditions, it generates posture information including the time point, part, posture value, evaluation rank, and comments. If the movement of the part (change in position over time) satisfies the conditions, it generates movement information including a list of time points, part, and posture values, evaluation rank, and comments. The evaluation information transmission unit 214 also generates checkpoint information including the time point corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated position information, posture information, movement information, and checkpoint information and transmits it to the user terminal 10. Note that the evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

[0069] The improvement plan information storage unit 234 stores information related to improvement plans (hereinafter referred to as improvement plan information). Figure 14 shows an example of the configuration of improvement plan information stored in the improvement plan information storage unit 234. As shown in the figure, the improvement plan information includes advice associated with purpose, category, and conditions. The conditions may be conditions for the tool itself (e.g., barbell weight), conditions for how to use the tool, conditions for physical condition (e.g., flexibility), conditions for the position, orientation, and movement of parts of the tool, or conditions for the position and movement of parts of the body.

[0070] The improvement request receiving unit 215 receives improvement request transmissions from the user terminal 10. The improvement request receiving unit 215 may also receive improvement request transmissions from the supporter terminal 50.

[0071] The improvement measure information transmission unit 216 searches for improvement measure information corresponding to the mode and purpose included in the improvement measure request, and for which the user's physical information included in the evaluation request, as well as the position, orientation, movement, etc. of each part or body part identified by the image analysis unit 212, satisfy the conditions. The improvement measure information transmission unit 216 obtains advice from the searched improvement measure information, creates improvement measure information with the purpose and advice set, and responds with the created improvement measure information to the user terminal 10. The improvement measure information transmission unit 216 also transmits the position, orientation, speed, angle, etc. of each part or body part included in the reference information in the improvement measure information. Note that the improvement measure information transmission unit 216 may also search for improvement measures based on evaluation information and reference values ​​even without an improvement measure request, and may transmit such improvement measures to the user terminal 10.

[0072] The improvement measure information transmission unit 216 should transmit improvement measure information corresponding to the mode (symptoms, disease name, condition, etc.) and / or purpose (desire, goal, needs, etc.) to the user terminal 10.

[0073] The improvement measure information transmission unit 216 may search for improvement measure information corresponding to the mode (symptoms, disease name, condition, etc.) and / or purpose (wishes, goals, needs, etc.) that satisfies the conditions, including the user's physical information included in the evaluation request, particularly information such as ADL, range of motion, level of support needed, level of care needed, or the position, orientation, movement, etc. of each part or body part identified by the image analysis unit 212.

[0074] The improvement measure information transmission unit 216 may search for improvement measure information corresponding to the user's motor function, daily living function, and cognitive function, which includes the user's physical information included in the evaluation request, particularly information such as ADL, range of motion, level of support needed, and level of care needed, and which satisfies the conditions regarding the position, orientation, and movement of each part or body part identified by the image analysis unit 212.

[0075] The improvement measure information transmission unit 216 may search for improvement measure information corresponding to the mode (symptoms, disease name, condition, etc.) and / or purpose (hope, goal, needs, etc.) and the user's motor function, daily living function, and cognitive function, among the user's physical information included in the evaluation request, particularly information such as ADL, range of motion, level of support needed, level of care needed, and the position, orientation, movement, etc. of each part or body part identified by the image analysis unit 212, which satisfies the conditions.

[0076] The improvement measure information transmission unit 216 may search for improvement measure information corresponding to the mode (symptoms, disease name, condition, etc.) and / or purpose (hope, goal, needs, etc.) and the items among the user's motor function, daily living function, and cognitive function that have been poorly evaluated, by searching for information that satisfies the conditions of the user's physical information included in the evaluation request, particularly information such as ADL, range of motion, level of support needed, and level of care needed, as well as the position, orientation, movement, etc. of each part or body part identified by the image analysis unit 212.

[0077] The implementation status information acquisition unit 217 acquires the implementation status of improvement measures (training menus such as rehabilitation menus and training menus) transmitted to the user terminal 10 from the user terminal 10 and stores it in the implementation status information storage unit 235. The implementation status information acquisition unit 217 presents the user terminal 10 with a form for inputting the implementation date, number of times, etc., for each training menu, and acquires the input information from the user and their supporter as implementation status information. Alternatively, the implementation status information acquisition unit 217 may acquire images captured by the user terminal 10, and the image analysis unit 212 may analyze these images to obtain information on whether the specified training menu was performed and the number of times the specified training menu was performed. In this case, the image analysis unit 212 can analyze whether the specified training menu was performed by comparing the information stored on how each part of the body or part of the tool moves for each training menu with the information on the movement of each part obtained by analyzing the user's image, and can count the number of times the movement of each part was repeated. Furthermore, the evaluation unit 213 may evaluate the image to assess whether effective training was conducted, and the evaluation information transmission unit 214 may display the evaluation results on the user terminal 10.

[0078] The document information generation unit 218 generates documents to be submitted to the health and welfare departments established in each local government, etc., in order to receive subsidies under the long-term care insurance system in the long-term care business. These documents include documents for applying for or notifying the application for long-term care insurance, and include documents related to designation applications, documents related to remuneration claims, and documents related to guidance and audits. For example, these include, but are not limited to, documents necessary for additional payments such as individual functional training additions, musculoskeletal function improvement additions, ADL maintenance additions, life function improvement collaboration additions, and scientific care promotion additions. Furthermore, these documents may also include documents created and stored by the organization to which the support worker belongs, such as handover documents, reports, medical records (user's condition, monitoring, etc.), training implementation records, daily logs, care plans, assessment results, various plans, care provision schedules, and long-term care benefit statements (records leading to remuneration claims). The document information generation unit 218 may acquire information about the template of the document to be generated, which is stored in the template storage unit 236, and insert the relevant information, such as user information, care provided, and training implementation status (implementation content, number of sessions, etc.), which is stored in the implementation status information storage unit 235, into the template, and output it to the supporter terminal 50 as a Word file, PDF file, etc. Alternatively, the document information generation unit 218 may cooperate with a management system or the like and store the relevant information, such as user information, implementation content, and number of sessions, stored in the implementation status information storage unit 235, in the database of the management system.

[0079] Figure 15 shows an example of the processing flow performed in the care support device of this embodiment.

[0080] In the user terminal 10, the imaging unit 111 receives a mode input, images the user's body during exercise, and acquires video data (S321). The evaluation request transmission unit 112 sends an evaluation request to the server device 20, including the user ID indicating the user, the received mode, body information, and video data (S322).

[0081] When the evaluation request receiving unit 211 in the server device 20 receives an evaluation request, the image analysis unit 212 analyzes the video data to extract features (S323) and identifies the position of each part and body part (S324). Here, the image analysis unit 212 may identify the position on the image, or it may identify the position in actual size (such as the height from the ground or the distance from a reference point such as the center of gravity of the body) using body information. The evaluation unit 213 obtains an evaluation rank and comments from evaluation condition information that satisfies the conditions for the position of each part and body part, and the movement of each part and body part (time-series change of position) (S325). The evaluation information transmission unit 214 creates evaluation information and transmits it to the user terminal 10 (S326).

[0082] On the user terminal 10, the evaluation display unit 114 displays the position, orientation, and movement of the tools on the video data based on the received evaluation information (S327). The evaluation display unit 114 on the user terminal 10 may also display the positions (bones) of each part indicating the posture of the body, as well as the evaluation rank and comments (S327). The evaluation display unit 114 may also display the position, orientation, and movement of the parts, as well as the time-series changes in the position and movement of the parts, in a graph. The checkpoint display unit 115 may also extract and display images of checkpoints from the video. The improvement request transmission unit 116 transmits an improvement request to the server device 20 in response to instructions from the user (S328).

[0083] In the server device 20, when the improvement request receiving unit 215 receives an improvement request transmitted from the user terminal 10, the improvement information transmitting unit 216 searches for improvement information that satisfies the conditions, obtains the advice contained in the searched improvement information (S329), creates improvement information including the obtained advice, and transmits it to the user terminal 10 (S330).

[0084] When the improvement measure information receiving unit 117 in the user terminal 10 receives improvement measure information, the improvement measure information display unit 118 displays the advice included in the received improvement measure information and can also display the appropriate way to use the tool superimposed on video data (S331).

[0085] When the improvement measure information receiving unit 117 in the user terminal 10 receives improvement measure information, the improvement measure information display unit 118 displays the advice included in the received improvement measure information and can also display a suitable body posture superimposed on the video data in the form of bones (S331).

[0086] In the user terminal 10, the imaging unit 111 captures images of the user's body during exercise and acquires additional video data. The user terminal 10 may send an evaluation request to the server device 20 via the evaluation request transmission unit 112, including the user ID, the accepted mode, body information, and video data, or it may send only the video data capturing body movement to the server device 20 (S332). Upon receiving the video data, the implementation status information acquisition unit 217 acquires implementation status information based on the analysis of the video data by the image analysis unit 212 and the evaluation by the evaluation unit 213 (S333). The document information generation unit 218 generates document information based on at least the said implementation status information and a document template stored in the server device 20 (S334).

[0087] As described above, the care support device of this embodiment allows for easy evaluation of physical movement. In particular, for physical movement related to sports, it is possible to evaluate the positional relationships and movements of each part of the equipment and each part of the body, which makes it easier to make concrete improvement efforts and is expected to lead to improved performance. Furthermore, since the care support device of this embodiment also provides comments and advice, users can easily understand their current situation and possible solutions.

[0088] Figure 16 shows an example of a screen displaying the evaluation of physical movement using equipment. Figure 16 illustrates the case where a video is captured in weightlifting mode. As shown in Figure 16, screen 41 displays a mark 411 indicating the position of the barbell shaft. The movement of the barbell shaft is shown by a line 412.

[0089] Figure 17 is another diagram showing an example of a screen displaying an evaluation of physical movement using equipment. Figure 17 illustrates the case where a video is captured in weightlifting mode. In weightlifting mode, the evaluation unit 213 performs an evaluation, and as a result, for example, the angle of the barbell shaft, the distance the shaft travels, and the speed of movement (line 421) are displayed. In Figure 17, the reference value (line 422) is displayed. The actual measurement result (line 423) is displayed, and the difference from the reference value may also be displayed numerically or in a graph. Users can use this as a reference to consider movements and postures that need to be corrected.

[0090] Figure 18 is another diagram showing an example of a screen displaying the evaluation of physical movement using equipment. Figure 18 explains the case when a video is captured in weightlifting mode. In weightlifting mode, the evaluation unit 213 performs an evaluation, and the evaluation results (line 432) are displayed along with reference values ​​(line 431) such as the angles of the body's joints at the lowest point of the barbell. The difference from the reference values ​​may also be displayed numerically or as a graph. Furthermore, the evaluation results based on the relationship between the body and the equipment may also be displayed. Users can use this as a reference to consider movements and postures that need to be corrected.

[0091] Figure 19 is another diagram showing an example of a screen displaying the evaluation of physical movement using equipment. Figure 19 explains the case where a video is captured in weightlifting mode. In weightlifting mode, as a result of the evaluation performed by the evaluation unit 213, line 441 displays bones, which are displayed by connecting predetermined positions of parts of the equipment and body parts identified from the image with lines. Note that the bones may be displayed superimposed on the captured image. Line 442 displays the acceleration of each part of the body. Furthermore, the count result, such as the number of times the weight was lifted, may also be displayed. In addition, as shown by line 443, the evaluation result and the next training result tailored to the objective may be displayed.

[0092] Figure 20 is another diagram showing an example of a screen displaying an evaluation of physical movement using equipment. Figure 20 explains the case where a video is captured in weightlifting mode. In weightlifting mode, the evaluation unit 213 performs an evaluation, and the evaluation (line 441) obtained by comparison with various reference values ​​is displayed.

[0093] Although these embodiments have been described above, they are intended to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included.

[0094] For example, in this embodiment, the server device 20 performs the image analysis, but it is not limited to this, and the user terminal 10 may perform the image analysis to identify the positional relationships of each part and component.

[0095] Furthermore, in this embodiment, the position of a part or section is assumed to be a position on a two-dimensional image, but it is not limited to this and may be a three-dimensional position. For example, if the user terminal 10 is equipped with a depth camera in addition to the camera 106, the three-dimensional position of the part or section can be determined based on the image from the camera 106 and the depth map from the depth camera. Alternatively, for example, the three-dimensional position of the part or section may be determined by estimating the three dimensions from a two-dimensional image. It is also possible to replace the camera 106 with a depth camera and determine the three-dimensional position solely from the depth map from the depth camera. In this case, the user terminal 10 can transmit the depth map to the server device 20 along with the image data, or in place of the image data, and the image analysis unit 212 of the server device 20 can analyze the three-dimensional position.

[0096] Furthermore, in this embodiment, images of the user's body during exercise using the equipment are transmitted from the user terminal 10 to the server device 20. However, the system is not limited to this, and the user terminal 10 may extract features from the image and transmit the features to the server device 20. Alternatively, the user terminal 10 may estimate parts of the equipment and body based on the features, and obtain the absolute position of the parts and body (which may be the position on the XY coordinate system of the image, the actual distance from a reference position (for example, the ground, toes, head, or center of gravity of the body), or the position in any other arbitrary coordinate system) or the relative positional relationships between multiple parts, multiple body parts, or multiple body parts, and transmit these absolute positions and relative positional relationships to the server device 20.

[0097] Furthermore, in this embodiment, the improvement information is provided as content prepared on the server device 20 side. However, this is not limited to this, for example, reference values ​​may be included, and marks or bones representing the correct movements and postures (position and orientation of each part, position and angle of each part, etc.) based on the reference values ​​may be superimposed on the video or still images extracted from the video. This makes it easy to understand what kind of movements and postures should be achieved.

[0098] Furthermore, in this embodiment, the evaluation focuses on the parts and orientation of the tool, the position or movement of body parts (position over time), etc. However, the evaluation is not limited to this, and the position of the tool worn by the user may be identified and evaluated.

[0099] Furthermore, in this embodiment, improvement measures are provided as content such as advice, but for example, tool recommendations may also be provided. In this case, the server device 20 stores reference values ​​for tools and their sizes (length, etc.) in association with the user's physical information (height, weight, etc.), extracts feature quantities of the tools the user is using from image data to identify the shape of the tool, estimates the size of the tool based on the shape and the user's size (e.g., height, etc.) included in the physical information, and if the difference between the estimated size of the tool and the reference value is greater than or equal to a predetermined threshold, a tool of the size of the reference value can be recommended. In addition, tools appropriate to the purpose may be recommended based on information such as conditions for the tool itself (weight of the barbell, etc.), how to use the tool, physical conditions (flexibility, etc.), and the position, orientation, and movement of parts of the tool.

[0100] Furthermore, in this embodiment, improvement measures are provided as content such as advice, but for example, the physical exercise being performed may be interrupted. In this case, the server device 20 stores a reference value for when physical exercise should be interrupted, associated with the user's physical information (purpose, height, weight, etc.), and interrupts the physical exercise if the number of repetitions or speed of the physical exercise being performed by the user (for example, if the speed of lifting the barbell drops drastically, or if too many repetitions are performed at once) deviates from the reference value based on the image data. In this case, a comment may be sent to the user terminal 10 instructing it to stop, or the user may be notified by changing the display, such as turning off the screen, or by emitting an alert sound, or by vibrating.

[0101] Furthermore, in this embodiment, the system provides content such as advice regarding improvement measures, but for example, it may also provide disease diagnosis and physical exercises aimed at improving the condition. In this case, the server device 20 extracts candidate diseases that the user is presumed to have based on the symptoms and evaluation information entered by the user in the physical information, and presents a screening test to narrow down the possibilities. Once the user has taken the screening test and the disease name has been narrowed down, the system may recommend that the user see a doctor, perform physical exercises for improvement, or recommend items such as tools or food for performing physical exercises.

[0102] Furthermore, by estimating the position of the tool's parts, the server device 20 can estimate the tool's speed, acceleration, distance traveled, trajectory, etc. Additionally, by extracting patterns of tool position changes over time, the server device 20 can estimate the number of times a tool is used based on the number of occurrences of these patterns.

[0103] Furthermore, while this embodiment evaluates movement, it is not limited to this; when a certain posture or movement is detected, a task may be suggested for that movement. In this case, the server device 20 can store the task in association with one or a series of postures or movements, instead of providing evaluation comments, and output the task.

[0104] Furthermore, while this embodiment evaluates movement, it is not limited to this. When the movement of a tool, the orientation of a tool, the posture, or the movement of a body part is detected, the system may also present content to improve physical movement according to the purpose, such as training, rehabilitation, performance, or preparatory stages such as stretching, strength training, or posture. In this case, the server device 20 can store the content of the training, etc., in correspondence with the movement of one or a series of tool parts, the orientation of tool parts, the posture of the body, or the movement of body parts, instead of providing evaluation comments, and output the content.

[0105] Furthermore, while this embodiment evaluates movement, it is not limited to this, and the system can also automatically detect actions performed by the user. In this case, the server device 20 stores the position and posture (position of various body parts) of each part of a tool used to perform a predetermined action, such as shooting or passing, as reference information, and can identify the action performed by the user in the image by comparing the position of the tool parts and body parts analyzed from the image with the reference information.

[0106] Furthermore, while this embodiment evaluates movement by analyzing previously captured images, it is not limited to this. Real-time analysis processing may be performed, and when a predetermined movement is detected, recommendations may be made for the next posture or position, the body part to move, the direction, and the number of repetitions. In this case, the correct posture or position is stored in correspondence with the posture or movement, instead of evaluation comments, and the correct posture or position, the difference between the current posture and the correct posture or position, and the actions necessary to bridge that difference can be output.

[0107] Furthermore, in this embodiment, improvement information is presented to the user terminal 10 by extracting improvement measures stored in the improvement information storage unit 234 based on the mode, purpose, physical information, or evaluation results performed by the evaluation unit 213. However, for example, the evaluation unit 213 may evaluate images of multiple training sessions performed at different times and determine whether the evaluation results have improved. Based on this determination, the improvement information transmission unit 216 may present the user terminal 10 with a different training menu or an increase or decrease in the number of sessions compared to the previous time. In this case, the improvement information storage unit 234 compares the results of the evaluation by the evaluation unit 213, which includes the physical movements performed by the user, with the results of the evaluation by the evaluation by the user, which includes the physical movements performed by the user at a different time after the aforementioned time. If the evaluation based on the second image is closer to the standard value than the evaluation based on the first image, it determines that improvement has been achieved. The improvement measure information storage unit 234 determines that there has been no improvement if the evaluation based on the second image is further from the standard value than the evaluation based on the first image. The improvement measure information storage unit 234 determines that there has been no change if the evaluation based on the second image differs from the evaluation based on the first image by only a predetermined value. The improvement measure information storage unit 234 may extract improvement measures again based on the mode, purpose, physical information, evaluation results performed by the evaluation unit 213, as well as the determination results such as improvement, no improvement, or no change, and present them to the user terminal 10.

[0108] Furthermore, in this embodiment, the execution of predetermined functions and the storage of information are performed by the user terminal 10 or the server device 20, but this is not limited to this, and the execution of such functions and the storage of information may be performed by either device. Alternatively, the functional unit and the storage unit may be provided in a distributed manner, in a configuration different from that of this embodiment. [Explanation of symbols]

[0109] 10. User terminals 20 Server Devices 30 Imaging terminals 40 Communication Networks 50 Supporter terminals 111 Imaging Unit 112 Evaluation Request Submission Section 113 Evaluation Information Receiving Unit 114 Evaluation display section 115 Checkpoint display section 116 Improvement Measure Request Submission Section 117 Improvement Measures Information Receiving Unit 118 Improvement information display section 130 Physical information storage unit 131 Image storage unit 132 Evaluation Information Storage Unit 133 Improvement plan storage section 211 Evaluation Request Receiving Unit 212 Image Analysis Department 213 Evaluation Department 214 Evaluation Information Transmission Unit 215 Improvement Measure Request Receiving Unit 216 Improvement Measures Information Transmission Unit 217 Implementation Status Information Acquisition Department 218 Document Information Generation Department 231 Image data storage unit 232 Standard information storage section 233 Evaluation Condition Information Storage Unit 234 Improvement information storage unit 235 Group Analysis Information Storage Unit

Claims

1. A care support device that assists in caregiving tasks, An evaluation unit that evaluates the physical movements based on first image information including the user's physical movements, Based on the evaluation of the physical movements, the improvement measure information transmission unit selects a suitable training menu and presents it to the supporter, An implementation status information acquisition unit acquires implementation status information, including whether or not the training menu was performed and the number of times it was performed, based on second image information including the user's physical movements. A document information generation unit generates document information to be submitted to an administrative agency for claiming care service fees by associating information including at least the implementation status information of the training menu by the user, information about the user, and information about the training menu with a template for the generated document, A care support device characterized by having the following features.

2. The aforementioned document information concerns documents that demonstrate that the requirements for at least one additional payment selected from the group consisting of individual functional training additional payment, musculoskeletal function improvement additional payment, ADL maintenance additional payment, life function improvement collaboration additional payment, and scientific care promotion additional payment. The care support device according to claim 1.

3. A care support program that assists with caregiving tasks, In the processor, An evaluation step to evaluate the physical movement based on first image information including the user's physical movement, Based on the aforementioned physical movement evaluation, a step of transmitting improvement information is taken to select a suitable training menu and present it to the support staff. A step to acquire implementation status information, which acquires implementation status information including whether or not the training menu was performed or the number of times it was performed, based on second image information including the user's physical movements, A document information generation step that generates document information to be submitted to an administrative agency for claiming care service fees by associating information including at least the implementation status information of the training menu by the user, information about the user, and information about the training menu with a template of the document to be generated, A care support program that enables the implementation of care.

4. A care support method that assists caregiving work, The processor, An evaluation step to evaluate the physical movement based on first image information including the user's physical movement, Based on the aforementioned physical movement evaluation, a step of transmitting improvement information is taken to select a suitable training menu and present it to the support staff. A step to acquire implementation status information, which acquires implementation status information including whether or not the training menu was performed or the number of times it was performed, based on second image information including the user's physical movements, A document information generation step that generates document information to be submitted to an administrative agency for claiming care service fees by associating information including at least the implementation status information of the training menu by the user, information about the user, and information about the training menu with a template of the document to be generated, A care support method characterized by performing the following.

Images