Subject evaluation system, subject evaluation device, and subject evaluation program

Abstract

To provide an object person evaluation system, an object person evaluation device and an object person evaluation program capable of performing processing suitable for a plurality of kinds of sensors in various places according to needs of a site.SOLUTION: A target person evaluation system 100 for evaluating a state of a target person 3 includes one or more sensors 2 for measuring the state of the target person 3, an acquiring unit 11 for acquiring target person information and feature information of the target person 3 via the sensors 2, and a converting unit 12 for performing image conversion into an evaluation target image on a two-dimension plane. An evaluation device includes an evaluation part 13 for generating an evaluation result for an evaluation object image, an instruction information acquisition part 14 for acquiring instruction information corresponding to the generated evaluation result by referring to an instruction information database, and an output part 15 for outputting the instruction information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a subject evaluation system, a subject evaluation device, and a subject evaluation program for evaluating the condition of a subject. [Background technology]

[0002] BACKGROUND ART Conventionally, a determination device such as that disclosed in Patent Document 1 has been proposed as a device for supporting evaluation of a subject's behavior.

[0003] The video processing device disclosed in Patent Document 1 includes an acquisition unit that acquires video data of a subject in a room, an analysis unit that analyzes the video data acquired by the acquisition unit and extracts first information about people including the subject and second information about objects in the room, and a tagging processing unit that associates and assigns multiple tags related to the first information and second information extracted by the analysis unit to the video data in order to assist in the evaluation of a specified action performed by the subject, thereby efficiently assisting in the evaluation of the specified action of the person being evaluated. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-001531 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, for example, the video processing device disclosed in Patent Document 1 is based on the premise of analyzing only video data captured by a near-infrared camera of a room. For this reason, it is difficult to perform processing suitable for multiple types of sensors in addition to the near-infrared camera in various locations depending on the needs of the site.

[0006] Therefore, the present invention has been devised in consideration of the above-mentioned problems, and its purpose is to provide a subject evaluation system, subject evaluation device, and subject evaluation program that can perform processing suitable for multiple types of sensors in various locations according to on-site needs. [Means for solving the problem]

[0007] The subject evaluation system according to the first invention is a subject evaluation system for evaluating the condition of a subject, and is characterized by comprising: one or more sensors for measuring the condition of the subject; an acquisition means for acquiring subject information indicating at least one of the subject's physical condition and behavior via the sensors and characteristic information indicating the characteristics of the sensors; a conversion means for image-converting the acquired subject information into an evaluation target image on a two-dimensional plane based on the characteristic information of the sensors; a reference database that stores the correlation between past evaluation target images that have been image-converted in advance and reference information linked to the past evaluation target images; an evaluation means that refers to the reference database and generates an evaluation result for the evaluation target image; an instruction information database that stores past evaluation results and instruction information that instructs a supporter to act in accordance with the evaluation result; an instruction information acquisition means that refers to the instruction information database and acquires the instruction information in accordance with the generated evaluation result; and an output means that outputs the instruction information.

[0008] The subject evaluation system according to the second invention is characterized in that, in the first invention, the correlation is constructed by machine learning using the past evaluation target images and the reference information as learning data.

[0009] The subject evaluation system of the third invention is characterized in that, in the first or second invention, it further comprises an update means for reflecting a new relationship between the past evaluation target image and the reference information in the correlation when the relationship is newly acquired.

[0010] The subject evaluation system of the fourth invention is characterized in that, in the first invention, the instruction information includes projection destination information onto which the instruction information is projected, and the output means outputs the instruction information based on the projection destination information.

[0011] The subject evaluation system according to the fifth invention is characterized in that, in the first invention, the instruction information includes information about a supporter who will provide support to the subject, and the output means outputs the instruction information based on the supporter information.

[0012] The subject evaluation system of the sixth invention is characterized in that, in the first invention, the instruction information includes suggestion information that suggests an action to the supporter, and the output means outputs the instruction information based on the suggestion information.

[0013] The subject evaluation system according to the seventh invention is the first invention, further comprising a behavioral evaluation means for evaluating the behavior of the supporter, wherein the behavioral evaluation means acquires behavioral information indicating whether or not the supporter has taken action in response to the output instruction information, and evaluates the behavior resulting from the instruction information based on the behavioral information.

[0014] The subject evaluation device of the eighth invention is a subject evaluation device that evaluates the condition of a subject, and is characterized by comprising: one or more sensors that measure the condition of the subject; an acquisition unit that acquires subject information indicating at least one of the subject's physical condition and behavior via the sensors; feature information that indicates the features of the sensors; a conversion unit that image-converts the acquired subject information into an evaluation target image on a two-dimensional plane based on the feature information of the sensors; a reference database that stores the correlation between past evaluation target images that have been image-converted in advance and reference information linked to the past evaluation target images; an evaluation unit that references the reference database and generates an evaluation result for the evaluation target image; an instruction information database that stores past evaluation results and instruction information that instructs a supporter to act in accordance with the evaluation results; an instruction information acquisition unit that references the instruction information database and acquires the instruction information according to the generated evaluation result; and an output unit that outputs the instruction information.

[0015] The subject evaluation program of the ninth invention is a subject evaluation program for evaluating the condition of a subject, and is characterized in that it causes a computer to execute the following steps: a measurement step of measuring the condition of the subject using one or more sensors; an acquisition step of acquiring subject information indicating at least one of the subject's physical condition and behavior via the sensors and feature information indicating the features of the sensors; a conversion step of image-converting the acquired subject information into an evaluation target image on a two-dimensional plane based on the feature information of the sensors; a first storage step of storing in a reference database the correlation between a previous evaluation target image that has been image-converted in advance and reference information linked to the previous evaluation target image; an evaluation step of referring to the reference database to generate an evaluation result for the evaluation target image; a second storage step of storing in an instruction information database the past evaluation results and instruction information that instructs a supporter to act in accordance with the evaluation result; an instruction information acquisition step of referring to the instruction information database to acquire the instruction information in accordance with the generated evaluation result; and an output step of outputting the instruction information. [Effects of the Invention]

[0016] According to the first to seventh inventions, the acquisition means acquires subject information and characteristic information from one or more sensors that measure the subject's condition. Therefore, subject information indicating at least one of the subject's physical condition and behavior, and characteristic information indicating the characteristics of the sensors can be acquired via multiple sensors. This makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0017] Furthermore, according to the first to seventh inventions, the conversion means converts the acquired subject information into an evaluation target image on a two-dimensional plane based on the characteristic information of the sensor. Therefore, it is possible to generate an evaluation result for the evaluation target image by referring to the reference database. This makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0018] In particular, according to the first aspect of the present invention, the instruction information database stores past evaluation results and instruction information that instructs the supporter on how to act in accordance with the evaluation results. Therefore, the instruction information acquisition means can refer to the instruction information database and acquire instruction information corresponding to the generated evaluation results. This makes it possible to give instructions appropriate to the projection destination and the supporter in various locations according to the needs of the site.

[0019] In particular, according to the second invention, correlations are constructed by machine learning using past evaluation target images and reference information as learning data. Therefore, quantitative evaluation can be performed even when evaluating unknown evaluation target images that are different from past evaluation target images. This makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0020] In particular, according to the third aspect of the present invention, when a relationship between an image to be evaluated and reference information is newly acquired, the update means reflects the relationship in the correlation. Therefore, even when evaluating a new image to be evaluated, quantitative evaluation can be performed. This makes it possible to further improve the evaluation accuracy and perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0021] In particular, according to the fourth aspect of the present invention, the instruction information includes projection destination information to which the instruction information is to be projected. Therefore, the instruction information can be output based on the projection destination information. This makes it possible to project the instruction information based on the projection destination information, and to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0022] In particular, according to the fifth aspect of the present invention, the instruction information includes information about a supporter who will be supporting the subject. Therefore, the output means can output the instruction information based on the supporter information. This makes it possible to project the instruction information based on the supporter information, and to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0023] In particular, according to the sixth aspect of the present invention, the instruction information includes suggestion information that suggests an action to the supporter. Therefore, the output means can output the instruction information based on the suggestion information. This makes it possible to project the instruction information based on the suggestion information, and to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0024] In particular, according to the seventh aspect of the present invention, the behavior evaluation means acquires behavior information indicating whether or not the supporter has performed a behavior in response to the output instruction information. Therefore, the behavior resulting from the instruction information can be evaluated based on the behavior information. This makes it possible to improve the accuracy of evaluating the supporter's behavior and to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0025] According to the eighth aspect of the present invention, the evaluation unit references a reference database and generates an evaluation result for the image to be evaluated. The reference information includes physical information. This allows the evaluation result to be generated based on past evaluations of the subject's condition. This improves the accuracy of evaluating the subject's condition and enables processing suitable for multiple types of sensors in various locations depending on on-site needs.

[0026] According to the ninth aspect of the present invention, the evaluation step refers to a reference database to generate an evaluation result for the image to be evaluated. The reference information includes physical information. Therefore, it is possible to generate an evaluation result based on the results of past evaluations of the subject's condition. This makes it possible to improve the accuracy of evaluating the subject's condition and to perform processing suitable for multiple types of sensors in various locations depending on on-site needs. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a schematic diagram showing an example of a subject evaluation system according to this embodiment. [Figure 2] Fig. 2(a) is a schematic diagram showing an example of the operation of the subject evaluation device of this embodiment at location A. Fig. 2(b) is a schematic diagram showing an example of the operation of the subject evaluation device of this embodiment at location B. [Figure 3] Fig. 3(a) is a schematic diagram showing an example of sensor data of the subject evaluation system of this embodiment, and Fig. 3(b) is a schematic diagram showing an example of an evaluation target image of the subject evaluation system of this embodiment. [Figure 4] FIG. 4(a) is a schematic diagram showing an example of the configuration of a subject evaluation device in this embodiment, and FIG. 4(b) is a schematic diagram showing an example of the function of a subject evaluation device in this embodiment. [Figure 5] FIG. 5 is a schematic diagram showing an example of the reference database in this embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the operation of the subject evaluation system in this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, an example of a subject evaluation system 100 and a subject evaluation device 1 according to an embodiment of the present invention will be described with reference to the drawings.

[0029] An example of a subject evaluation system 100 and a subject evaluation device 1 according to this embodiment will be described with reference to FIG.

[0030] (Subject Evaluation System 100) 1, the subject evaluation system 100 in this embodiment has a subject evaluation device 1. The subject evaluation device 1 is connected to, for example, sensors 2 (2a to 2f), and may also be connected to another terminal 5 or a server 6 via, for example, a communication network 4.

[0031] The subject evaluation system 100 evaluates the state of the subject 3 at location A or location B, generates an evaluation result, and acquires instruction information by referring to the database of the server 6 based on the evaluation result. The subject evaluation system 100 presents the acquired instruction information to an appropriate projection destination, the supporter 7, at location A or location B. The subject evaluation system 100 can be used, for example, for evaluations made after observing the subject 3 at location A or location B (for example, observations made by a caregiver in a care facility or at home), as well as for giving instructions to a supporter 7 who cares for and supports the subject 3.

[0032] Furthermore, the subject evaluation system 100 can be used to provide instructions to the support staff 7 regarding the evaluation of the subject 3's condition while working in various situations, such as transferring, eating, bathing, administering medication, using the toilet, rehabilitation, checking vital signs, suctioning phlegm, providing respiratory support, intravenous drips, blood transfusions, etc., as well as delivery work at a logistics site, assembly work at a manufacturing site, and sales work at a sales site.

[0033] Furthermore, the subject evaluation system 100 may be configured to evaluate a trend based on the current condition of the subject 3. The subject evaluation system 100 can also be used to evaluate, for example, "there is a tendency for the condition to improve" based on an evaluation made after observing the subject 3 at a care facility, or "there is a tendency for the level of care required to increase if the current situation remains unchanged" based on an evaluation made after observing the subject 3 in home care.

[0034] Furthermore, the subject evaluation system 100 may obtain appropriate instruction information from the database of the server 6 depending on, for example, the projection destination (e.g., the material and condition of the walls and ceiling of room A or room B, the brightness of the room, the position of the windows, the presence or absence of curtains or a projection screen, and the material, etc.), the attributes of the support person 7 (e.g., family member, non-family member, etc.), experience and skills (e.g., presence or absence of experience in support or caregiving, number of years, etc.), and the status of the work (e.g., currently working, scheduled work, etc.).

[0035] For example, if the generated evaluation result for subject 3 is "supine posture: ○ time elapsed," subject evaluation system 100 refers to the instruction information database and acquires, for example, "change posture" as instruction information to be linked as instruction information for support person 7. Furthermore, if the generated evaluation result for subject 3 is "blood and lymph flow: decreased," subject evaluation system 100 acquires, for example, "perform massage" as instruction information to be linked as instruction information for support person 7.

[0036] The subject evaluation system 100 outputs the acquired instruction information. The instruction information may be output from a projection device such as a monitor, projector, or laser irradiator provided in the room, or from various known devices provided by the supporter 7 (for example, a smartphone, tablet, head-mounted display, smart glasses, etc.).

[0037] The subject evaluation system 100 may further acquire, for example, information regarding the state of the room, the projection destination, information regarding the supporter 7 assisting the subject 3, and information regarding the posture of the supporter 7, via sensors 2 (2a to 2f) installed in a room (such as location A or location B). The subject evaluation system 100 may, for example, determine the state of the room and the posture information of the supporter 7, and output the acquired instruction information after adjusting it so that it is displayed appropriately in accordance with the state of the room and the posture of the supporter 7.

[0038] Furthermore, the subject evaluation system 100 further acquires behavioral information regarding the behavior of the support person 7, for example, via sensors 2 (2a to 2f) installed in a room (such as location A or location B). The subject evaluation system 100 evaluates, for example, whether or not the support person 7 performs a confirmation behavior resulting from the output of instruction information by the support person 7. The subject evaluation system 100 may request a specific behavior (such as a gesture, pose, posture, mannerism, or call) from the support person 7 as a confirmation of the output of instruction information, and evaluate the result of the request (such as "understanding the instruction information," "completing the instruction information," "redisplaying the instruction information," or "unable to execute the instruction information") via sensors 2 (2a to 2f).

[0039] Here, an example of the operation of the subject evaluation system 100 at place A will be described with reference to Fig. 2. First, Fig. 2(a) shows an example of the operation of the subject evaluation system 100 targeting, for example, place A (bedroom). As shown in Fig. 2(a), the subject evaluation system 100 measures data relating to the state of the subject 3 while sleeping using, for example, one or more sensors 2 (2a to 2f) that measure the state of the subject 3 while sleeping.

[0040] The sensor 2 may be, for example, sensor 2a, a motion sensor that measures the movement of the subject 3 or the supporter 7, and quantifies the movement of the subject 3 or the supporter 7. Sensor 2b may be, for example, a near-infrared or non-contact vital sensor that measures the body temperature and vital signs of the subject 3, and quantifies the body temperature and vital signs of the subject 3. Sensor 2c may be, for example, a near-infrared or non-contact vital sensor that measures the posture, center of gravity, etc. of the subject 3, and quantifies the body temperature and vital signs of the subject 3. Sensor 2c may be attached to, for example, the leg of a bed, and may measure the body orientation, posture, and center of gravity of the subject 3, and quantify the body orientation, posture, and center of gravity of the subject 3. Sensors 2a to 2e may measure movements that indicate the behavior of the supporter 7, for example.

[0041] The subject evaluation system 100 acquires various types of measurement data measured by each sensor 2 (2a to 2c). For example, when acquiring various types of measurement data measured by each sensor 2 (2a to 2c), the subject evaluation system 100 also acquires characteristic information indicating the characteristics of each sensor (e.g., sensor ID, measurement data ID, measurement data characteristics, measurement date and time, measurement location, etc.). For example, the subject evaluation system 100 may set acquisition conditions, such as the date and time of transmission to the subject evaluation system 100 and the measurement data to be transmitted, in advance for each sensor 2 (2a to 2c), and each sensor 2 may transmit the data to the subject evaluation system 100 based on the set acquisition conditions.

[0042] Next, Fig. 2(b) shows an example of the operation of the subject evaluation system 100 targeting, for example, location B (living room). As shown in Fig. 2(b), the subject evaluation system 100 measures data on the behavioral state of the subject 3 when the subject 3 wakes up and moves from location A using one or more sensors 2 (2e to 2f), as well as data on the behavioral state of the supporter 7 who supports the subject 3.

[0043] Sensor 2, for example, sensor 2e, is a motion sensor that measures the posture and movement of subject 3 and supporter 7, and quantifies the posture and movement of subject 3 and supporter 7. Sensor 2f, for example, is a wearable sensor worn by subject 3 and supporter 7, and measures and quantifies the number of steps, limb movements, heart rate, breathing rate, etc. of subject 3.

[0044] The sensors 2 (2a to 2f) shown in FIGS. 2(a) and (b) may be installed in multiple locations, for example, in a facility or home. They may be installed, for example, on ceilings, walls, tables, beds, chairs, wheelchairs, automobiles, etc., or may be worn by evaluators or supporters. If the installation location is, for example, a bathroom, image data of the subject's 3's physical condition may not be acquired, but sensor data may be acquired using a near-infrared camera or the like. This makes it possible to acquire the subject's 3's condition using multiple types of sensors at the site or in various locations depending on the site's needs.

[0045] The subject evaluation system 100 acquires various types of measurement data measured by each sensor 2 (2e to 2f). For example, when acquiring various types of measurement data measured by each sensor 2 (2e to 2f), the subject evaluation system 100 also acquires characteristic information indicating the characteristics of each sensor (e.g., sensor ID, measurement data ID, measurement data characteristics, measurement date and time, measurement location, etc.). For example, the subject evaluation system 100 may set acquisition conditions, such as the date and time of transmission to the subject evaluation system 100 and the measurement data to be transmitted, in advance for each sensor 2 (2e to 2f), and each sensor 2 may transmit the data to the subject evaluation system 100 based on the set acquisition conditions.

[0046] The subject evaluation system 100 acquires subject information indicating at least one of the physical condition and behavior of the subject 3 via the sensor 2, and characteristic information indicating the characteristics of the sensor 2, and converts the acquired subject information into an evaluation target image on a two-dimensional plane based on the characteristic information of the sensor.

[0047] The subject evaluation system 100 generates an evaluation result for the evaluation target image by referring to a reference database that stores, for example, correlations between past evaluation target images that have been converted into images in advance and reference information linked to the past evaluation target images. This makes it possible to evaluate the condition of the subject 3 using the current evaluation target image of the subject 3. For example, multiple evaluation target images may be combined for evaluation. It is possible to check various conditions of the subject 3 or predict future occurrences depending on, for example, a combination of characteristic evaluation target images from among the multiple evaluation target images.

[0048] After acquiring evaluation target information including measurement data, the subject evaluation system 100 refers to a reference database (described later) and generates an evaluation result for the evaluation target image. The subject evaluation system 100 acquires instruction information corresponding to the generated evaluation result from a database (instruction information database) and outputs it to a display unit 109 or the like.

[0049] The evaluation results indicate not only the current confirmed condition of the subject 3 as the evaluation target, but also predictions of unconfirmed conditions that may occur or develop in the future. The evaluation results indicate evaluation results regarding the physical or behavioral condition of the subject 3, such as "normal," "abnormal," or "requires observation," but may also indicate tendencies of possible cases, conditions, or required care for the subject 3, such as "tendency toward XX symptom" or "60% possibility of XX symptom," or indicate probabilities of possible future illnesses, conditions, or required care. This allows, for example, the caregiver of the subject 3 to check these evaluation results and make preparations in advance for future care and nursing for the subject 3.

[0050] The devices constituting the subject evaluation system 100 may be electronic devices such as personal computers (PCs), as well as electronic devices such as smartphones, tablet devices, wearable devices, and IoT (Internet of Things) devices, and single-board computers such as Raspberry Pi (registered trademark), and may have a built-in sensor 2. For example, if a head-mounted display (HMD) with a built-in sensor 2 is used as the subject evaluation system 100 (subject evaluation device 1), the evaluator can recognize the evaluation results of the subject 3 by visually checking the subject 3 through the display. This reduces the difficulty of the physical evaluation task for the subject 3 and shortens the evaluation task time.

[0051] Here, an example of sensor data of the subject evaluation system 100 in this embodiment will be described with reference to Fig. 3. Fig. 3(a) is an example of sensor data of the subject evaluation system 100 in this embodiment, and various types of sensor data measured by multiple sensors 2 are acquired by the subject evaluation system 100.

[0052] As the sensor data measured by the various sensors 2, for example, the sensor 2a measures the state of the subject 3 (such as behavioral information while sleeping) and records it as sensor data 200a. As the sensor data 200a, for example, coordinates of positions and movements of the subject 3's head position (XX.XX), right shoulder position (XX.XX), left shoulder position (XX.XX), right upper arm position (XX.XX), left upper arm position (XX.XX), and numerical values ​​indicating actions are measured in chronological order, and the measured multiple sensor data 200a are recorded.

[0053] The sensor 2a is, for example, a known motion sensor, which measures and digitizes the movements of the subject 3 and the supporter 7. The sensor data 200a may be measured using, for example, a known motion sensor, a near-infrared sensor, an image sensor, or the like (not shown), and is acquired using known measurement techniques.

[0054] The sensor data 200a measured by the sensor 2a is acquired, for example, by the subject evaluation system 100, and multiple pieces of sensor data 200a are recorded within the subject evaluation system 100, or may be recorded, for example, on another terminal 5 or server 6.

[0055] Furthermore, for example, the sensor 2b measures the state of the subject 3 (for example, vital signs while sleeping) and the behavior of the supporter 7 (for example, gestures in response to instructions from instruction information), and records them as sensor data 201a. For example, when measuring the state of the subject 3, numerical values ​​indicating each vital sign of the subject 3, such as body temperature (35.50), respiratory rate (18.00), high blood pressure value (130.00), low blood pressure value (85.00), and pulse rate (70.00), are measured in chronological order as the sensor data 200b, and the measured multiple sensor data 200b are recorded.

[0056] The sensor 2b measures and digitizes vital information of the subject 3 using, for example, a known near-infrared or non-contact vital sensor. The sensor data 201a may be measured using, for example, a known vital sensor or various image sensors (not shown), and is acquired using known measurement techniques.

[0057] The sensor data 201a measured by the sensor 2b is acquired, for example, by the subject evaluation system 100, and multiple pieces of sensor data 201a are recorded within the subject evaluation system 100, or may be recorded, for example, on another terminal 5 or server 6.

[0058] In addition to the sensor data 200a from sensor 2a and the sensor data 201a from sensor 2b, sensor data is also measured from other sensors 2c to 2f in the same manner, and multiple pieces of sensor data are recorded within the subject evaluation system 100 as individual sensor data, and are also recorded, for example, on another terminal 5 or server 6.

[0059] Also, Figure 3(b) is an example of an evaluation target image of the subject evaluation system 100 in this embodiment, in which various sensor data measured by the sensor 2 is converted into evaluation target images 200b to 202b on a two-dimensional plane based on the characteristic information of each sensor 2 (model, model, conversion parameters, performance / characteristics, visualization characteristics, format, etc.).

[0060] The evaluation target image 200b is, for example, the result of image conversion of the sensor data 200a into the evaluation target image 200b on a two-dimensional plane based on the feature information of the sensor 2a. The image conversion is, for example, a known graphing process, and shows the state in which the sensor data measured in time series is graphed based on certain parameters.

[0061] The evaluation target image 200b is an image converted from the numerical values ​​measured by the sensor 2a into an evaluation target image 200b on a two-dimensional plane as a radar chart, but the image may be converted into a graph other than a radar chart if other characteristics are to be expressed.

[0062] Similarly, the evaluation target image 201b is, for example, sensor data 200b converted into an evaluation target image 201b on a two-dimensional plane based on the characteristic information of sensor 2b, and is, for example, an image converted from the changes in each vital information over time into an evaluation target image 201b on a two-dimensional plane.

[0063] Furthermore, the evaluation target image 202b may be, for example, sensor data measured by another sensor 2 that has been image-converted into the evaluation target image 202b on a two-dimensional plane based on feature information of the sensor 2. Alternatively, the respective values ​​measured by a plurality of sensors 2 may be compiled, and their ratios or the like may be image-converted into the evaluation target image 201b on a two-dimensional plane.

[0064] There may be multiple images to be evaluated for each sensor 2a to 2f shown in Figure 3(b), and if measurements are taken in time series, the time series progression or the total values ​​over a certain period may be converted into an image as a percentage.

[0065] The subject evaluation system 100 displays the evaluation target images 200b-202b and other image-converted evaluation target images on a two-dimensional plane on the display unit 109. For example, when multiple sensor data are measured by one sensor 2, the subject evaluation system 100 can generate separate evaluation results for the subject 3, and can display the evaluation results for the subject 3 on the display unit 109. Note that, for example, when generating an evaluation result for one subject 3, it may be based on multiple sensor data. Furthermore, for example, the type and number of multiple evaluation target images to be combined and displayed are arbitrary.

[0066] The sensor data may be generated using, for example, an RGB camera. The sensor data may be generated using, for example, a multispectral camera with an arbitrary wavelength selected, or may be generated based on imaging through a polarizing filter. The sensor data may be extracted from, for example, a portion of a video and converted into an image to be evaluated.

[0067] The subject information may be directly input into the subject evaluation system 100 by an evaluator or the like so as to be linked to the sensor data acquired by the subject evaluation system 100. Alternatively, for example, a plurality of pieces of subject information may be stored in the subject evaluation system 100 in advance, and the subject evaluation system 100 may select the subject information based on the image to be evaluated. When the subject evaluation system 100 selects the subject information, for example, a learning model previously stored in the subject evaluation system 100 may be used to select the subject and sensor 2 for the acquired image to be evaluated. In this case, the learning model is generated by known machine learning using previously prepared images to be evaluated and subject information as training data.

[0068] The subject information includes information regarding at least one of the date and time, posture, and movement of the subject 3 measured as sensor data when the subject 3 was observed, the observer, and the planned date and time when the subject 3 is to be observed. The subject information may also include information regarding the subject 3's daily life, such as physical strength, training, and diet.

[0069] The subject information may be directly input into the subject evaluation system 100 by an evaluator or the like so that it is linked to the sensor data acquired by the subject evaluation system 100, or may be transmitted from another terminal 5, for example.

[0070] The subject evaluation system 100 (subject evaluation device 1) acquires instruction information for the supporter 7 assisting the subject 3, for example, according to the generated evaluation result (prediction of the subject 3's current state, future condition, onset, or likelihood of change, etc.). The instruction information may be, for example, projection destination information for the room onto which the instruction information is projected, information instructing the supporter 7 to take an appropriate action while assisting the subject 3, or suggestion information suggesting an action to the supporter 7.

[0071] The instruction information may include, for example, various projection destination information regarding the ceiling or wall of the room onto which the instruction information is projected, the projection device, etc., various supporter information regarding the attributes, skills, etc. of the supporter 7 who is supporting the subject 3, and further suggestive information regarding information that motivates or encourages the supporter 7 to take action. This allows the subject evaluation system 100 to, for example, provide appropriate displays and instructions to the room of the subject 3 and the supporter 7, and to provide instructions appropriate to the projection destination and supporter in various locations depending on the needs of the site.

[0072] (Subject Evaluation Device 1) Next, an example of the subject evaluation device 1 in this embodiment will be described with reference to Fig. 4. Fig. 4(a) is a schematic diagram showing an example of the configuration of the subject evaluation device 1 in this embodiment, and Fig. 4(b) is a schematic diagram showing an example of the function of the subject evaluation device 1 in this embodiment.

[0073] 4(a), the subject evaluation device 1 includes a housing 10, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, and I / Fs 105 to 107. The components 101 to 107 are connected via an internal bus 110.

[0074] The CPU 101 controls the entire subject evaluation device 1. The ROM 102 stores the operation code of the CPU 101. The RAM 103 is a working area used when the CPU 101 is operating. The storage unit 104 stores various information such as evaluation target images and reference databases. As the storage unit 104, for example, a data storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) is used. Note that the subject evaluation device 1 may also have a GPU (Graphics Processing Unit) (not shown). Having a GPU enables faster calculation processing than usual.

[0075] I / F105 is an interface for sending and receiving various information with sensor 2, and may also be an interface for sending and receiving various information with other terminals 5, servers 6, etc. via a communication network 4 such as the Internet.

[0076] The I / F 106 is an interface for transmitting and receiving information to and from the input unit 108. For example, a keyboard is used as the input unit 108, and an evaluator or the like using the subject evaluation device 1 inputs various pieces of information or control commands for the subject evaluation device 1 via the input unit 108.

[0077] The I / F 107 is an interface for transmitting and receiving various types of information to and from the display unit 109. The display unit 109 outputs various types of information such as the evaluation results stored in the storage unit 104, or the processing status of the subject evaluation device 1. A display is used as the display unit 109, and may be, for example, a touch panel type.

[0078] <Reference database> The reference database stored in the storage unit 104 stores associations between previously acquired past evaluation target images and reference information linked to the past evaluation target images, and stores, for example, a learning model having the associations. The reference database may store, for example, past evaluation target images and reference information. The associations are constructed by machine learning using multiple learning data, with the past evaluation target images and the reference information serving as a set of learning data. For example, deep learning such as a convolutional neural network is used as a learning method.

[0079] In this case, for example, the correlation indicates the degree of connection between many-to-many information (multiple data included in past evaluation target images versus multiple data included in reference information). The correlation is updated as appropriate during the machine learning process. That is, the correlation indicates a function optimized based on, for example, past evaluation target images (image data on a two-dimensional plane) and reference information. Therefore, the evaluation result for the evaluation target image is generated using the correlation constructed based on all past evaluation results of the subject 3's condition. This makes it possible to generate optimal evaluation results even when the subject 3's physical condition and behavior are in various states.

[0080] Furthermore, it is possible to quantitatively generate optimal evaluation results not only when the image to be evaluated is identical to or similar to previous evaluation target images, but also when it is dissimilar. Note that by improving the generalization ability when performing machine learning, it is possible to improve the evaluation accuracy for unknown evaluation target images.

[0081] The correlation may include multiple correlation degrees indicating the degree of connection between multiple data included in the past evaluation target image and multiple data included in the reference information. For example, when the learning model is constructed using a neural network, the correlation degrees can be associated with weight variables.

[0082] The past evaluation target images indicate the same type of information as the evaluation target images described above. The past evaluation target images include, for example, a plurality of evaluation target images acquired when the subject 3 was evaluated in the past.

[0083] The reference information is linked to a past image to be evaluated and indicates information relating to the condition of the subject 3. The reference information indicates an evaluation based on the condition of the subject 3 (for example, "normal," "abnormal," "requires observation," "tendency toward XX symptom," "60% possibility of XX symptom," etc.), and may also include physical information, response information, preparation information, prediction information, etc. relating to the causes of the condition of the subject 3.

[0084] The reference information may indicate, for example, the tendency of possible cases or conditions, necessary care, etc., that may occur in the subject 3, or the probability of possible future illnesses or conditions, necessary care, etc. Note that the specific content included in the reference information can be set arbitrarily.

[0085] In the case of an elderly person, the physical information indicates the names of specific factors of the condition, such as dementia, dehydration, walking disorder, mental and psychological disorder, mobility disorder, excretory dysfunction, sensory disorder, nutritional intake disorder, etc. Various factors are generally at least partially related to the physical condition and behavior of the subject 3.

[0086] The correlation may indicate the degree of connection between a past evaluation target image and reference information, for example, as shown in Fig. 5. In this case, by using the correlation, the degree of relationship between each of a plurality of data included in the reference information ("Reference A" to "Reference C" in Fig. 5) and each of a plurality of data included in the past evaluation target image ("Image Data A" to "Image Data C" in Fig. 5) can be linked and stored. Therefore, for example, through the correlation, a plurality of data included in the reference information can be linked to one piece of data included in the past evaluation target image, thereby realizing the generation of a multifaceted evaluation result.

[0087] Furthermore, the correlation has, for example, multiple correlation degrees linking multiple data included in the past evaluation target image with multiple data included in the reference information. The correlation degree is expressed in three or more levels, such as a percentage, a 10-point scale, or a 5-point scale, and is expressed, for example, by line characteristics (e.g., thickness, etc.). For example, "image data A" included in the past evaluation target image shows a correlation degree AA of "85%" with "reference A" included in the reference information, and a correlation degree AB of "55%" with "reference B" also included in the reference information. In other words, the "correlation degree" indicates the degree of connection between each image data, and, for example, the higher the correlation degree, the stronger the connection between each data. Note that when constructing the correlation using the above-mentioned machine learning, the correlation may be set to have three or more levels of correlation degree.

[0088] For example, past evaluation target images may be stored in the reference database after being divided into past image data A to C and past physical condition information or behavior information. In this case, the correlation is calculated based on the relationship between the combination of past image data and past physical condition information or behavior information and the reference information. In addition to the above, past evaluation target images may also be stored in the reference database after being divided into past observation information (various sensor data, etc.).

[0089] The past evaluation target image may also include, for example, composite data and a similarity. The composite data is represented by three or more levels of similarity between the past image data, or past physical condition information, or behavior information. The composite data may be stored in the reference database in the form of a numerical value, a matrix, a histogram, or the like, or may be stored in the form of an image, a character string, or the like.

[0090] Fig. 4(b) is a schematic diagram showing an example of the functions of the subject evaluation device 1. The subject evaluation device 1 includes an acquisition unit 11, a conversion unit 12, an evaluation unit 13, an instruction information acquisition unit 14, an output unit 15, and a storage unit 16 (database), and may also include, for example, an update unit 17 and a behavior evaluation unit 18. Note that each function shown in Fig. 4(b) is realized by the CPU 101 using the RAM 103 as a work area to execute a program stored in the storage unit 104 or the like, and may be controlled by, for example, artificial intelligence.

[0091] <Instruction Information Database> The instruction information database (not shown) stored in the storage unit 104 stores, for example, past evaluation results and instruction information that instructs the supporter 7 on how to act in accordance with the evaluation results. The past evaluation results and instruction information may each be assigned a unique ID, for example, and a ratio or the like indicating their correlation may be assigned. When the subject evaluation system 100 refers to the instruction information database and acquires instruction information corresponding to the generated evaluation results, it may acquire an appropriate instruction information ID based on, for example, the ratio or the like assigned to the evaluation result ID or the like.

[0092] The instruction information may be linked to, for example, projection destination information including various information about the projection destination to which the instruction information is output (projected), and stored in the instruction information database. The projection destination information may also include information indicating, for example, the material and condition of the walls and ceiling of Room A or Room B, the brightness of the room, the position of windows, the presence and material of curtains or a projection screen, etc., and each may be identified by a unique ID (projection destination information ID) and stored in the instruction information database. The projection destination information ID may be linked to, for example, the instruction information ID, and may be assigned a ratio or the like indicating its correlation with the evaluation result and stored.

[0093] The instruction information may also be stored in association with various types of information (supporter information) about the supporter 7 who provides support to the subject 3. The supporter information may include information such as the attributes of the supporter 7 (e.g., family member, non-family member, etc.), experience and skills (e.g., whether or not there is experience in support or caregiving, number of years, etc.), and the status of the work (e.g., currently working, scheduled work, etc.), and each of these may be identified by a unique ID (supporter information ID) and stored in the instruction information database. The various supporter information IDs may be associated with the instruction information ID, and may be assigned a ratio or the like indicating their relevance to the evaluation result and stored.

[0094] The instruction information may be linked to various suggestion information that suggests actions to be taken by the supporter 7 to support the subject person 3, and may be stored in the instruction information database. The suggestion information is, for example, information that suggests actions to be taken by the supporter 7, and may be, for example, a message expressed in a simple, short sentence. The suggestion information may be linked to information such as an instruction information ID, a projection destination information ID, and a supporter information ID, and stored.

[0095] Furthermore, the suggestion information may be, for example, text or character information indicating a suggestion, or image information indicating an icon, illustration, motif, or the like, audio data such as a voice, alarm, or melody, or a unique vibration pattern. Each type of suggestion information may be assigned a unique ID, linked to, for example, an instruction information ID, and may be assigned a ratio or the like indicating its relevance to the evaluation result and stored.

[0096] When the evaluation result (evaluation result ID) generated by the evaluation unit 13 is, for example, "supine posture: ○ time has passed," the instruction information acquisition unit 14 refers to the instruction information database and acquires an instruction information ID linked to the evaluation result ID. The instruction information acquisition unit 14 acquires, for example, instruction information (for example, "change position") stored in the record of the acquired instruction information ID. The output unit 15 outputs, for example, the acquired instruction information ("change position") within the room or to the supporter 7.

[0097] Furthermore, for example, when the evaluation result generated by the evaluation unit 13 is "blood and lymph flow: decreased," the instruction information acquisition unit 14 refers to the instruction information database and acquires an instruction information ID linked to the evaluation result ID. The instruction information acquisition unit 14 acquires, for example, instruction information (for example, "perform a massage") stored in the record of the acquired instruction information ID. The output unit 15 outputs, for example, the acquired instruction information (for example, "perform a massage") within the room or to the support person 7.

[0098] The instruction information database may store, for example, multiple instruction information IDs. The instruction information ID may be, for example, a specific numerical value, a certain numerical range, an image pattern of an evaluation target image, or a combination of multiple numerical values, numerical ranges, and image patterns, and may be configured to store these individually or in pairs. The instruction information database may have a configuration similar to that of a reference database, and may be constructed by machine learning using, for example, correlations between previously generated evaluation target images and instruction information linked to past evaluation target images as learning data.

[0099] <Acquisition part 11> The acquisition unit 11 acquires subject information indicating at least one of the physical condition and behavior of the subject 3 and feature information indicating features of the sensor 2 via one or more sensors that measure the condition of the subject 3. The acquisition unit 11 acquires sensor data indicating the subject information of the subject 3 from the sensor 2, etc., and may also acquire image data of the subject 3, spatial image data of a location, etc. from the sensor 2, if a camera is built in, for example. The acquisition unit 11 acquires physical condition information and behavior information of the subject 3 input in advance by an evaluator, etc., and also acquires feature information identifying the sensor 2, for example, from the sensor 2, etc.

[0100] For example, when acquiring sensor data, the acquiring unit 11 acquires the sensor data together with characteristic information of the sensor 2. Note that the frequency and cycle at which the acquiring unit 11 acquires the subject information and characteristic information are arbitrary.

[0101] The acquisition unit 11 receives various information including the sensor data and characteristic information of the evaluator measured by the sensor 2. The acquisition unit 11 may also receive various information such as physical condition information, behavioral information, observation information, and location-related environmental information of the subject 3 transmitted from an external terminal such as another terminal 5 via, for example, the communication network 4 and the I / F 105.

[0102] The acquisition unit 11 may refer to the learning model stored in the storage unit 104, for example, and select the sensor 2, the sensor data, and the physical condition information and behavior information corresponding to the feature information, and acquire them as an image to be evaluated.

[0103] <Conversion unit 12> The conversion unit 12 converts the subject information acquired by the acquisition unit 11 into an evaluation target image on a two-dimensional plane based on the characteristic information of the sensor 2. As shown in Fig. 3, the conversion unit 12 converts, for example, sensor data 200a measured by various sensors 2a into an evaluation target image 200b. The conversion unit 12 converts, for example, into a graph (for example, a radar chart) as shown in Fig. 3(b) based on various information (for example, model, conversion parameters, performance / characteristics, visualization characteristics, format, etc.) that is sensor characteristic information of the sensor 2a or characteristic information of the sensor data.

[0104] The conversion unit 12 performs image conversion by graphing based on the numerical values ​​of each of the items, such as the head position (XX.XX), right shoulder position (XX.XX), left shoulder position (XX.XX), right upper arm position (XX.XX), and left upper arm position (XX.XX), which are items indicating the state of the subject 3 acquired by the sensor 2a. The conversion unit 12 may, for example, refer to a correspondence table (not shown) for graphing, graph the sensor data 200a measured by the sensor 2a, and perform image conversion.

[0105] For example, if the measured sensor data 200a has partial features, the conversion unit 12 may graph only the range and numerical values ​​of the partial features. For example, based on an instruction from another terminal 5, the conversion unit 12 may graph the sensor data of the target sensor 2 and convert the image. The conversion unit 12 may perform similar image conversion for, for example, sensor 2b and other sensors 2. For example, the conversion unit 12 may combine numerical data measured by multiple sensors 2 to generate a new graph, and use the combined graph as the evaluation target image.

[0106] Furthermore, the conversion unit 12 may, for example, compile the numerical values ​​measured by the multiple sensors 2, graph their ratios and trends, and perform image conversion. For example, if the sensor data is measured in chronological order and there is a significant increase or decrease trend over a certain period, the conversion unit 12 may graph the trend and store the period when the graph was created, conditions and parameters such as numerical values, and various setting information when the graph was created. This makes it possible to check the image to be evaluated by going back to the graphed period, and image conversion processing suitable for multiple types of sensors can be performed in various locations depending on the needs of the site.

[0107] <Evaluation Section 13> The evaluation unit 13 refers to the reference database and generates an evaluation result for the image to be evaluated. For example, the evaluation unit 13 uses the image to be evaluated as input data, selects optimal reference information associated with the solution calculated based on the correlation, and generates an evaluation result based on the optimal reference information.

[0108] For example, when referring to the reference database shown in FIG. 5, the evaluation unit 13 selects data that is identical to or similar to the data included in the image to be evaluated (for example, "image data A": first data). As the first data, image data that partially or completely matches the image to be evaluated may be selected, or similar image data may be selected. When the image to be evaluated is expressed as numerical values ​​such as a matrix, the range of numerical values ​​included in the selected first data may be set in advance.

[0109] The evaluation unit 13 selects reference information linked to the selected first data and a degree of association (first degree of association) between the selected first data and the reference information, and generates an evaluation result based on the selected reference information and the first degree of association. Note that the first degree of association may be selected from associations established in advance, or may be calculated by the evaluation unit 13.

[0110] For example, the evaluation unit 13 selects data "Reference A" included in reference information linked to the first data "Image data A" and a first correlation (correlation AA) of "85%" between "Image data A" and "Reference A." The reference information and the first correlation may include a plurality of data. In this case, in addition to the above-mentioned "Reference A" and "85%," the evaluation unit 13 may select reference information "Reference B" linked to the first data "Image data A" and a first correlation (correlation AB) of "55%" between "Image data A" and "Reference B," and generate an evaluation result based on "Reference A" and "85%" as well as "Reference B" and "55%."

[0111] The evaluation result may include the image to be evaluated. The evaluation result may indicate factors of the state (physical condition or behavior) of the subject 3 expressed as a probability using, for example, reference information and a degree of association.

[0112] The evaluation unit 13 generates an evaluation result indicating the selected reference information, the first correlation degree, etc. in a format (e.g., character strings) that can be understood by an evaluator, etc., by using format data such as an output format stored in advance in the storage unit 104, etc. Note that the format setting when generating the evaluation result may be performed using, for example, known technology.

[0113] The evaluation unit 13 determines the content of the evaluation result based on, for example, the selected first correlation. For example, the evaluation unit 13 may be configured to generate the evaluation result based on reference information associated with a first correlation of "50%" or more, and not reflect reference information associated with a first correlation of less than "50%" in the evaluation result. Note that the determination criteria based on the first correlation may be, for example, a threshold value set in advance by an evaluator, and the range of the threshold value can be set arbitrarily. Furthermore, the evaluation unit 13 may determine the content of the evaluation result based on, for example, the result of calculating two or more first correlations or a comparison of two or more first correlations.

[0114] The evaluation unit 13 generates an evaluation result such as "supine posture: ○ time elapsed" in a format understandable to the evaluator (e.g., a character string) using format data such as an output format previously stored in the storage unit 104. The evaluation result is collected in the conversion unit 12 by collecting the respective numerical values ​​measured by the various sensors 2 as shown in FIG. 3(a), and is converted into an image such as a graph or figure as shown in FIG. 3(b), thereby generating an evaluation target image. The evaluation unit 13 references a reference database and generates the evaluation result based on the evaluation target image generated in the conversion unit 12.

[0115] The evaluation unit 13 refers to the reference database as described above, and generates an evaluation result, for example, an evaluation of the subject 3 as "blood and lymph flow: decreased" from, for example, a generated evaluation target image based on the correlation between a previous evaluation target image that has been converted in advance and reference information linked to the previous evaluation target image. Also, the evaluation unit 13 generates an evaluation result, for example, an evaluation of the subject 3 as "supine posture: ○ time elapsed" from, for example, another evaluation target image.

[0116] <Instruction information acquisition unit 14> The instruction information acquisition unit 14 refers to, for example, an instruction information database and acquires instruction information according to the evaluation result generated by the evaluation unit 13. If the evaluation result for the evaluation target image generated by the evaluation unit 13 is, for example, "supine posture: ○ time elapsed," the instruction information acquisition unit 14 refers to the instruction information database and acquires instruction information linked to the evaluation result.

[0117] The instruction information acquisition unit 14 refers to the instruction information acquisition database, acquires one or more instruction information IDs linked to the evaluation result (evaluation result ID) generated by the evaluation unit 13, and acquires the instruction information recorded in the record of the acquired instruction information ID. For example, when the evaluation result (evaluation result ID) is "supine posture: ○ time elapsed (A001)", the instruction information acquisition unit 14 refers to the instruction information database, and acquires the instruction information ID (B001) linked to the evaluation result ID (A001). For example, the instruction information acquisition unit 14 acquires the instruction information (for example, "change posture") stored in the record of the acquired instruction information ID (B001).

[0118] For example, if the evaluation result (evaluation result ID) is "Blood and lymph flow: decreased (A010)," the instruction information acquisition unit 14 refers to the instruction information database and acquires the instruction information ID (B010) linked to the evaluation result ID (A010). The instruction information acquisition unit 14 acquires, for example, instruction information (e.g., "perform massage") stored in the record of the acquired instruction information ID (B010).

[0119] <Output section 15> The output unit 15 outputs instruction information. The output unit 15 transmits the evaluation result to the display unit 109 via the I / F 107, and also transmits the evaluation result to another terminal 5 or the like via, for example, the I / F 105. The output unit 15 outputs, for example, the evaluation target image of the subject 3 shown in FIG. 3 , and data indicating the optimal evaluation result, recommendation, etc. corresponding to the physical condition and behavior of the subject 3 to the display unit 109 or the like.

[0120] The output unit may display, for example, the results of an evaluation based on the condition of the subject 3 (for example, "normal," "abnormal," "requires observation," "tendency toward XX symptom," "60% possibility of XX symptom," etc.), as well as physical information, response information, preparation information, prediction information, etc. regarding factors in the current or future condition of the subject 3.

[0121] The output unit 15 outputs the acquired instruction information. The output unit 15 may output the instruction information to a projection device such as a monitor, projector, or laser irradiator provided in the room, or to various known devices (such as a smartphone, tablet, head-mounted display, or smart glasses) provided by the supporter 7.

[0122] The subject evaluation system 100 may acquire information about the state of the room, the projection destination, information about the supporter 7 assisting the subject 3, and information about the posture of the supporter 7, for example, via sensors 2 (2a to 2f) installed in a room (such as location A or location B). The subject evaluation system 100 may, for example, determine the state of the room and information about the posture of the supporter 7, and adjust, for example, correct the instruction information acquired by the instruction information acquisition unit 14 to make it appropriate for the state of the room and the posture of the supporter 7, and the output unit 15 may output the adjusted instruction information.

[0123] <Storage section 16> The storage unit 16 retrieves, as necessary, various pieces of information such as the reference database stored in the storage unit 104. The storage unit 16 stores in the storage unit 104 various pieces of information acquired or generated by the components 11, 13 to 15.

[0124] <Updated part 17> The update unit 17 updates, for example, the reference database. For example, when a relationship between a past evaluation target image and reference information is newly acquired, the update unit 17 reflects the relationship in the correlation. For example, when the subject evaluation system 100 acquires a judgment result in which an evaluator or the like judges the accuracy of the evaluation result based on the evaluation result generated by the evaluation unit 13, the update unit 17 updates the correlation stored in the reference database based on the judgment result.

[0125] <Behavioral Evaluation Unit 18> The behavior evaluation unit 18 evaluates the behavior of the support person 7. The behavior evaluation unit 18 acquires behavior information regarding the behavior of the support person 7 via sensors 2 (2a to 2f) installed in a room (such as location A or location B), for example. The behavior evaluation unit 18 evaluates whether or not various types of checking behaviors, such as the work target, work procedure, and gaze point, have been performed as the behavior of the support person 7 resulting from the instruction information output by the output unit 15, for example.

[0126] The behavior evaluation unit 18 acquires, via the sensor 2, the movement of a specific behavior (such as a gesture, pose, posture, behavior, or calling) performed by the supporter 7 as a confirmation behavior of the supporter 7, and evaluates the behavior of the supporter 7 based on the acquired sensor data. The behavior of the supporter 7 evaluated by the behavior evaluation unit 18 may be related to the execution of various instruction information given to the subject 3, such as, for example, "understanding the instruction information," "completion of the instruction information," "redisplay of the instruction information," or "inability to execute the instruction information."

[0127] <Display section 109> The display unit 109 displays the evaluation results. For example, as shown in Fig. 3, the display unit 109 displays evaluation target images 200b to 202b and the evaluation results. The evaluation target images 200b to 202b are evaluation target images obtained by converting the subject information acquired by the acquisition unit 11 via the sensor 2 and feature information indicating the features of the sensor 2 into evaluation target images on a two-dimensional plane based on the feature information of the sensor 2, and the evaluation results.

[0128] The display unit 109 may display the evaluation results using, for example, only a list or character strings. Publicly known techniques can be used for the display method. For example, when an HMD is used as the subject evaluation device 1, a transmissive display is used as the display unit 109. In this case, the display unit 109 can display the evaluation target image and the evaluation results to the subject 3, which is viewed by, for example, an evaluator through the display unit 109.

[0129] <Sensor 2> The sensor 2 is a variety of known sensors that measure the physical condition and behavioral state of the subject 3 and generate sensor data. As the sensor 2, various sensors such as a motion sensor, a near-infrared camera, an RGB camera, an ultrasonic sensor, and a distinguishing part sensor may be used, and multiple sensors may be used simultaneously or in conjunction with each other in different locations. The sensor 2 may be built into the subject evaluation device 1, for example, or may be held or worn by the subject 3.

[0130] <Communication Network 4> The communication network 4 is, for example, an internet network to which the subject evaluation system 100 (subject evaluation device 1), multiple sensors 2, etc. are connected via communication circuits. The communication network 4 may be configured as a so-called optical fiber communication network. Furthermore, the communication network 4 may be realized by a known communication network such as a wired communication network or a wireless communication network.

[0131] <Other terminal 5> The other terminal 5 may be, for example, an electronic device embodied in the same way as the subject evaluation device 1. The other terminal 5 may, for example, be a central control device or the like that is capable of communicating with multiple subject evaluation devices 1. The other terminal 5 may, for example, be connectable to multiple subject evaluation systems 100 (subject evaluation devices 1), and may acquire evaluation results generated by the subject evaluation systems 100 and each subject evaluation device 1. This may enable, for example, analysis of the evaluation results of the subject 3 measured at multiple locations, making it possible to improve the physical condition of the subject 3, for example.

[0132] <Server 6> The server 6 stores, for example, the various types of information described above. The server 6 accumulates, for example, various types of sensor data and various types of information related to the multiple sensors 2 transmitted via the communication network 4. The server 6 may store, for example, information similar to that stored in the storage unit 104, and may transmit and receive, via the communication network 4, various types of sensor data, various types of information related to the multiple sensors 2, image data, evaluation results, and the like, to and from one or more subject evaluation systems 100 (subject evaluation devices 1). That is, the subject evaluation device 1 may use the server 6 instead of the storage unit 104.

[0133] (An example of the operation of the subject evaluation system 100) Next, an example of the operation of the subject evaluation system 100 in this embodiment will be described. FIG. 6 is a flowchart showing an example of the operation of the subject evaluation system 100 in this embodiment. It is.

[0134] <Acquisition means S110> 6, a subject image and feature information are acquired (acquisition means S110). The acquisition unit 11 acquires, for example, subject information indicating at least one of the physical condition and behavior of the subject 3 via one or more sensors 2 that measure the state of the subject 3, and feature information indicating the features of the sensors 2. The acquisition unit 11 stores the evaluation target information and feature information in the storage unit 104 via, for example, the memory unit 16.

[0135] The acquisition unit 11 acquires, for example, subject information indicating at least one of the physical condition and behavior of the subject 3, and feature information indicating features of the sensor 2. The acquisition unit 11 may be, for example, a plurality of sensors 2. The subject information and feature information are input to the subject evaluation system 100 by an evaluator or the like so as to be linked to image data or the like, or, for example, the acquisition unit 11 may select each piece of information suitable for the image data based on the image data. In this case, the acquisition unit 11 selects each piece of information suitable for the image data from information (sensor data, etc.) obtained by observing a plurality of subjects 3 that has been stored in advance in the storage unit 104.

[0136] For example, when one subject 3 is imaged using multiple sensors 2, the acquisition unit 11 acquires multiple pieces of sensor data (subject information) and feature information measured by the multiple sensors 2 as source data for one evaluation target image to be converted. The acquisition unit 11 acquires sensor data (subject information and feature information) indicating the state of the subject 3 according to the physical condition and behavioral patterns of the evaluator, and may also acquire sensor data according to, for example, the lifestyle and behavioral patterns of the subject 3.

[0137] The acquisition unit 11 may convert the acquired sensor data of the sensor 2 measured over an arbitrary period into an evaluation target image on a two-dimensional plane so as to receive the sensor data at once. Note that, for example, if the subject evaluation device 1 has already acquired the feature information of each sensor 2 in advance and there has been no change, only the sensor data of the subject information may be acquired from the sensor 2. In this case, the subject evaluation system 100 may distinguish the sensor data acquired from the sensor 2 and separately acquire the feature information of the sensor 2 linked to the sensor data.

[0138] <Conversion means S120> Next, the conversion unit 12 performs image conversion into an evaluation target image on a two-dimensional plane (conversion means S120). The conversion unit 12 performs image conversion, for example, on subject information (sensor data) of the subject 3 acquired by the acquisition unit 11 via the sensor 2, into an evaluation target image visualized as a graph on a two-dimensional plane based on feature information of the corresponding sensor 2. The conversion unit 12 performs image conversion, for example, on the subject information acquired by the acquisition unit 11, into an evaluation target image on a two-dimensional plane, such as a graph, based on feature information of a plurality of sensors 2.

[0139] The conversion unit 12 converts the image into a graph as a target image on a two-dimensional plane, but may also convert the image into an image other than a graph, for example. The evaluation target image converted by the conversion unit 12 may be an image that represents the state of the subject 3 in a two-dimensional plane, and the type, scale, number of graphs, expression, etc. of the graph may be arbitrary.

[0140] <Evaluation Means S130> Next, the evaluation unit 13 refers to the reference database and converts the image into an evaluation target image on a two-dimensional plane (evaluation means S130). The evaluation unit 13 acquires the evaluation target image acquired by the acquisition unit 11, and acquires the evaluation target image from a reference database stored in, for example, the storage unit 104. The evaluation unit 13 uses, for example, the evaluation target image as input data, selects optimal reference information associated with a solution calculated based on a correlation indicated by a function, etc., and generates an evaluation result based on the optimal reference information. At this time, for example, the evaluation unit 13 may select multiple reference information for one evaluation target image.

[0141] The evaluation unit 13 generates one evaluation result for one evaluation target image, or may generate one evaluation result for multiple evaluation target images, for example. The evaluation unit 13 generates the evaluation result using format data such as an output format stored in the storage unit 104, for example. The evaluation unit 13 stores the evaluation result in the storage unit 104 via the memory unit 16, for example.

[0142] <Instruction information acquisition means S140> Next, the instruction information acquisition unit 14 acquires instruction information corresponding to the evaluation result (instruction information acquisition means S140). The instruction information acquisition means 14, for example, refers to an instruction information database and acquires the instruction information corresponding to the generated evaluation result.

[0143] For example, if the evaluation result (evaluation result ID) generated by the evaluation unit 13 is, for example, "supine posture: ○ time has elapsed," the instruction information acquisition unit 14 refers to the instruction information database, acquires the instruction information ID linked to the evaluation result ID, and acquires the instruction information (for example, "change position") stored in the record of the instruction information ID.

[0144] Furthermore, if the evaluation result generated by the evaluation unit 13 is, for example, "blood and lymph flow: decreased," the instruction information acquisition unit 14 refers to the instruction information database, acquires the instruction information ID linked to the evaluation result ID, and acquires the instruction information (for example, "perform a massage") stored in the record of the instruction information ID.

[0145] <Output Means S150> Next, the output unit 15 outputs the instruction information (output means S150). The output unit 15 outputs the evaluation result to the display unit 109 or the like. The output unit 15 may output the evaluation result to another terminal 5 or the server 6 via the communication network 4, for example.

[0146] The output unit 15 may output to the display unit 109, for example, a subject 3 based on image data, a sensor 2 capable of measuring subject information of the subject 3, sensor data measured by the sensor 2, the sensing status and results of the sensing by the sensor 2, a display image showing a plurality of evaluation target images, evaluation results of the subject 3 based on a combination of the evaluation target images, recommendation information based on the evaluation results, etc., a designation unit (not shown) that designates the subject 3 to be evaluated in the display image, and information for displaying the evaluation results for the subject 3 to be evaluated via the designation unit. As a result, the display unit 109 displays the display image, the designation unit, and the evaluation results.

[0147] <Behavioral evaluation means S160> Next, the behavior evaluation unit 18 evaluates the behavior caused by the instruction information (behavior evaluation means S160). The behavior evaluation unit 18 evaluates, for example, the behavior of the support person 7 after the instruction information is output. The behavior evaluation unit 18 acquires behavior information indicating whether or not the support person 7 has taken any action in response to the output instruction information, and evaluates, based on the acquired behavior information, whether or not the behavior caused by the instruction information has been performed reliably, appropriately, and without error.

[0148] Furthermore, the behavior evaluation unit 18 may acquire data indicating the positional relationship between the subject 3 and the supporter 7, for example, using sensors 2 in the room, or may acquire the data via devices held by the subject 3 or the supporter 7. The behavior evaluation unit 18 may evaluate the behavior of the supporter 7 based on, for example, the time when instruction information is output and the time when the supporter 7 starts to act in response to the output instruction information. Furthermore, when there are multiple supporters 7 in a room supporting the subject 3, for example, the behavior evaluation unit 18 may perform individual behavior evaluation for each supporter 7, for example, based on differences between each of the multiple supporters 7 and the instruction information for each supporter 7.

[0149] This completes the operation of the subject evaluation system 100 in this embodiment. The update unit 17 may perform the update at any timing.

[0150] According to this embodiment, the evaluation unit 13 references a reference database and generates an evaluation result for the image to be evaluated. The reference information includes various types of physical information and behavioral information of the subject 3. Therefore, it is possible to generate an evaluation result based on the results of past evaluations of the condition of the subject 3. This improves the accuracy of evaluating the condition of the subject 3, and makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0151] Furthermore, according to this embodiment, the image to be evaluated includes physical condition information. Therefore, it is possible to realize an evaluation that takes into account the characteristics of the factors that cause the condition, which differ depending on the sensor 2 and the physical condition of the subject 3. This further improves the accuracy of evaluating the condition of the subject 3, and makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0152] Furthermore, according to this embodiment, the image to be evaluated includes behavioral information. Therefore, it is possible to realize an evaluation that takes into account the surface posture of the subject 3, which varies depending on the conditions of the sensor 2 and the measurement of the subject 3. This further improves the accuracy of evaluating the state of the subject 3, and makes it possible to perform processing suitable for multiple types of sensors in various locations according to on-site needs.

[0153] Furthermore, according to this embodiment, evaluation results can be obtained that are linked to the type of sensor 2 of the subject 3, the specifications of the sensor 2, and the conditions of placement, making it possible to identify conditions that may cause physical abnormalities in the subject 3 and understand physical changes that accompany changes in observation conditions, etc. This can lead to physical improvements for the subject 3 and a reduction in the burden of care.

[0154] Furthermore, according to this embodiment, the image to be evaluated includes characteristic information. Therefore, evaluation can be performed based on the characteristics of measurement factors that differ depending on the type, characteristics, number, and placement of the sensors 2. This further improves the accuracy of evaluating the condition of the subject 3, and enables processing suitable for multiple types of sensors in various locations according to on-site needs.

[0155] Furthermore, according to this embodiment, the correlation is constructed by machine learning using past evaluation target images and reference information as learning data. Therefore, quantitative evaluation can be performed even when evaluating an unknown evaluation target image that is different from past evaluation target images. This makes it possible to further improve the evaluation accuracy.

[0156] Furthermore, according to this embodiment, when a relationship between a past evaluation target image and reference information is newly acquired, the update unit 17 reflects the relationship in the correlation. This makes it possible to easily update the correlation and continuously improve the evaluation accuracy.

[0157] Furthermore, according to this embodiment, the evaluation means S130 refers to a reference database and generates an evaluation result for the evaluation target image. The reference information includes information related to the body. Therefore, it is possible to generate an evaluation result based on the results of past evaluations of the condition of the subject 3. This makes it possible to improve the accuracy of evaluating the condition of the subject 3.

[0158] Furthermore, according to this embodiment, the subject evaluation program for evaluating the condition of the subject can cause a computer to execute an acquisition step of an acquisition means S110, a conversion step of a conversion means S120 for converting an image into an evaluation target image on a two-dimensional plane, an evaluation step of an evaluation means S130 for generating an evaluation result for the evaluation target image, an instruction information acquisition step of an instruction information acquisition means S140 for acquiring instruction information according to the evaluation result, an output step of an output means S150 for outputting the instruction information, and a behavior evaluation step of a behavior evaluation means S160 for evaluating behavior resulting from the instruction information.

[0159] Although the embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0160] 1: Subject evaluation device 2: Sensor 2a~2f: Sensor 3: Target audience 4: Communication network 5: Other devices 6: Server 7:Supporter 10: Housing 11: Acquisition part 12: Conversion section 13: Evaluation section 14: Instruction information acquisition section 15: Output section 16: Storage section 17: Update section 18: Behavioral Evaluation Department 100: Subject evaluation system 101: CPU 102:ROM 103:RAM 104: Preservation Department 105: Interface 106: Interface 107: Interface 108: Input section 109: Display section 110: Internal bus 200a: Center Data 201a: Sensor data 200b: Image to be evaluated 201b: Image to be evaluated 202b: Image to be evaluated A: Location B: Location S110: Acquisition means S120: Conversion method S130: Evaluation method S140: Instruction information acquisition means S150: Output means S160: Behavioral assessment instruments

Claims

1. A subject evaluation system for evaluating a subject's condition, comprising: one or more sensors for measuring a condition of the subject; an acquisition means for acquiring subject information indicating at least one of the subject's physical condition and behavior via the sensor and characteristic information indicating characteristics of the sensor; a conversion means for converting the acquired subject information into an evaluation target image on a two-dimensional plane based on the feature information of the sensor; a reference database storing associations between past evaluation target images that have been converted in advance and reference information associated with the past evaluation target images; evaluation means for referencing the reference database and generating an evaluation result for the evaluation target image; an instruction information database storing past evaluation results and instruction information for instructing the supporter on how to act in accordance with the evaluation results; an instruction information acquisition means for referencing the instruction information database and acquiring the instruction information corresponding to the generated evaluation result; an output means for outputting the instruction information; A subject evaluation system comprising:

2. The association is constructed by machine learning using the past evaluation target image and the reference information as learning data. The subject evaluation system according to claim 1,

3. The apparatus further comprises an update means for, when a relationship between the past evaluation target image and the reference information is newly acquired, reflecting the relationship in the association.

3. The subject evaluation system according to claim 1 or 2,

4. the instruction information includes projection destination information to which the instruction information is to be projected, the output means outputs the instruction information based on the projection destination information; The subject evaluation system according to claim 1,

5. The instruction information includes supporter information that provides support to the subject, the output means outputs the instruction information based on the supporter information; The subject evaluation system according to claim 1,

6. the instruction information includes suggestion information that suggests an action to the supporter, the output means outputs the instruction information based on the suggestion information; The subject evaluation system according to claim 1,

7. further comprising a behavior evaluation means for evaluating the behavior of the supporter; The behavior evaluation means acquiring behavior information indicating whether or not the supporter has taken an action in response to the output instruction information, and evaluating the action resulting from the instruction information based on the behavior information; The subject evaluation system according to claim 1,

8. A subject evaluation device for evaluating a subject's condition, one or more sensors for measuring a condition of the subject; an acquisition unit that acquires subject information indicating at least one of a physical condition and a behavior of the subject via the sensor and feature information indicating features of the sensor; a conversion unit that converts the acquired subject information into an evaluation target image on a two-dimensional plane based on the feature information of the sensor; a reference database storing associations between past evaluation target images that have been converted in advance and reference information associated with the past evaluation target images; an evaluation unit that references the reference database and generates an evaluation result for the evaluation target image; an instruction information database storing past evaluation results and instruction information for instructing the supporter on how to act in accordance with the evaluation results; an instruction information acquisition unit that refers to the instruction information database and acquires the instruction information corresponding to the generated evaluation result; an output unit that outputs the instruction information; A subject evaluation device comprising:

9. 1. A subject assessment program for assessing a subject's condition, comprising: measuring a condition of the subject with one or more sensors; an acquisition step of acquiring subject information indicating at least one of a physical condition and a behavior of the subject via the sensor and feature information indicating features of the sensor; a conversion step of converting the acquired subject information into an evaluation target image on a two-dimensional plane based on the feature information of the sensor; a first storage step of storing in a reference database a correlation between a past evaluation target image that has been converted in advance and reference information associated with the past evaluation target image; an evaluation step of referring to the reference database and generating an evaluation result for the evaluation target image; a second storage step of storing past evaluation results and instruction information instructing the supporter on an action according to the evaluation results in an instruction information database; an instruction information acquisition step of referring to the instruction information database and acquiring the instruction information corresponding to the generated evaluation result; an output step of outputting the instruction information; A subject evaluation program characterized by causing a computer to execute the above.

Images