Object evaluation device and object evaluation system

Abstract

The invention provides a subject evaluation device and a subject evaluation system, which can perform processing suitable for various types of sensors in various places according to field demands. A subject evaluation device (1) for evaluating the state of a subject (3), the subject evaluation device (1) being characterized by comprising: one or more sensors (2) for measuring the state of the subject (3); an acquisition unit that acquires subject information and feature information of the subject (3) via the sensor (2); a conversion unit that converts the subject information image into an evaluation target image on a two-dimensional plane on the basis of the feature information of the sensor (2); a reference database in which the correlation between a past image to be evaluated obtained by performing image conversion in advance and reference information associated with the past image to be evaluated is stored; an evaluation unit for generating an evaluation result for the evaluation target image by referring to the reference database; and an output unit that outputs the evaluation result.

Description

Technical Field

[0001] The present invention relates to a subject person evaluation device and a subject person evaluation system for evaluating the state of a subject person. Background Art

[0002] Conventionally, as a device that assists in evaluating the behavior of a subject person, for example, a determination device disclosed in Patent Document 1 has been proposed.

[0003] The dynamic image processing device disclosed in patent document 1 comprises: an acquisition unit, which acquires dynamic image data of a subject person photographed in a living room; an analysis unit, which analyzes the dynamic image data acquired by the acquisition unit and extracts first information related to people including the subject person and second information related to items in the living room; and a label adding processing unit, which associates with the dynamic image data to assign multiple labels related to the first information and the second information extracted by the analysis unit in order to assist in the evaluation of prescribed actions performed on the subject person. The dynamic image processing device efficiently assists in the evaluation of prescribed actions performed on the evaluation subject.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2023-001531 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Here, for example, in the moving image processing device such as Patent Document 1, the analysis of moving image data acquired by a near-infrared camera in the living room is only premised. Therefore, it is difficult to perform processing suitable for a variety of sensors in various places according to the needs of the site, except for the near-infrared camera.

[0009] Therefore, the present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a subject person evaluation device and a subject person evaluation system capable of performing processing suitable for a variety of sensors in various places according to on-site needs.

[0010] Means for solving problems

[0011] The subject person evaluation device of the first invention evaluates the state of the subject person, and is characterized in that the subject person evaluation device has: one or more sensors that measure the state of the subject person; an acquisition unit that acquires subject person information showing at least any one of the physical condition and behavior of the subject person and feature information showing the characteristics of the sensor via the sensor; a conversion unit that converts the acquired subject person information image into an evaluation object image on a two-dimensional plane based on the feature information of the sensor; a reference database that stores the correlation between past evaluation object images obtained by pre-image conversion and reference information associated with the past evaluation object images; an evaluation unit that generates an evaluation result for the evaluation object image with reference to the reference database; and an output unit that outputs the evaluation result.

[0012] The subject person evaluation device according to a second invention is characterized in that, in the first invention, the correlation is constructed by machine learning using the past evaluation target image and the reference information as learning data.

[0013] The subject person evaluation device of the third invention, in the first invention or the second invention, is characterized in that the subject person evaluation device further includes an updating unit that reflects the relationship in the correlation when a relationship between the past evaluation target image and the reference information is newly acquired.

[0014] The subject person evaluation system of the fourth invention evaluates the state of the subject person, and is characterized in that the subject person evaluation system has: one or more sensors that measure the state of the subject person; an acquisition unit that acquires subject person information showing at least any one of the physical condition and action of the subject person and feature information showing the characteristics of the sensor via the sensor; a conversion unit that converts the acquired subject person information image into an evaluation object image on a two-dimensional plane based on the feature information of the sensor; a reference database that stores the correlation between past evaluation object images obtained by pre-image conversion and reference information associated with the past evaluation object images; an evaluation unit that generates an evaluation result for the evaluation object image with reference to the reference database; and an output unit that outputs the evaluation result.

[0015] Effects of the Invention

[0016] According to the first to third inventions, the acquisition unit acquires the subject person information and feature information from one or more sensors that measure the state of the subject person. Therefore, the subject person information indicating at least one of the subject person's physical condition and behavior and the feature information indicating the characteristics of the sensors can be acquired through multiple sensors. Thus, processing suitable for multiple sensors can be performed in various places according to the needs of the site.

[0017] Furthermore, according to the first to third inventions, the conversion unit converts the acquired subject person information image into an evaluation object image on a two-dimensional plane based on the characteristic information of the sensor. Therefore, it is possible to refer to the reference database and generate an evaluation result for the evaluation object image. Thus, it is possible to perform processing suitable for a variety of sensors in various places according to the needs of the site.

[0018] In particular, according to the second invention, the association is constructed by machine learning using past evaluation target images and reference information as learning data. Therefore, even when evaluating an unknown evaluation target image different from the past evaluation target image, quantitative evaluation can be performed. As a result, processing suitable for multiple sensors can be performed in various places according to the needs of the site.

[0019] In particular, according to the third invention, when the updating unit newly acquires the relationship between the evaluation target image and the reference information, the relationship is reflected in the correlation. Therefore, even when evaluating a new evaluation target image different from the past evaluation target image, quantitative evaluation can be performed. Thus, the evaluation accuracy can be further improved, and processing suitable for a variety of sensors can be performed in various places according to the needs of the site.

[0020] According to the fourth invention, the evaluation unit generates an evaluation result for the evaluation object image by referring to the reference database. The reference information includes body information. Therefore, it is possible to generate an evaluation result based on the results of the state of the evaluation object person in the past. As a result, the accuracy of the state of the evaluation object person can be improved, and processing suitable for multiple sensors can be performed in various places according to the needs of the site. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 1 is a schematic diagram showing an example of a subject evaluation system according to the present embodiment.

[0022] Figure 2 (a) is a schematic diagram showing an example of the operation of the site A of the subject person evaluation device according to the present embodiment. Figure 2 (b) is a schematic diagram showing an example of the operation of the site B of the subject person evaluation device according to the present embodiment.

[0023] Figure 3 (a) is a schematic diagram showing an example of sensor data of the subject evaluation system according to the present embodiment. Figure 3 (b) is a schematic diagram showing an example of an evaluation target image of the subject person evaluation system according to the present embodiment.

[0024] Figure 4(a) is a schematic diagram showing an example of the structure of the subject evaluation device of this embodiment, Figure 4 (b) is a schematic diagram showing an example of the function of the subject evaluation device according to the present embodiment.

[0025] Figure 5 This is a schematic diagram showing an example of a reference database according to the present embodiment.

[0026] Figure 6 This is a flowchart showing an example of the operation of the subject evaluation system according to the present embodiment. DETAILED DESCRIPTION

[0027] Hereinafter, an example of a subject person evaluation system and a subject person evaluation device to which an embodiment of the present invention is applied will be described with reference to the drawings.

[0028] Reference Figure 1 , an example of the subject person evaluation system 100 and the subject person evaluation device 1 of the present embodiment will be described.

[0029] For example Figure 1 As shown, the subject evaluation system 100 of the present embodiment includes a subject evaluation device 1. The subject evaluation device 1 may be connected to other terminals 5 and a server 6 via a communication network 4, for example, in addition to being connected to sensors 2 (2a to 2f).

[0030] The subject evaluation system 100 evaluates the condition of the subject 3. The subject evaluation system 100 can be used, for example, for evaluation after diagnosis or observation of the subject 3 at the site A (e.g., examination or treatment by a doctor in a nursing facility, observation by a caregiver in home care).

[0031] Furthermore, the subject evaluation system 100 can also perform evaluations in scenarios other than medical treatment and nursing, for example, it can also be used to evaluate the status of the subject 3 during operations in various on-site scenarios such as transfer, diet, bathing, medication, toileting, rehabilitation, vital sign checks, suctioning, respiratory assistance, intravenous drips, blood transfusions, etc. of the subject 3, as well as distribution operations at logistics sites, assembly operations at manufacturing sites, and business operations at sales sites.

[0032] Furthermore, the subject evaluation system 100 can also evaluate the future state based on the current state of the subject 3. For example, the subject evaluation system 100 can also be used for the following evaluations: "The state has recovered, so there is no problem with home care" based on the evaluation performed after diagnosing the subject 3 in the nursing facility; or "Nursing in a nursing facility is necessary" based on the evaluation performed after observing the subject 3 in home care.

[0033] Here, refer to Figure 2, an example of the operation of the subject person evaluation device 1 in the location A is described. Figure 2 (a) of FIG. 1 shows an example of the operation of the subject person evaluation device 1 for example, taking place A (bedroom) as the subject. Figure 2 As shown in (a) of FIG. 1 , the subject evaluation device 1 measures data related to the state of the subject 3 while sleeping, for example, using one or more sensors 2 ( 2 a to 2 f ) that measure the state of the subject 3 while sleeping.

[0034] Among the sensors 2, for example, the sensor 2a is a motion sensor that measures the movement of the subject 3, and measures and digitizes the movement of the subject 3 to be measured. Also, for example, the sensor 2b is a near-infrared or non-contact vital sign sensor that measures the body temperature or vital sign of the subject 3 to be measured, and measures and digitizes the body temperature or vital sign of the subject 3 to be measured. Also, for example, the sensor 2c is a near-infrared or non-contact vital sign sensor that measures the posture or center of gravity of the subject 3 to be measured, and measures and digitizes the body temperature or vital sign of the subject 3 to be measured. Also, for example, the sensor 2c is installed at a bed leg or the like, and measures and digitizes the body orientation or posture and body center of gravity of the subject 3 to be measured by using a sensor that measures the body orientation or posture and body center of gravity of the subject 3 to be measured.

[0035] The subject person evaluation device 1 acquires various measurement data measured by each sensor 2 (2a to 2c). For example, when acquiring various measurement data measured by each sensor 2 (2a to 2c), the subject person evaluation device 1 also acquires characteristic information (for example, sensor ID, measurement data ID, measurement data characteristics, measurement date and time, measurement location, etc.) showing the characteristics of each sensor. For example, the subject person evaluation device 1 can pre-set acquisition conditions such as the date and time to be sent to the subject person evaluation device 1 and the measurement data to be sent for each sensor 2 (2a to 2c), and each sensor 2 sends to the subject person evaluation device 1 according to the set acquisition conditions.

[0036] Next, in Figure 2 (b) of FIG. 1 shows an example of the operation of the subject person evaluation device 1 for example, taking place B (living room) as the subject. Figure 2 As shown in (b) of FIG. 1 , for example, regarding the behavior state of the subject person 3 getting up and moving from the location A, the subject person evaluation device 1 measures data related to the behavior state of the subject person 3 using one or more sensors 2 ( 2 e to 2 f ).

[0037] Among the sensors 2, for example, the sensor 2e is a motion sensor that measures the motion of the subject 3, and measures and digitizes the motion of the subject 3. Also, for example, the sensor 2f is a wearable sensor that the subject 3 wears, and measures and digitizes the number of steps, movements of the hands and feet, heartbeat, breathing rate, etc. of the subject 3.

[0038] Figure 2 The sensors 2 (2a to 2f) shown in (a) and (b) can be installed in multiple locations, for example, in a facility or a home. As for the installation location, for example, it can be installed on a ceiling, a wall, a table, a bed, a chair, a wheelchair, a car, etc., or worn by the evaluator. In the case where the installation location is a bathroom, for example, it is not necessary to obtain image data of the body state of the subject 3, but to obtain sensor data using a near-infrared camera or the like. As a result, it is possible to use a variety of sensors to obtain the state of the subject 3 in various locations, etc., according to the needs of the site or the site.

[0039] The subject person evaluation device 1 acquires various measurement data measured by each sensor 2 (2e to 2f). For example, when acquiring various measurement data measured by each sensor 2 (2e to 2f), the subject person evaluation device 1 also acquires characteristic information (for example, sensor ID, measurement data ID, measurement data characteristics, measurement date and time, measurement location, etc.) showing the characteristics of each sensor. For example, the subject person evaluation device 1 can pre-set acquisition conditions such as the date and time to be sent to the subject person evaluation device 1 and the measurement data to be sent for each sensor 2 (2e to 2f), and each sensor 2 sends to the subject person evaluation device 1 according to the set acquisition conditions.

[0040] The subject person evaluation device 1 obtains subject person information showing at least any one of the physical condition and behavior of the subject person 3 and feature information showing the characteristics of the sensor 2 via the sensor 2, and converts the obtained subject person information image into an evaluation subject image on a two-dimensional plane based on the feature information of the sensor.

[0041] The subject person evaluation system 100 generates an evaluation result for the evaluation object image by, for example, referring to a reference database that stores the association between the past evaluation object images obtained by pre-image conversion and the reference information associated with the past evaluation object images. Thus, the state of the subject person 3 can be evaluated using the current evaluation object image of the subject person 3. For example, a plurality of evaluation object images can be combined for evaluation. From among the plurality of evaluation object images, various states of the subject person 3 can be confirmed or future occurrences can be predicted based on, for example, a combination of evaluation object images having features.

[0042] The subject evaluation system 100 has an output unit for outputting evaluation results, and after acquiring evaluation subject information including measurement data, generates evaluation results for evaluation subject images by referring to a reference database described later. The subject evaluation device 1 outputs the generated evaluation results to the display unit 109 or the like.

[0043] The evaluation results show not only the confirmed status of the current situation with the status of the subject 3 as the evaluation object, but also the predicted results of unconfirmed status that may occur or develop in the future. Regarding the evaluation results, in addition to the evaluation results of the status related to the body or behavior of the subject 3, such as "normal", "abnormal", "need to observe", etc., the tendency of the cases, diseases, and required care that may occur in the subject 3, such as "having a tendency of ○○ symptoms", "the possibility of ○○ symptoms is 60%, etc., can also be shown, or the symptoms, diseases, and required care that may develop in the future can be shown in probability. Thus, for example, by confirming these evaluation results, the caregiver of the subject 3 can make preparations for the future care and attention of the subject 3 in advance.

[0044] As the subject person evaluation device 1, in addition to using electronic devices such as personal computers (PCs), electronic devices such as smartphones, tablet terminals, wearable terminals, IoT (Internet of Things) devices, single-board computers such as Raspberry Pi (registered trademark) may be used, and sensors 2 may be built in. For example, when an HMD (Head Mounted Display) with a built-in sensor 2 is used as the subject person evaluation device 1, the evaluator can recognize the evaluation result of the subject person 3 by visually confirming the subject person 3 through the display. Therefore, the difficulty of the physical evaluation work for the subject person 3 can be reduced, and the evaluation work time can be shortened.

[0045] Here, refer to Figure 3 , an example of sensor data of the subject evaluation system 100 of the present embodiment will be described. Figure 3 (a) is an example of sensor data of the subject person evaluation system 100 of the present embodiment, and various sensor data measured by a plurality of sensors 2 are acquired by the subject person evaluation device 1 .

[0046] As sensor data measured by various sensors 2, for example, sensor 2a measures the state of subject 3 (e.g., information on sleeping movements, etc.), and records it as sensor data 200a. For example, coordinates and numerical values ​​of each position or movement such as the position of subject 3's head (XX.XX), the position of right shoulder (XX.XX), the position of left shoulder (XX.XX), the position of right upper arm (XX.XX), and the position of left upper arm (XX.XX) are measured in time series as sensor data 200a, and a plurality of measured sensor data 200a are recorded.

[0047] Sensor 2a measures and digitizes the movement of subject 3 using, for example, a known motion sensor. Sensor data 200a may be measured using, for example, a known motion sensor, a near infrared sensor, an image sensor, etc. (not shown), and a known measurement technique.

[0048] The sensor data 200 a measured by the sensor 2 a is acquired by, for example, the subject person evaluation device 1 , and a plurality of pieces of sensor data 200 a are recorded in the subject person evaluation device 1 . Alternatively, the sensor data 200 a may be recorded in, for example, another terminal 5 or a server 6 .

[0049] Furthermore, for example, the sensor 2b measures the state of the subject 3 (e.g., vital sign information during sleep) and records it as the sensor data 201a. For example, the numerical values ​​showing each vital sign information of the subject 3, such as the body temperature (35.50), the respiratory rate (18.00), the blood pressure value High (130.00), the blood pressure value Low (85.00), and the pulse rate (70.00), are measured in time series as the sensor data 201a, and the measured plurality of sensor data 201a are recorded.

[0050] The sensor 2b measures and digitizes the vital sign information of the subject 3 using, for example, a known near-infrared or non-contact vital sign sensor. The sensor data 201a can be measured by using, for example, a known vital sign sensor or various image sensors (not shown) and a known measurement technique.

[0051] The sensor data 201 a measured by the sensor 2 b is acquired by the subject person evaluation device 1 , for example, and a plurality of pieces of sensor data 201 a are recorded in the subject person evaluation device 1 . Alternatively, the sensor data 201 a may be recorded in another terminal 5 or server 6 .

[0052] In addition to the sensor data 200a of sensor 2a and the sensor data 201a of sensor 2b, the sensor data of other sensors 2c to 2f are also measured in the same manner, and a plurality of them are recorded as respective sensor data in the subject evaluation device 1, and in other cases, for example, in other terminals 5 or servers 6.

[0053] and, Figure 3 (b) is an example of an evaluation object image of the subject person evaluation system 100 of the present embodiment, in which various sensor data measured by the sensor 2 are converted into evaluation object images 200b~202b on a two-dimensional plane based on the characteristic information of each sensor 2 (machine type, model, conversion parameters, performance and characteristics, visualization characteristics, format, etc.).

[0054] The evaluation target image 200b is obtained by, for example, converting the sensor data 200a into the evaluation target image 200b on a two-dimensional plane based on the characteristic information of the sensor 2a. The image conversion is, for example, a known graphing process, which shows the state of the sensor data measured in time series in a graph based on a specific parameter.

[0055] The evaluation target image 200b is obtained by converting the numerical values ​​measured by the sensor 2a into the evaluation target image 200b on a two-dimensional plane in the form of a radar chart. However, if other characteristics are to be expressed, the image 200b may be converted into a chart other than a radar chart.

[0056] Similarly, the evaluation object image 201b is obtained by, for example, converting the sensor data 201a image into the evaluation object image 201b on a two-dimensional plane based on the characteristic information of the sensor 2b, for example, by converting the change image of each vital sign information in a time series into the evaluation object image 201b on a two-dimensional plane.

[0057] Furthermore, the evaluation target image 202b may be obtained by, for example, converting a sensor data image measured by another sensor 2 into the evaluation target image 202b on a two-dimensional plane based on the characteristic information of the sensor 2. Alternatively, it may be obtained by summarizing the values ​​measured by a plurality of sensors 2 and converting an image such as their ratio into the evaluation target image 201b on a two-dimensional plane.

[0058] about Figure 3 There may be a plurality of evaluation target images shown in (b) for each sensor 2a to 2f, and when measurement is performed in time series, the transition of the time series or the total value of a certain period may be converted into an image in the form of a ratio.

[0059] The subject person evaluation system 100 displays the evaluation object images 200b to 202b and other evaluation object images on a plurality of two-dimensional planes after image conversion on the display unit 109. For example, when a plurality of sensor data are measured using one sensor 2, the subject person evaluation device 1 can generate evaluation results for the subject person 3 respectively, and can display the evaluation results for the subject person 3 on the display unit 109. In addition, for example, when generating one evaluation result for the subject person 3, it can also be based on a plurality of sensor data. And, for example, the type and number of the plurality of evaluation object images combined and displayed are arbitrary.

[0060] The sensor data may be generated, for example, using an RGB camera, etc. The sensor data may be generated, for example, using a multispectral camera that selects an arbitrary wavelength, or based on shooting through a polarization filter. The sensor data may be extracted from a portion of a dynamic image, and converted into an evaluation target image by image conversion.

[0061] Regarding the subject person information, for example, the evaluator or the like directly inputs it into the subject person evaluation device 1 in a manner associated with the sensor data acquired by the subject person evaluation device 1. In addition, for example, a plurality of subject person information may be stored in advance in the subject person evaluation device 1, and the subject person evaluation device 1 may select the subject person and the sensor 2 based on the evaluation object image acquired. In the case where the subject person evaluation device 1 selects the subject person information, for example, the learning model pre-stored in the subject person evaluation device 1 may be used to select the subject person and the sensor 2 for the acquired evaluation object image. In this case, the learning model is generated by known machine learning using the evaluation object image and the subject person information prepared in advance as learning data.

[0062] The subject person information includes information related to at least one of the date and time when the subject person 3 is observed, the posture and movement measured as sensor data, the observer, and the scheduled date and time of observing the subject person 3. The subject person information may include information related to the daily life of the subject person 3, such as physical strength, training, and diet.

[0063] The subject person information may be directly input into the subject person evaluation device 1 by the evaluator or the like in a manner associated with the sensor data acquired by the subject person evaluation device 1 , or may be transmitted from another terminal 5 or the like, for example.

[0064] (Subject Evaluation Device 1)

[0065] Next, refer to Figure 4 An example of the subject evaluation device 1 according to the present embodiment will be described. Figure 4 (a) is a schematic diagram showing an example of the structure of the subject evaluation device 1 of this embodiment, Figure 4 (b) is a schematic diagram showing an example of the function of the subject evaluation device 1 according to the present embodiment.

[0066] For example Figure 4 As shown in (a) of FIG. 1 , the subject evaluation device 1 includes a housing 10 , a CPU (Central Processing Unit) 101 , a ROM (Read Only Memory) 102 , a RAM (Random Access Memory) 103 , a storage unit 104 , and I / Fs 105 to 107 . The components 101 to 107 are connected via an internal bus 110 .

[0067] The CPU 101 controls the entire subject person evaluation device 1. The ROM 102 stores the action code of the CPU 101. The RAM 103 is a work area used when the CPU 101 operates. The storage unit 104 stores various information such as evaluation object images and reference databases. As the storage unit 104, in addition to using, for example, a HDD (Hard Disk Drive), a data storage device such as an SSD (Solid State Drive) can also be used. In addition, for example, the subject person evaluation device 1 can also have a GPU (Graphics Processing Unit) not shown in the figure. By having a GPU, high-speed computing processing can be performed compared to normal.

[0068] The I / F 105 may be an interface for transmitting and receiving various information with the sensor 2 or may be an interface for transmitting and receiving various information with other terminals 5 or servers 6 via a communication network 4 such as the Internet.

[0069] The I / F 106 is an interface for transmitting and receiving information to and from the input unit 108. The input unit 108 may be, for example, a keyboard, and an evaluator using the subject evaluation device 1 inputs various information or control commands for the subject evaluation device 1 through the input unit 108.

[0070] I / F 107 is an interface for transmitting and receiving various information to and from display unit 109. Display unit 109 outputs various information such as evaluation results stored in storage unit 104 or processing status of subject evaluation device 1. Display unit 109 uses a display, such as a touch panel.

[0071] <Reference database>

[0072] The reference database stored in the storage unit 104 stores the association between the past evaluation target images acquired in advance and the reference information associated with the past evaluation target images, for example, a learning model with the association is stored. The reference database may also store, for example, past evaluation target images and reference information. For example, the past evaluation target images and reference information are used as a set of learning data, and the association is constructed by machine learning using a plurality of learning data. As a learning method, deep learning such as a convolutional neural network is used, for example.

[0073] In this case, for example, the correlation represents the degree of association between many-to-many information (multiple data included in the past evaluation object image and multiple data included in the reference information). The correlation is appropriately updated during the machine learning process. That is, the correlation, for example, shows a function optimized based on the past evaluation object image (image data on a two-dimensional plane) and the reference information. Therefore, the evaluation result for the evaluation object image is generated using the correlation constructed based on all the results of the state of the evaluation object person 3 in the past. As a result, even when the physical condition or behavior of the object person 3 is in a variety of states, the optimal evaluation result can be generated.

[0074] Furthermore, in addition to the case where the evaluation target image is the same or similar to the past evaluation target image, the optimal evaluation result can be quantitatively generated even when the evaluation target image is not similar. In addition, by improving the generalization ability when performing machine learning, the evaluation accuracy for unknown evaluation target images can be improved.

[0075] Furthermore, the relevance may include, for example, a plurality of relevance degrees indicating the degree of relevance between a plurality of data included in a past evaluation target image and a plurality of data included in the reference information. For example, when the learning model is constructed by a neural network, the relevance degree may correspond to a weight variable.

[0076] The past evaluation target images show the same type of information as the above-mentioned evaluation target images. For example, the past evaluation target images include evaluation target images acquired when a plurality of past evaluation subjects 3 are present.

[0077] The reference information is associated with past evaluation target images, and shows information related to the state of the subject 3. In addition to showing an evaluation based on the state of the subject 3 (e.g., "normal," "abnormal," "observation required," "tendency to have ○○ symptoms," "possibility of ○○ symptoms is 60%," etc.), the reference information may also include physical information related to the factors of the state of the subject 3, response information, preparation information, prediction information, etc.

[0078] The reference information may also show the tendency of possible cases, conditions, required care, etc. of the subject 3, or show the symptoms, conditions, required care, etc. that may occur in the future with probability. In addition, the specific content included in the reference information can be set arbitrarily.

[0079] Regarding the physical information, for an elderly person, the names of specific factors of the condition are shown, such as dementia, dehydration symptoms, walking disorder, mental and psychological disorder, mobility disorder, excretion disorder, sensory disorder, and nutritional intake disorder. The various factors are usually related to at least a part of the physical condition or behavior of the subject 3.

[0080] For example, Figure 5As shown in FIG. 1 , the correlation can indicate the degree of correlation between the past evaluation target image and the reference information. In this case, by using the correlation, the multiple data (in Figure 5 Each of the "image data A" to "image data C" in the reference information and the plurality of data (in Figure 5 Therefore, for example, one data included in a past evaluation target image can be associated with a plurality of data included in the reference information through association, and various evaluation results can be generated.

[0081] Furthermore, the correlation has, for example, a plurality of correlation degrees that respectively associate a plurality of data contained in a past evaluation object image with a plurality of data contained in the reference information. The correlation degree is expressed, for example, in three or more stages such as a percentage, 10 stages, or 5 stages, and is expressed, for example, by line features (such as thickness, etc.). For example, the "image data A" contained in the past evaluation object image shows a correlation degree AA "85%" with the "reference A" contained in the reference information, and shows a correlation degree AB "55%" with the "reference B" contained in the reference information. That is, the "correlation degree" indicates the degree of correlation between each image data, for example, the higher the correlation degree, the stronger the correlation between each data. In addition, when constructing the correlation through the above-mentioned machine learning, it can also be set that the correlation has a correlation degree of more than three stages.

[0082] The past evaluation target images may be divided into past image data A to C and past physical condition information or action information and stored in the reference database, for example. In this case, the correlation degree is calculated based on the relationship between the combination of the past image data and the past physical condition information or action information and the reference information. In addition, the past evaluation target images may be divided into past observation information (various sensor data, etc.) in addition to the above and stored in the reference database, for example.

[0083] In addition, the past evaluation target image may also include, for example, synthetic data and similarity. The synthetic data is represented by similarity of more than three stages with past image data or past physical condition information or action information. In addition to being stored in the reference database in the form of numerical values, matrices, or histograms, the synthetic data may also be stored in the form of images, character strings, etc.

[0084] Figure 4 (b) is a schematic diagram showing an example of the functions of the subject person evaluation device 1. The subject person evaluation device 1 includes an acquisition unit 11, a conversion unit 12, an evaluation unit 13, and an output unit 14, and may also include an update unit 16. Figure 4Each function shown in (b) is implemented by the CPU 101 executing a program stored in the storage unit 104 or the like using the RAM 103 as a work area, and can be controlled by artificial intelligence, for example.

[0085] <Acquisition Unit 11>

[0086] The acquisition unit 11 acquires subject information indicating at least any one of the physical condition and behavior of the subject 3 and feature information indicating the feature of the sensor 2 via one or more sensors that measure the state of the subject 3. In addition to acquiring sensor data indicating the subject information of the subject 3 from the sensor 2, for example, when a camera is built in, the acquisition unit 11 may acquire image data of the subject 3, spatial image data of the place, etc. from the sensor 2. In addition to acquiring the physical condition information and behavior information of the subject 3 input in advance from the evaluator, etc., the acquisition unit 11 acquires feature information for identifying the sensor 2 from the sensor 2, etc.

[0087] For example, when acquiring sensor data, the acquisition unit 11 acquires the characteristic information of the sensor 2 together with the sensor data. The frequency and period at which the acquisition unit 11 acquires the subject person information and the characteristic information are arbitrary.

[0088] The acquisition unit 11 receives various information including sensor data and feature information of the evaluator measured by the sensor 2. The acquisition unit 11 can receive various information such as physical condition information and action information of the subject 3, observation information, and environmental information related to the location, which are sent from external terminals such as other terminals 5 via the communication network 4 and the I / F 105.

[0089] For example, the acquisition unit 11 may refer to the learning model stored in the storage unit 104 , select the physical condition information and action information corresponding to the sensor 2 , the sensor data, and the feature information, and acquire them as the evaluation target image.

[0090] <Conversion Unit 12>

[0091] The conversion unit 12 converts the subject person information image acquired by the acquisition unit 11 into an evaluation object image on a two-dimensional plane based on the characteristic information of the sensor 2. Figure 3 As shown in FIG. 1 , the conversion unit 12 converts, for example, sensor data 200a measured by various sensors 2a into an image 200b to be evaluated. The conversion unit 12 converts the image into an evaluation target image 200b based on, for example, sensor characteristic information of the sensor 2a or various information (such as machine type, model, conversion parameter, performance and characteristics, visualization characteristics, format, etc.) as characteristic information of the sensor data. Figure 3 A graph such as a radar chart as shown in (b) .

[0092] The conversion unit 12 performs image conversion by, for example, graphing the position of the head (XX.XX), the position of the right shoulder (XX.XX), the position of the left shoulder (XX.XX), the position of the right upper arm (XX.XX), the position of the left upper arm (XX.XX), which are items indicating the state of the subject 3 acquired by the sensor 2a, and the values ​​of the respective items. The conversion unit 12 may, for example, refer to a graphed correspondence table (not shown) and graph the sensor data 200a measured by the sensor 2a to perform image conversion.

[0093] For example, if the measured sensor data 200a has a local feature, the conversion unit 12 may graph only the range and value of the local feature. The conversion unit 12 may, for example, graph the sensor data of the target sensor 2 based on an instruction from another terminal 5 and perform image conversion. The conversion unit 12 may also perform the same image conversion for the sensor 2b and other sensors 2. The conversion unit 12 may, for example, combine the numerical data measured by a plurality of sensors 2 to generate a new graph, and use the combined graph as the evaluation target image.

[0094] Furthermore, the conversion unit 12 can, for example, aggregate the values ​​measured by the plurality of sensors 2, and perform image conversion by graphing their ratios or shifts. For example, when the sensor data is measured in a time series and there is a significant increase or decrease trend in a certain period, the conversion unit 12 can graph the trend and store the period, value and other conditions and parameters when the graph is formed, and various setting information when the graph is formed. Thus, the evaluation target image can be confirmed by tracing back the period of the graph, and image conversion processing suitable for a variety of sensors can be performed in various places according to the needs of the site.

[0095] <Evaluation Department 13>

[0096] The evaluation unit 13 refers to the reference database and generates an evaluation result for the evaluation target image. For example, the evaluation unit 13 uses the evaluation target image as input data, selects optimal reference information associated with the solution calculated based on the correlation, and generates an evaluation result based on the optimal reference information.

[0097] The evaluation unit 13 refers to Figure 5 In the case of a reference database shown in FIG. 1 , data identical or similar to the data included in the evaluation target image is selected (e.g., "image data A": set as the first data). As the first data, in addition to selecting image data that is partially or completely identical to the evaluation target image, similar image data can also be selected. In the case where the evaluation target image is represented by numerical values ​​such as a matrix, the numerical range included in the first data to be selected can be preset.

[0098] The evaluation unit 13 selects reference information associated with the selected first data and a degree of association (first degree of association) between the selected first data and the reference information, and generates an evaluation result based on the selected reference information and the first degree of association. In addition, the first degree of association may be selected from pre-constructed degrees of association or calculated by the evaluation unit 13.

[0099] For example, the evaluation unit 13 selects the data "reference A" included in the reference information associated with the first data "image data A" and the first degree of association (degree of association AA) "85%" between "image data A" and "reference A". In addition, the reference information and the first degree of association may also include a plurality of data. In this case, in addition to the above-mentioned "reference A" and "85%", the reference information "reference B" associated with the first data "image data A" and the first degree of association (degree of association AB) "55%" between "image data A" and "reference B" may also be selected, and an evaluation result may be generated based on "reference A" and "85%" and "reference B" and "55%".

[0100] The evaluation result may include an evaluation target image. The evaluation result may represent the factors of the state (physical condition or behavior) of the subject 3 in a probabilistic manner using reference information and a degree of association, for example.

[0101] The evaluation unit 13 generates an evaluation result in a format (e.g., a character string) that enables an evaluator to understand the selected reference information and the first degree of association, etc., using format data such as an output format pre-stored in the storage unit 104, etc. In addition, the setting of the format when generating the evaluation result, etc., can use, for example, a well-known technology.

[0102] The evaluation unit 13 determines the content of the evaluation result based on the selected first degree of association, for example. For example, the evaluation unit 13 can be set to generate an evaluation result based on reference information associated with a first degree of association greater than "50%", and not reflect reference information associated with a first degree of association less than "50%" in the evaluation result. In addition, regarding the judgment criterion based on the first degree of association, for example, a threshold value can be pre-set by an evaluator, and the range of the threshold value can be arbitrarily set. Furthermore, the evaluation unit 13 can also determine the content of the evaluation result based on the result obtained by calculating more than two first degrees of association or comparing more than two first degrees of association, for example.

[0103] <Output Unit 14>

[0104] The output unit 14 outputs the evaluation result. In addition to sending the evaluation result to the display unit 109 via the I / F 107, the output unit 14 also sends the evaluation result to other terminals 5, etc. via, for example, the I / F 105. The output unit 14, for example, displays the evaluation result. Figure 3The evaluation target image of the subject 3 shown, the optimal evaluation result corresponding to the physical condition and behavior of the subject 3, the data of recommendations, etc. are output to the display unit 109 and the like.

[0105] In addition to displaying the results of an evaluation based on the status of the subject 3 (for example, "normal", "abnormal", "observation required", "tendency to have ○○ symptom", "the possibility of ○○ symptom is 60%", etc.), the output unit can also display, for example, physical information, response information, preparation information, prediction information, etc. related to factors of the current or future status of the subject 3.

[0106] <Storage Unit 15>

[0107] The storage unit 15 retrieves various information such as a reference database stored in the storage unit 104 as needed. The storage unit 15 stores various information acquired or generated by each of the components 11 and 13 to 15 in the storage unit 104 .

[0108] <Update Section 16>

[0109] For example, when the relationship between the past evaluation target image and the reference information is newly acquired, the updating unit 16 reflects the relationship in the association. For example, when the subject person evaluation device 1 acquires the determination result of the accuracy of the evaluation result determined by the evaluator or the like based on the evaluation result generated by the evaluation unit 13, the updating unit 16 updates the association stored in the reference database based on the determination result.

[0110] <Display Unit 109>

[0111] The display unit 109 displays the evaluation result. The display unit 109 displays, for example, Figure 3 The evaluation object images 200b to 202b and the evaluation results are displayed as shown. The evaluation object images 200b to 202b are displayed based on the subject person information acquired by the acquisition unit 11 via the sensor 2 and the characteristic information indicating the characteristics of the sensor 2. The evaluation object image is obtained by converting the subject person information image into the evaluation object image on the two-dimensional plane based on the characteristic information of the sensor 2.

[0112] In addition, the display unit 109 may display the evaluation result using only a list or a character string, for example. The above display method may use a known technique. In addition, for example, when an HMD is used as the subject person evaluation device 1, a transmissive display is used as the display unit 109. In this case, the display unit 109 may display the evaluation object image and the evaluation result for the subject person 3 visually recognized by the evaluator or the like through the display unit 109.

[0113] <Update Section 16>

[0114] The updating unit 16 updates the reference database, for example. When the updating unit 16 newly acquires the relationship between the past evaluation target image and the reference information, the updating unit 16 reflects the relationship in the relevance. For example, when the evaluator or the like determines the accuracy of the content of the evaluation result based on the evaluation result generated by the evaluation unit 13 and the subject evaluation device 1 acquires the determination result, the updating unit 16 updates the relevance included in the reference database based on the determination result.

[0115] <Sensor 2>

[0116] The sensor 2 is a well-known sensor that measures the physical condition and behavior of the subject 3 and generates sensor data. As the sensor 2, various sensors such as a motion sensor, a near-infrared camera, an RGB camera, an ultrasonic sensor, and a position determination sensor are used, and a plurality of sensors can be used simultaneously or in cooperation at different locations. The sensor 2 can be built into the subject evaluation device 1, or can be held or worn by the subject 3.

[0117] <Communication Network 4>

[0118] The communication network 4 is, for example, the Internet that connects the subject evaluation device 1, the plurality of sensors 2, etc. via a communication circuit. The communication network 4 may be formed by a so-called optical fiber communication network. In addition to a wired communication network, the communication network 4 may also be implemented by a known communication network such as a wireless communication network.

[0119] <Other Terminals 5>

[0120] As the other terminal 5, for example, a device embodied by an electronic device can be used in the same manner as the subject person evaluation device 1. As for the other terminal 5, for example, a central control device capable of communicating with a plurality of subject person evaluation devices 1 is shown. The other terminal 5 can be connected to a plurality of subject person evaluation devices 1, and can acquire evaluation results generated by each subject person evaluation device 1. Thus, for example, the evaluation results of the subject 3 measured at a plurality of parts can be analyzed, and the physical condition of the subject 3 can be improved.

[0121] <Server 6>

[0122] The server 6 stores, for example, the above-mentioned various information. In the server 6, for example, various sensor data or various information related to the plurality of sensors 2 transmitted via the communication network 4 are stored. In the server 6, for example, the same information as the storage unit 104 can be stored, and various sensor data, various information related to the plurality of sensors 2, image data, evaluation results, etc. can be sent and received with one or more subject person evaluation devices 1 via the communication network 4. That is, the subject person evaluation device 1 can also use the server 6 instead of the storage unit 104.

[0123] (An example of operation of the subject evaluation system 100)

[0124] Next, an example of the operation of the subject evaluation system 100 according to the present embodiment will be described. Figure 6 This is a flowchart showing an example of the operation of the subject evaluation system 100 according to the present embodiment.

[0125] <Acquisition Unit S110>

[0126] like Figure 6 As shown, the subject person image and feature information are acquired (acquisition unit S110). The acquisition unit 11 acquires, for example, subject person information indicating at least one of the physical condition and behavior of the subject person 3 and feature information indicating the features of the sensor 2 from one or more sensors 2 that measure the state of the subject person 3 via the sensor 2. The acquisition unit 11 stores the evaluation subject information and feature information in the storage unit 104 via the storage unit 15, for example.

[0127] The acquisition unit 11 acquires, for example, subject person information indicating at least one of the physical condition and behavior of the subject person 3 and feature information indicating the features of the sensor 2. The acquisition unit 11 may be, for example, a plurality of sensors 2. The subject person information and feature information are input to the subject person evaluation device 1 by an evaluator or the like in a manner associated with image data or the like, and, for example, each piece of information suitable for the image data may be selected by the acquisition unit 11 based on the image data. In this case, the acquisition unit 11 selects each piece of information suitable for the image data from the plurality of pieces of information (sensor data, etc.) of the observed subject person 3 pre-stored in the storage unit 104.

[0128] For example, when a subject person 3 is photographed using a plurality of sensors 2, the acquisition unit 11 acquires a plurality of sensor data (subject person information) and feature information measured by the plurality of sensors 2 as conversion source data of a single evaluation subject image to be converted. The acquisition unit 11 may acquire sensor data (subject person information and feature information) showing the state of the subject person 3 based on the physical condition and behavior pattern of the evaluator, for example, based on the life and behavior pattern of the subject person 3.

[0129] The acquisition unit 11 may, for example, receive the sensor data of the sensor 2 measured in an arbitrary period at once and convert the image thereof into the acquired evaluation object image on the two-dimensional plane. In addition, when the characteristic information of each sensor 2 has been previously acquired by the subject person evaluation device 1 and has not been changed, for example, only the sensor data of the subject person information may be acquired from the sensor 2. In this case, the subject person evaluation device 1 may identify the sensor data acquired from the sensor 2 and acquire the characteristic information of the sensor 2 associated with the sensor data.

[0130] <Conversion Unit S120>

[0131] Next, the conversion unit 12 converts the image into an evaluation target image on a two-dimensional plane (conversion unit S120). The conversion unit 12 converts the subject information (sensor data) of the subject 3 acquired by the acquisition unit 11 via the sensor 2 into an evaluation target image according to the characteristic information of the corresponding sensor 2, and the evaluation target image is visualized as a chart on a two-dimensional plane. The conversion unit 12 converts the subject information acquired by the acquisition unit 11 into a chart or the like according to the characteristic information of multiple sensors 2 as an evaluation target image on a two-dimensional plane.

[0132] In addition, the conversion unit 12 converts the image into a graph as the object image on the two-dimensional plane, but it can also be converted into an image other than a graph. The evaluation object image converted by the conversion unit 12 can be an image that converts the state of the subject 3 into a two-dimensional plane, and the type, scale, number and expression of the graph are arbitrary.

[0133] <Evaluation Unit S130>

[0134] Next, the evaluation unit 13 refers to the reference database and converts the image into an evaluation target image on a two-dimensional plane (evaluation means S130). The evaluation unit 13 obtains the evaluation target image obtained by the acquisition unit 11, for example, from the reference database stored in the storage unit 104. The evaluation unit 13, for example, uses the evaluation target image as input data, selects the optimal reference information associated with the solution calculated based on the association shown by the function, etc., and generates an evaluation result based on the optimal reference information. At this time, for example, the evaluation unit 13 can select a plurality of reference information for one evaluation target image.

[0135] The evaluation unit 13 may generate one evaluation result for one evaluation target image or, for example, generate one evaluation result for a plurality of evaluation target images. The evaluation unit 13 generates the evaluation result using, for example, format data such as an output format stored in the storage unit 104. The evaluation unit 13 stores the evaluation result in the storage unit 104 via, for example, the storage unit 15.

[0136] <Output Unit S140>

[0137] Next, the output unit 14 outputs the evaluation result (output means S140). The output unit 14 outputs the evaluation result to the display unit 109, etc. The output unit 14 may output the evaluation result to other terminals 5 or the server 6 via the communication network 4, for example.

[0138] The output unit 14 can output information for causing the display unit 109 to display the following contents, for example: the subject 3 based on the image data; the sensor 2 capable of measuring the subject information of the subject 3; a display image reflecting the sensor data measured by the sensor 2, the sensing conditions and sensing results of the sensor 2, a plurality of evaluation target images, the evaluation results of the subject 3 based on the combination of the evaluation target images, and recommendation information based on the evaluation results; a designation unit (not shown) that designates the subject 3 as the evaluation target in the display image; and the evaluation results for the subject 3 that became the evaluation target via the designation unit. Thus, the display image, the designation unit, and the evaluation results are displayed on the display unit 109.

[0139] Thus, the operation of the subject evaluation system 100 of the present embodiment is completed. Note that the timing at which the updating unit 16 performs the updating is arbitrary.

[0140] According to the present embodiment, the evaluation unit 13 generates an evaluation result for the evaluation object image by referring to the reference database. The reference information includes various body information and action information of the subject 3. Therefore, it is possible to generate an evaluation result based on the results of the past evaluation of the state of the subject 3. As a result, the accuracy of the evaluation of the state of the subject 3 can be improved, and processing suitable for multiple sensors can be performed in various places according to the needs of the site.

[0141] Furthermore, according to the present embodiment, the evaluation object image includes physical condition information. Therefore, it is possible to implement evaluation based on the characteristics of the factors of the state, which are different depending on the sensor 2 and the physical condition of the subject 3. As a result, the accuracy of evaluating the state of the subject 3 can be further improved, and processing suitable for multiple sensors can be performed in various places according to the needs of the site.

[0142] Furthermore, according to the present embodiment, the evaluation target image includes the action information. Therefore, it is possible to realize the evaluation based on the surface posture of the subject 3, which is different according to the measurement conditions of the sensor 2 and the subject 3. As a result, the accuracy of the evaluation of the state of the subject 3 can be further improved, and processing suitable for a variety of sensors can be performed in various places according to the needs of the site.

[0143] Furthermore, according to this embodiment, it is possible to obtain evaluation results associated with the type of sensor 2 of the subject 3, the specifications of the sensor 2, and the conditions of the configuration, so that it is possible to identify the conditions that are the cause of the physical abnormality of the subject 3, and to grasp the physical changes accompanying the change of the observation conditions. As a result, it is possible to achieve physical improvement of the subject 3 and reduction of the care burden, etc.

[0144] Furthermore, according to the present embodiment, the evaluation object image includes feature information. Therefore, it is possible to implement evaluation based on features of factors of measurement, which differ according to the type, characteristics, number, and arrangement of the sensors 2. As a result, it is possible to further improve the accuracy of the status of the evaluation object 3, and to perform processing suitable for a variety of sensors in various locations according to the needs of the site.

[0145] Furthermore, according to the present embodiment, the correlation is constructed by machine learning using past evaluation target images and reference information as learning data. Therefore, even when evaluating an unknown evaluation target image different from past evaluation target images, quantitative evaluation can be performed. Thus, further improvement in evaluation accuracy can be achieved.

[0146] Furthermore, according to the present embodiment, when the relationship between the past evaluation target image and the reference information is newly acquired, the updating unit 16 reflects the relationship in the relevance. Therefore, the relevance can be easily updated, and continuous improvement of evaluation accuracy can be achieved.

[0147] Furthermore, according to the present embodiment, the evaluation unit S130 generates an evaluation result for the evaluation object image by referring to the reference database. The reference information includes information related to the body. Therefore, an evaluation result based on the results of the state of the evaluation object 3 in the past can be generated. Thus, the accuracy of the state of the evaluation object 3 can be improved.

[0148] The embodiments of the present invention have been described, but the embodiments are presented as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the main purpose of the invention. These embodiments and their variations are included in the scope and main purpose of the invention, and are included in the invention described in the claims and the scope of their equivalents.

[0149] Description of symbols

[0150] 1: subject person evaluation device; 2: sensor; 2a~2f: sensor; 3: subject; 4: communication network; 5: other terminals; 6: server; 10: housing; 11: acquisition unit; 12: conversion unit; 13: evaluation unit; 14: output unit; 15: storage unit; 16: update unit; 100: subject person evaluation system; 101: CPU; 102: ROM; 103: RAM; 104: storage unit; 105: I / F; 106: I / F; 107: I / F; 108: input unit; 109: display unit; 110: internal bus; 200a: sensor data; 201a: sensor data; 200b: evaluation object image; 201b: evaluation object image; 202b: evaluation object image; A: location; B: location; S110: acquisition unit; S120: conversion unit; S130: evaluation unit; S140: output unit.

Claims

1. A subject person evaluation device, which evaluates the state of a subject person, characterized in that: The subject person evaluation device comprises: One or more sensors for measuring the state of the subject; an acquisition unit that acquires, via the sensor, subject person information indicating at least any one of a physical condition and an action of the subject person and feature information indicating a feature of the sensor; a conversion unit that converts the acquired subject person information image into an evaluation object image on a two-dimensional plane based on the feature information of the sensor; a reference database storing a correlation between past evaluation target images obtained by performing image conversion in advance and reference information associated with the past evaluation target images; an evaluation unit that refers to the reference database and generates an evaluation result for the evaluation target image; as well as An output unit outputs the evaluation result.

2. The subject evaluation device according to claim 1, characterized in that: The association is established by machine learning using the past evaluation target images and the reference information as learning data.

3. The subject evaluation device according to claim 1 or 2, characterized in that: The subject person evaluation device further includes an updating unit that, when a relationship between the past evaluation target image and the reference information is newly acquired, reflects the relationship in the association.

4. A subject evaluation system, which evaluates the status of a subject, characterized in that: The subject evaluation system comprises: One or more sensors for measuring the state of the subject; an acquisition unit that acquires, via the sensor, subject person information indicating at least any one of a physical condition and an action of the subject person and feature information indicating a feature of the sensor; a conversion unit that converts the acquired object person information image into an evaluation object image on a two-dimensional plane according to the characteristic information of the sensor; a reference database storing a correlation between past evaluation target images obtained by performing image conversion in advance and reference information associated with the past evaluation target images; an evaluation unit, which generates an evaluation result for the evaluation object image by referring to the reference database; as well as An output unit outputs the evaluation result.

Images