Fatigue estimation system, fatigue estimation method, posture estimation device, and program

Abstract

A fatigue estimation system (200) comprises: a plurality of imaging devices (201) that individually output images in which different parts of the body of a subject (11) are imaged; a posture estimation device (posture estimation unit (105)) that estimates the posture of the subject (11) on the basis of a plurality of images separately outputted by the plurality of imaging devices (201); and a fatigue estimation device (100) that estimates and outputs a level of fatigue of the subject (11) on the basis of the estimation result of the posture of the subject (11).

Description

[Technical field]

[0001] The present disclosure relates to a fatigue estimation system for estimating a fatigue level of a subject, a posture estimation device, a fatigue estimation method, and a program used in the fatigue estimation system. [Background technology]

[0002] In recent years, there have been many cases where accumulated fatigue has led to poor health, injuries, accidents, etc. In response to this, attention has been focused on technology that can prevent poor health, injuries, accidents, etc. by estimating the degree of fatigue. For example, Patent Document 1 discloses a fatigue determination device as a fatigue estimation system for estimating the degree of fatigue, which determines the presence or absence of fatigue and the type of fatigue based on force measurement and bioelectrical impedance measurement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-023311 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional fatigue determination device exemplified in the above Patent Document 1, the posture estimated may not be appropriate, and the fatigue level may not be estimated. Therefore, the present disclosure provides a fatigue estimation system that estimates the posture more appropriately to estimate the fatigue level. [Means for solving the problem]

[0005] A fatigue estimation system according to one aspect of the present disclosure includes a plurality of imaging devices, each of which outputs images capturing different parts of a subject's body, a posture estimation device that estimates a posture of the subject based on the images output by each of the plurality of imaging devices, and a fatigue estimation device that estimates and outputs a fatigue level of the subject based on the estimated result of the subject's posture.

[0006] Moreover, a posture estimation device according to one aspect of the present disclosure is the posture estimation device described above.

[0007] Furthermore, a fatigue estimation method according to one aspect of the present disclosure is a fatigue estimation method executed by a fatigue estimation device, which acquires images from each of a plurality of imaging devices, each outputting images of a different part of a subject's body part, estimates joint positions of body parts including one body part and the other body part as the posture of the subject based on one image of the plurality of images in which the one body part is captured and another image of the plurality of images in which another body part sharing at least one joint with the one body part, and estimates a fatigue level of the subject based on the estimated posture of the subject and outputs the result to an output device connected to the fatigue estimation device.

[0008] A program according to an aspect of the present disclosure is a program for causing a computer to execute the fatigue estimation method described above. Effect of the Invention

[0009] A fatigue estimation system according to an embodiment of the present disclosure can more appropriately estimate posture and estimate fatigue level. [Brief description of the drawings]

[0010] [Figure 1A] FIG. 1A is a first diagram for explaining estimation of a fatigue level according to an embodiment. [Figure 1B] FIG. 1B is a second diagram for explaining estimation of a fatigue level according to the embodiment. [Figure 1C] FIG. 1C is a third diagram for explaining estimation of a fatigue level according to the embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating a functional configuration of the fatigue estimation system according to the embodiment. [Diagram 3] FIG. 3 is a flowchart showing a method for estimating a fatigue level according to the embodiment. [Figure 4A] FIG. 4A shows a subject resting in posture A. [Figure 4B] FIG. 4B shows a subject resting in posture B. [Figure 5A] FIG. 5A is a first diagram illustrating an estimated accumulation of a fatigue level of a subject according to an embodiment. [Figure 5B] FIG. 5B is a second diagram illustrating the estimated accumulation of a subject's fatigue level in the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of synthesis of joint positions according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of displaying an estimation result according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims will be described as optional components.

[0012] In addition, each drawing is a schematic diagram and is not necessarily a precise illustration. In each drawing, the same reference numerals are used for substantially the same configurations, and duplicated explanations may be omitted or simplified.

[0013] (Embodiment) [overview] In the fatigue estimation system 200 (see FIG. 2 described later) in this embodiment, an image of the subject 11 (see FIG. 1A described later) whose fatigue level is to be estimated is captured, the posture of the subject is estimated, and the fatigue level of the subject 11 is estimated from the estimated posture. Here, depending on the positional relationship between the subject 11 and the imaging device 201 (see FIG. 1A described later) for acquiring an image, there may be an obstruction between them, making it difficult to obtain an image of the subject 11 required to estimate the posture of the subject 11. As a result, in some cases, it may be impossible to estimate the fatigue level of the subject 11.

[0014] Therefore, in the present disclosure, a plurality of imaging devices 201, each capable of imaging at least a portion of the subject 11, are used to compensate for missing parts in an image of the subject 11 required to estimate the posture of the subject 11. In this way, a fatigue estimation system 200 is provided that enables estimation of the fatigue level of the subject 11 even when the posture of the subject 11 cannot be estimated by one imaging device 201. In addition, although details will be described later, in the present disclosure, in order to estimate fatigue of the subject 11, the load on at least one of the muscles and joints and the deteriorating blood flow are estimated from the estimated posture of the subject 11.

[0015] Therefore, in the present disclosure, the posture estimation device used for posture estimation can also be used to estimate muscle load, joint load, and the degree of deterioration of blood flow. In addition, the posture estimation device in the present disclosure can be applied as a device for estimating the posture of the subject 11 for various other purposes. That is, the posture estimation device in the present disclosure can generate appropriate information on the posture of the subject 11 (two-dimensional skeletal information, three-dimensional skeletal information, and feature amounts such as the angle between the neck and the spine and the angle between the spine and the lower leg) even from images captured by the imaging devices 201 arranged such that only a part of the body of the subject 11 is captured in each imaging device 201, and this can be applied to various purposes.

[0016] [Fatigue estimation system] The overall configuration of a fatigue estimation system 200 according to an embodiment will be described below. Fig. 1A is a first diagram for explaining fatigue level estimation according to an embodiment. Fig. 1B is a second diagram for explaining fatigue level estimation according to an embodiment. Fig. 1C is a third diagram for explaining fatigue level estimation according to an embodiment.

[0017] In an embodiment, fatigue estimation system 200 in the present disclosure is a system that estimates a fatigue level of subject 11 by using an image output by imaging subject 11 using imaging device 201. Imaging device 201 is not limited in form as long as it is a camera that images subject 11 and outputs the image, and may be realized by a fixed camera installed on a wall or ceiling of a building or the like as shown in Fig. 1A.

[0018] Here, the subject is seated on a chair 12. In the fatigue estimation system 200 of the present disclosure, the fatigue level of the subject 11 is estimated based on fatigue accumulated by the subject 11 being in a fixed, stationary posture. In other words, fatigue accumulated by the load on at least one of the muscles and joints and the worsening blood flow (hereinafter also referred to as a decrease in blood flow rate) due to the fixed posture is estimated. Therefore, the subject 11 is in a stationary posture in a sitting, lying, or standing position for at least a certain period of time. The certain period is, for example, several tens of seconds or a few seconds, which is the minimum period during which fatigue can be estimated by the fatigue estimation system 200. Such a period is determined depending on the processing capacity of the imaging device 201 and the fatigue estimation device 100 (see FIG. 2 described later) constituting the fatigue estimation system 200.

[0019] Examples of subjects 11 who take such a stationary posture include desk workers in an office, drivers steering a moving object, people performing strength training using a load in a stationary posture, residents of facilities such as hospitals, passengers and crew of an airplane, etc.

[0020] An image captured by imaging device 201 and output is processed by estimation device 100, and the posture of subject 11 (e.g., joint position 11a) is estimated as shown in FIG. 1B. The estimated posture of subject 11 is output as a rigid link model, for example. Specifically, as shown in FIG. 1B, skeletons shown as straight lines are connected by joints shown by black dots, and the posture of subject 11 can be reproduced by the positional relationship between two skeletons connected by one joint. The posture is estimated by image recognition, and is output as the above-mentioned rigid link model based on the positional relationship between the joints.

[0021] By fitting the estimated rigid link model to a musculoskeletal model 11b as shown in FIG. 1C, the load on at least one of the muscles and joints of each body part is calculated as an estimate in order to maintain the positional relationship according to the estimated posture for each body part of the muscles that pull the skeletons together and the joints that variably connect the positional relationship between the skeletons. Since the estimated load on at least one of the muscles and joints of each body part is accumulated as the duration of the static posture is extended, the fatigue level of the subject 11 caused by maintaining the static posture is calculated by a calculation using the estimated load and the duration. In the following description, "at least one of the muscles and joints" is also expressed as "muscles and / or joints".

[0022] In addition, in this embodiment, in addition to the estimated value of the load on the muscles and / or joints, the fatigue level can be estimated based on the estimated value of the blood flow of the subject 11. In the following description, an example in which the fatigue level of the subject 11 is estimated using the estimated values ​​of the load on the muscles and the load on the joints will be mainly described, but it is also possible to combine the estimated value of the blood flow to estimate the fatigue level of the subject 11 with this and estimate the fatigue level of the subject 11 with higher accuracy. Furthermore, the fatigue level of the subject 11 can be estimated using any one of the estimated values ​​of the load on the muscles, the load on the joints, and the blood flow of the subject 11. In addition, in the present disclosure, the estimation of the fatigue level of the subject 11 based on the posture of the subject 11 is not limited to the above example, and any existing technology related to the estimation of the fatigue level can be applied.

[0023] As an example, in this embodiment, the fatigue estimation system 200 estimates the posture of the subject 11, and then estimates at least one of the load on the muscles, the load on the joints, and the blood flow of the subject 11 based on the duration of the posture. The fatigue estimation system 200 estimates the fatigue level of the subject 11 based on at least one of the estimated values ​​of the load on the muscles, the load on the joints, and the blood flow of the subject 11. Hereinafter, for simplification, the estimated value of the load may be simply expressed as the load or the estimated value. Furthermore, when the estimated value includes the estimated value of the blood flow, the load may be read as the blood flow, and a large load may be replaced with a decrease in the blood flow, and a small load may be replaced with an increase in the blood flow.

[0024] As described above, the blood flow rate is information for quantifying the blood flow that deteriorates when the subject 11 maintains a posture. The lower the blood flow rate, the worse the blood flow of the subject 11 is, and the blood flow rate can be used as an index of fatigue caused by the deterioration of blood flow. The blood flow rate may be acquired as an absolute value at the time of measurement, or may be acquired as a relative value between two different points in time. For example, the degree of deterioration of the blood flow of the subject 11 can be estimated based on the posture of the subject 11 and the relative value of the blood flow rate at two points in time, the start and end points of the posture. In addition, since there is a correlation between the posture of the subject 11, the duration of the posture, and the deterioration of blood flow, the blood flow rate of the subject may be estimated simply from the posture of the subject 11 and the duration of the posture.

[0025] In the following description, the above-mentioned musculoskeletal model is used to estimate at least one of the load on muscles, the load on joints, and the blood flow rate from the posture of the subject 11, but in addition to the above-mentioned musculoskeletal model, a method using actual measurement data can also be applied as a method for estimating the load on muscles, the load on joints, and the blood flow rate from the posture. This actual measurement data is a database constructed by accumulating the actual measurement values ​​of the load on muscles, the load on joints, and the blood flow rate measured for each posture in association with the posture. In this case, the fatigue estimation system 200 can input the estimated posture of the subject 11 into the database, and obtain the actual measurement values ​​of the load on muscles, the load on joints, and the blood flow rate in the corresponding posture as output.

[0026] The actual measurement data may be constructed using actual measurement values ​​for each individual, taking into account individual differences in the subject 11, or the big data obtained from an unspecified number of subjects may be qualified and constructed to suit each subject 11 through analytical processing such as statistical analysis or machine learning.

[0027] Next, a functional configuration of the fatigue estimation system 200 according to the present disclosure will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the fatigue estimation system according to the embodiment.

[0028] As shown in FIG. 2, a fatigue estimation system 200 in the present disclosure includes a fatigue estimation device 100, a plurality of imaging devices 201, a display device 205, and a recovery device 206.

[0029] The estimation device 100 includes an acquisition unit 101 , a classification unit 102 , a posture estimation unit 105 , a first calculation unit 106 , a second calculation unit 107 , a fatigue estimation unit 108 , and an output unit 109 .

[0030] The acquisition unit 101 is a communication module that is connected to each of the multiple imaging devices 201 and acquires an image of the subject 11 from each of the multiple imaging devices 201. The connection between the acquisition unit 101 and the imaging devices 201 is made by wire or wirelessly, and there is no particular limitation on the method of communication made through the connection.

[0031] In addition to the imaging device 201, the fatigue estimation device 100 may include a communication module that is connected to a clock device and acquires the time from the clock device.

[0032] Furthermore, the fatigue estimation device 100 may further include a communication module that is connected to the pressure sensor and acquires the pressure distribution from the pressure sensor.

[0033] Furthermore, the fatigue estimation device 100 may further include a communication module that is connected to a reception device and acquires personal information from the reception device.

[0034] The identification unit 102 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The identification unit 102 is installed to realize an identification function for identifying one subject 11 from other persons. The function of the identification unit 102 will be described in detail later.

[0035] The posture estimation unit 105 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The posture estimation unit 105 estimates the posture of the subject 11 based on the image acquired by the acquisition unit 101 and the additionally acquired pressure distribution, etc., through processing by the posture estimation unit 105. The posture estimation unit 105 is an example of a posture estimation device. That is, the posture estimation device in this embodiment is realized by being built into the fatigue estimation device 100 as the posture estimation unit 105.

[0036] The first calculation unit 106 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The processing of the first calculation unit 106 calculates the load applied to each muscle and / or joint based on the estimated posture of the subject 11 and additionally acquired personal information.

[0037] The second calculation unit 107 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The second calculation unit 107 calculates the amount of recovery from fatigue in each muscle and / or joint based on the amount of change in the estimated posture change of the subject 11 by processing.

[0038] The fatigue estimation unit 108 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The fatigue estimation unit 108 estimates the fatigue level of the subject 11 based on the duration of the estimated posture using the posture estimated by the posture estimation unit 105 and time acquired from a timing device or an internal oscillator.

[0039] The output unit 109 is a communication module that is connected to the display device 205 and the recovery device 206, and outputs content based on the fatigue level estimation result by the fatigue estimation device 100 to the display device 205 and the recovery device 206. The connection between the output unit 109 and the display device 205 or the recovery device 206 is made by wire or wirelessly, and there is no particular limitation on the method of communication made through the connection.

[0040] Furthermore, when posture estimation unit 105 fails to estimate the posture of subject 11, it means that there is subject 11 for whom a fatigue level cannot be estimated, and output unit 109 also has a function of notifying the outside of this fact. That is, output unit 109 outputs the estimation result of posture estimation unit 105 indicating that estimation of the posture of subject 11 has failed.

[0041] At this time, the output unit 109 outputs the estimation result as notification information for notifying the presence of the subject 11 whose posture estimation has failed. Based on this notification information, the administrator of the fatigue estimation system 200 can take measures such as estimating the fatigue level of the subject 11 whose posture estimation has failed using another system or directly asking the administrator. In this way, the output unit 109 is configured in advance so that the fatigue level of the subject 11 whose posture estimation has failed is not overlooked.

[0042] As described above, the imaging device 201 is a device that captures an image of the subject 11 and outputs the image, and is realized by a camera. As the imaging device 201, an existing camera such as a security camera or a fixed camera in the space to which the fatigue estimation system 200 is applied may be used, or a dedicated camera may be newly installed. Such an imaging device 201 is an example of an information output device that outputs an image as information regarding the position of the body part of the subject 11. Therefore, the output information is an image, and is information including the positional relationship of the body part of the subject 11 on the projected imaging element.

[0043] The timing device is a device that measures time, and is realized by a clock. The time measured by the timing device may be an absolute time, or may be a time elapsed from a relative starting point. The timing device may be realized in any form as long as it can measure the time between the time when the subject 11 is detected as still and the time when the fatigue level is estimated (i.e., the duration of the still posture).

[0044] The pressure sensor is a sensor having a detection surface, and measures the pressure applied to each of the unit detection surfaces that divide the detection surface into one or more. The pressure sensor measures the pressure for each unit detection surface in this manner, and outputs the pressure distribution on the detection surface. The pressure sensor is provided so that the subject 11 is positioned on the detection surface.

[0045] For example, the pressure sensor is provided on the seat and backrest of a chair on which the subject 11 sits. Also, for example, the pressure sensor may have a marker on the detection surface, and guide the subject 11 onto the detection surface by displaying, for example, "Please sit on the marker." Also, by guiding the subject 11 onto the detection surface of a pressure sensor provided on a part of the floor in this manner, the pressure sensor may output the pressure distribution of the subject 11 on the floor. Note that since the pressure distribution is used for the purpose of improving the estimation accuracy of the fatigue level, the fatigue estimation system 200 may be realized without a pressure sensor if sufficient accuracy is ensured.

[0046] The reception device is a user interface that receives input of personal information of the subject 11, and is realized by an input device such as a touch panel or a keyboard. The personal information includes at least one of age, sex, height, weight, muscle mass, stress level, body fat percentage, and exercise proficiency. The age of the subject 11 may be a specific numerical value, may be an age band divided every 10 years, such as teens, twenties, and thirties, may be a two-part age band with a predetermined age boundary, such as 59 years or younger or 60 years or older, or may be something else.

[0047] The gender of the subject 11 is selected from either male or female, whichever is appropriate for the subject 11. The height and weight are respectively accepted as the height and weight of the subject 11. The muscle mass is accepted as the muscle composition ratio of the subject 11 measured using a body composition meter or the like. The stress level of the subject 11 is selected by the subject 11 himself from among options such as high, medium, and low as the subjective level of stress felt by the subject 11.

[0048] The body fat percentage of the subject 11 is the ratio of the weight of body fat to the weight of the subject 11, and is expressed, for example, as a percentage.

[0049] Furthermore, the subject 11's exercise proficiency may be quantified by the score when the subject 11 performs a predetermined exercise program, or may be the status of the exercise that the subject 11 usually engages in. In the former case, it is quantified by, for example, the time required to perform 10 back exercises, the time required to run 50 meters, or the distance of a long throw. In the latter case, it is quantified by, for example, how many days a week the subject exercises, or how many hours the subject exercises. Since the personal information is used for the purpose of improving the accuracy of the estimation of the fatigue level, the fatigue estimation system 200 may be realized without a reception device if sufficient accuracy is ensured.

[0050] The display device 205 is a device for displaying the content based on the fatigue level estimation result output by the output unit 109. The display device 205 displays an image showing the content based on the fatigue level estimation result, for example, by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The content displayed by the display device 205 will be described later. Furthermore, when the fatigue estimation system 200 is configured to only reduce the fatigue level of the subject 11 using the recovery device 206, it is sufficient to include only the recovery device 206, and the display device 205 is not essential.

[0051] The recovery device 206 is a device that reduces the fatigue level of the subject 11 by promoting blood circulation of the subject 11. Specifically, the recovery device 206 actively changes the posture of the subject 11 who is seated by applying voltage, applying pressure, vibrating, heating, or the like, or by changing the arrangement of each part of the chair 12 by a mechanism provided in the chair 12. In this way, the recovery device 206 changes the state of load on at least one of the muscles and joints of the subject 11, and also promotes blood circulation. In terms of blood flow volume, the promotion of blood circulation in this way reduces the effect of blood flow deterioration caused by the subject 11 being in a stationary posture, and the fatigue level is restored. The recovery device 206 is attached to or contacted in advance to an appropriate body part of the subject 11 according to the configuration of the device.

[0052] When promoting blood circulation in the subject 11 by heating, the entire space around the subject 11 is heated, so in such a case, it is not necessary to attach the device to or make contact with an appropriate body part of the subject 11. Furthermore, when the fatigue estimation system 200 is configured only to display the fatigue level estimation result to the subject 11, it is sufficient to include only the display device 205, and the recovery device 206 is not essential.

[0053] [Operation] Next, estimation of a fatigue level of the subject 11 using the fatigue estimation system 200 in the embodiment will be described with reference to Fig. 3 to Fig. 6. Fig. 3 is a flowchart showing a method of estimating a fatigue level according to the embodiment.

[0054] The fatigue estimation system 200 first acquires personal information of the subject 11. The personal information is acquired by the subject 11 himself / herself or an administrator who manages the fatigue level of the subject 11 by inputting the information into a reception device. The input personal information of the subject 11 is stored in a storage device or the like (not shown) and is read out and used when estimating the fatigue level.

[0055] The fatigue estimation system 200 detects the subject 11 using the imaging device 201. The subject 11 is detected by counting the number of people within the angle of view of the camera, which is the imaging device 201, so that all people within the angle of view are considered to be the subject 11. That is, here, an operation of acquiring the number of people to be detected (A) is performed (S101). Next, the number of people (B) whose postures can be estimated from only one image without processing the image is calculated (S102). Then, the fatigue estimation device 100 judges whether A and B match (S103). If the fatigue estimation device 100 judges that A and B match (Yes in S103), that is, if the postures of all the people to be detected can be estimated from one image, the fatigue estimation device 100 estimates the postures of the subject 11 (here, estimates the joint positions) from each image (S108). Then, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 (S109).

[0056] Steps S108 and S109 are performed as follows. First, the fatigue estimation system 200 acquires an image output by the imaging device 201 using the acquisition unit 101. If it is detected in the acquired image that the subject 11 is stationary (in a stationary posture), the estimation device 100 estimates the posture of the subject 11. Specifically, first, the pressure distribution applied to the detection surface is acquired from the pressure sensor.

[0057] The posture estimation unit 105 estimates the posture of the subject 11 based on the acquired image and pressure distribution. For example, when biased pressure is applied, the pressure distribution is used to correct the estimated posture so that the bias is formed. Next, the first calculation unit 106 calculates the load amount in each muscle and / or joint of the subject 11 from the posture estimation result. At this time, the load amount is corrected and calculated using personal information acquired in advance. Note that the estimation of the posture of the subject 11 is as described using FIG. 1B, and the calculation of the load amount is as described using FIG. 1C, so a detailed description will be omitted.

[0058] In correcting the load using personal information, for example, the load is decreased as the age of subject 11 approaches the peak age of muscle development, and increased as the age of subject 11 moves away from the peak age. Such peak value may be based on the sex of subject 11. Also, the load may be decreased if subject 11 is male, and increased if subject 11 is female. Also, the load may be decreased as subject 11's height and weight are smaller, and increased as subject 11's height and weight are larger.

[0059] The load may be decreased as the muscle mass of subject 11 increases in composition ratio, and increased as the muscle mass of subject 11 decreases in composition ratio. The load may be decreased as the stress level of subject 11 decreases, and increased as the stress level of subject 11 increases. The load may be increased as the body fat percentage of subject 11 increases, and decreased as the body fat percentage of subject 11 decreases. The load may be decreased as subject 11 becomes more familiar with exercise, and increased as subject 11 becomes less familiar with exercise.

[0060] Here, the duration of the stationary posture of the subject 11 is measured based on the time acquired from the timing device. The fatigue estimation unit 108 adds the load calculated above every unit time of the duration, and estimates the fatigue level of the subject 11 at this point in time. These processes are continued until the stationary state of the subject 11 is released. Specifically, whether the stationary state has been released is determined based on whether the posture estimated by the posture estimation unit 105 has been changed from a certain stationary posture.

[0061] If it is not determined that the stationary state has been released, the fatigue level of subject 11 is accumulated as long as the stationary posture continues by measuring the duration and adding the load. That is, fatigue estimation unit 108 estimates the fatigue level of subject 11 using an increasing function of fatigue level having a slope corresponding to the calculated load with respect to the duration. Therefore, the greater the calculated load, the greater the increase in fatigue level of subject 11 per unit time. In this accumulation of fatigue level, the fatigue level of subject 11 is initialized (set to fatigue level 0) at the start timing of the stationary posture, which is the starting point.

[0062] On the other hand, when it is determined that the stationary state has been released, the posture estimation unit 105 calculates the amount of change in posture from the original stationary state posture to the changed current posture. The amount of change in posture is calculated for each muscle and / or joint, similar to the load amount described above. When the posture changes in this way, the load on at least one of the muscles and joints changes, and in terms of blood flow, the worsened blood flow is temporarily alleviated, and the fatigue level of the subject 11 starts to recover. The fatigue level reduced by the recovery is related to the amount of change in posture. Accordingly, the second calculation unit 107 calculates the amount of recovery, which is the degree of recovery of the fatigue level, based on the amount of change in posture.

[0063] Based on the time acquired from the timing device, a change time, which is the time during which the change in posture of the subject 11 continues, is measured. The relationship between the recovery amount and the change time is similar to the relationship between the load amount and the duration, and the recovery amount of the subject 11 is accumulated as long as the change in posture continues. In other words, at the timing when the posture of the subject 11 changes in this way, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 by subtracting the recovery amount every time a unit time elapses.

[0064] The calculation of the recovery amount, the measurement of the change time, and the estimation of the fatigue level of the subject 11 are continued until the posture of the subject 11 becomes stationary. Specifically, it is determined whether or not the posture estimated by the posture estimation unit 105 is a certain stationary posture. If the stationary posture of the subject 11 is not detected, the recovery amount is calculated, the change time is measured, and the recovery amount is subtracted, thereby accumulating so that the fatigue level of the subject 11 is recovered as long as the posture change continues.

[0065] That is, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 using a decreasing function of the fatigue level having a slope corresponding to the calculated recovery amount with respect to the change time. Since the recovery amount of the fatigue level depends on the change amount of the posture, the greater the change amount of the posture, the greater the decrease in the fatigue level of the subject 11 per unit time.

[0066] On the other hand, when the stillness of the subject 11 is detected, the posture and fatigue level are estimated again for the new still posture. In this way, the fatigue estimation system 200 calculates the fatigue level of the subject 11 based on the image and taking into account the duration of the still posture, so that the fatigue level of the subject 11 can be estimated with less burden on the subject 11 and with higher accuracy.

[0067] The above will be described in more detail with reference to Fig. 4A to Fig. 5B. Fig. 4A is a diagram showing a subject standing still in posture A. Fig. 4B is a diagram showing a subject standing still in posture B.

[0068] The subject 11 shown in Figures 4A and 4B is in a static posture in a seated position on a chair 12, similar to that shown in Figure 1A. Note that in Figures 4A and 4B, a table, PC, etc., which are not shown, are actually present, but only the subject 11 and the chair 12 are shown here. The static posture of the subject 11 shown in Figure 4A is posture A, which places a relatively heavy load on the shoulders. On the other hand, the static posture of the subject 11 shown in Figure 4B is posture B, which places a relatively light load on the shoulders.

[0069] The fatigue level estimated for subject 11 standing still in posture A or posture B accumulates as shown in Fig. 5A and Fig. 5B. Fig. 5A is a first diagram for explaining the accumulation of the estimated fatigue level of the subject according to the embodiment. Fig. 5B is a second diagram for explaining the accumulation of the estimated fatigue level of the subject according to the embodiment.

[0070] As shown in FIG. 5A, when subject 11 remains stationary in posture A shown in FIG. 4A or posture B shown in FIG. 4B, the fatigue level of subject 11 is expressed by a linear function whose slope is the amount of load calculated from the posture.

[0071] As described above, posture A is a posture that imposes a greater load than posture B. Therefore, for example, in a certain muscle of subject 11 (here, a muscle related to shoulder movement), the load amount in posture A (the slope of the straight line in posture A) is greater than the load amount in posture B (the slope of the straight line in posture B). For this reason, subject 11 accumulates (accumulates) a greater degree of fatigue in a shorter period of time in posture A compared to when subject 11 is stationary in posture B.

[0072] On the other hand, as shown in FIG. 5B, when the posture of subject 11 changes from posture A shown in FIG. 4A to posture B shown in FIG. 4B, the fatigue level of subject 11 is expressed by a function that combines a linear function whose slope is the amount of load calculated from the posture and a linear function whose slope is the amount of change in posture.

[0073] Therefore, for example, while a certain muscle of subject 11 remains stationary in posture A, the fatigue level of subject 11 is estimated as an accumulation (addition) of fatigue level by an increasing function with a positive slope corresponding to the load of posture A, as in FIG. 5A, and the accumulation (addition) turns to recovery (decrease) at the change point where subject 11 starts to change his posture. The fatigue level of subject 11 is recovered (decreased) by an amount shown as a change width in the figure during the period during which the change in posture continues, shown as the change time in the figure, by a decreasing function with a negative slope corresponding to the amount of change in posture. After the change point where subject 11 remains stationary again in posture B, the fatigue level of subject 11 is estimated as an accumulation (addition) of fatigue level by an increasing function with a positive slope corresponding to the load of posture B.

[0074] In this manner, the fatigue estimation system 200 in this embodiment estimates the fatigue level of the subject 11 that reflects accumulation and recovery in response to the subject 11's stillness and change in posture.

[0075] Returning to the explanation of the flowchart in FIG. 3, in step S103, if the fatigue estimation device 100 determines that A and B do not match (No in S103), that is, if the persons to be detected include a person whose posture cannot be estimated from one image, it identifies such a person (S104). Then, the posture estimation unit 105 selects and acquires a plurality of images that can be used to estimate the posture of the identified person (hereinafter, the subject 11 whose fatigue level is estimated through synthesis of joint positions) (S105). Here, if the posture of the subject 11 cannot be estimated even if the plurality of images are used (No in S106), the process of estimating the fatigue level of the subject 11 is terminated. At this time, for example, the notification information described as a function of the output unit 109 is output.

[0076] For the subject 11 whose posture can be estimated by using a plurality of images (Yes in S106), the posture of the subject 11 is estimated as shown in the following FIG. 6. FIG. 6 is a diagram for explaining an example of synthesis of joint positions according to an embodiment. FIG. 6 shows information generated while the posture (right end) of the subject 11 is estimated based on a plurality of acquired images (left end).

[0077] In addition, in FIG. 6, the image processing system is divided into upper and lower stages from the left side to the center, and one image 90a of the multiple images and a joint position 11c of the one body part estimated from the one image 90a are shown on the upper stage. Similarly, in FIG. 6, another image 90b of the multiple images and a joint position 11d of the other body part estimated from the other image 90b are shown on the lower stage. First, in step S105, the posture estimation unit 105 acquires multiple images (here, the one image 90a and the other image 90b). The one image 90a and the other image 90b are images of the one body part that is a part of the body part of the subject 11 and the other body part, respectively.

[0078] In addition, one body part and another body part are body parts that share at least one joint with each other. Therefore, if a combination of a plurality of images that share at least one joint with each other cannot be identified, the result in step S106 above becomes No and the process ends.

[0079] As shown in FIG. 3, the posture estimation unit 105 estimates the joint positions of the body parts of the subject 11 that are partially reflected in each of the multiple images (S107). In the example of FIG. 6, based on one image 90a, a joint position 11c of one body part reflected in one image 90a is estimated, and first information 90c including the joint position 11c is generated. Similarly, in the example of FIG. 6, based on another image 90b, a joint position 11d of another body part reflected in the other image 90b is estimated, and second information 90d including the joint position 11d is generated. Thereafter, the joint position 11c of the one body part and the joint position 11d of the other body part are synthesized to estimate the joint position 11a of the body part of the subject 11 including the one body part and the other body part. In this way, the posture estimation unit 105 can estimate the posture of the subject 11 using multiple images in which some body parts of the subject 11 are reflected.

[0080] Here, the synthesis of joint positions will be described with continued reference to FIG. 6. Synthesis of joint positions of the subject 11 requires at least two or more common coordinates between a joint position 11c in one body part and a joint position 11d in another body part. One of these common coordinates is the joint position of a shared joint, which is one of the joints shared between the one body part and the other body part. Then, by using a reference coordinate shared within the above-mentioned multiple images in addition to the joint position of the shared joint, it is possible to match the scales of the multiple images and the postures (orientations) of the body parts within the images.

[0081] As the reference coordinate, the joint position of a joint other than the shared joint among the joints shared between the one body part and the other body part is used. In this example, in a situation where two or more joints are shared between the one body part and the other body part, the posture estimation unit 105 adjusts the joint position 11c in the one body part and the joint position 11d in the other body part so that the direction and magnitude of the vector extending from one joint to the other joint between these joints match. Then, in a state where the posture (direction) and scale are matched, the posture estimation unit 105 estimates the joint position 11a of the body part of the subject 11 by superimposing and synthesizing the joint position 11c in the one body part and the joint position 11d in the other body part with the shared joint as a reference.

[0082] In addition to the above, a marker appearing in both the first image 90a and the other image 90b may be used as the reference coordinate. In the example of FIG. 6, an object marker M appearing in both the first image 90a and the other image 90b is shown. The object marker M may be a dedicated object, or any object that exists in space and can be identified as the same object by the multiple image capture devices 201 may be used. If the spatial coordinates of this object marker M are used as the reference coordinates, the poses (directions) and scales can be matched and the markers can be superimposed and synthesized with the shared joint as a reference, as in the above example. For this purpose, the pose estimation unit 105 may calculate the spatial coordinates Mc of the object marker M from the first image 90a, and calculate the spatial coordinates Md of the object marker M from the other image 90b.

[0083] As another example of the marker, a line marker L may be used. For example, the line marker L is realized by attaching a tape or the like to a location in space. The line marker L is configured so that both ends are in common positions and extend between these ends. Therefore, in an example in which the line marker L is present, the same effect as above can be obtained even if there is no shared joint in the body part of the subject 11.

[0084] Further, by utilizing the existence of these markers in real space, further effects can be obtained. That is, in the fatigue estimation system 200, the subject 11 can be identified from other people who are staying in the same space as the subject 11, based on the positional relationship between the markers and the subject 11. In this embodiment, a process such as superimposing the joint positions of each body part of the subject 11 from a plurality of images is performed. Therefore, when another person (for example, another person 11z in FIG. 6) is staying in the same space as the subject 11, if the joint position of one body part of the subject 11 is superimposed on the joint position of another body part of the other person, the posture of the subject 11 cannot be estimated correctly.

[0085] Therefore, the fatigue estimation system 200 includes an identification unit 102, which allows the subject 11 to be identified and allows some joint positions of the subject 11 to be reliably superimposed on other joint positions of the subject 11. For this purpose, the identification unit 102 identifies the relative positions of the subject 11 and the markers in an image acquired from the imaging device 201 (that is, the subject 11 is at a position a certain distance away in a certain direction relative to the marker, and another person 11z is at a position a different distance away in a different direction relative to the marker). This process requires only identification with a spatial resolution of one person, and therefore requires less processing load and can be easily implemented compared to the process of estimating the positions of each joint.

[0086] As described above, in this embodiment, even if an image shows only a portion of the body part of the subject 11, it is possible to estimate an appropriate posture that can be used to estimate the fatigue level by combining and complementing multiple images.

[0087] Furthermore, when the joint positions of the body parts including a part of the body part of the subject 11 and another part of the body part are estimated as the posture of the subject 11 by the above, it may not be possible to estimate the posture of other parts that are not included in the part and the other part of the body part of the subject 11. For example, the posture of an invisible part that is not shown in any of the multiple images, i.e., that is not captured in any of the images, cannot be estimated by the above processing.

[0088] In this case, for example, the joint positions of a part of the body parts of the subject 11 estimated as described above and another part may be input to a machine learning model in which the joint positions of the whole body of a person are learned, so that the joint positions of the whole body may be estimated as the posture of the whole body of the subject 11. In addition, even if it is not the whole body, the posture of the subject 11 may be estimated by adding only a part of the invisible parts that has sufficient reliability as the output of the machine learning model. In this way, the posture estimation unit 105 may be configured to estimate the joint positions of the body parts including the invisible parts that are not captured in the multiple images among the body parts of the subject 11 as the posture of the subject 11.

[0089] 3 again, in this manner, posture estimation unit 105 uses a plurality of images showing some body parts of subject 11 to perform processing such as synthesis to estimate the posture of subject 11 (S108). Then, proceeding to step S109, fatigue estimation unit 108 estimates the fatigue level of subject 11 as already described.

[0090] Next, an example of the output of the output unit 109 based on the estimated fatigue level will be described. Fig. 7 is a diagram showing an example of display of the estimation result according to the embodiment.

[0091] As shown in FIG. 7, in the fatigue estimation system 200, the estimation result of the fatigue level of the subject 11 can be displayed and fed back using a display device 205. Specifically, as shown in FIG. 7, by visualizing the fatigue level of the subject 11, it is possible to visually grasp how tired the subject 11 is. In the figure, a doll imitating the subject 11 and the fatigue levels of the subject 11's shoulders, back, and lower back are displayed together by the display device 205. To make it easier for the subject 11 to intuitively grasp the fatigue level, the fatigue level of the shoulders is displayed as a "stiff shoulder level," the fatigue level of the back is displayed as a "back pain level," and the fatigue level of the lower back is displayed as a "lower back pain level."

[0092] Here, in the display in the figure, the fatigue levels of three parts of the subject 11 are displayed all at once, but the fatigue levels of these three parts are estimated from an image captured at one time. That is, the estimation device 100 estimates the fatigue levels of the muscles and / or joints in each of a plurality of body parts including a first part (e.g., shoulders), a second part (e.g., back), and a third part (e.g., waist) of the subject 11 from one posture of the subject 11. Therefore, even if the posture of the subject 11 is constant, the fatigue levels accumulated in the muscles and / or joints differ for each body part, but the fatigue estimation system 200 can estimate such different fatigue levels simultaneously and individually.

[0093] 1C, in this embodiment, since the load is calculated for each muscle and / or joint of subject 11, if there is no limit to processing resources, the fatigue level of each muscle and / or joint can be estimated. Therefore, there is no limit to the number of body parts whose fatigue level is estimated from an image captured at one time, and it may be one, two, four or more.

[0094] The estimation device 100 calculates the load amount for each of multiple body parts, and in one posture of the subject 11, it is possible to estimate the fatigue level of a first part (the above-mentioned shoulder stiffness level) based on the load amount calculated for the first part, the fatigue level of a second part (the above-mentioned back pain level) based on the load amount calculated for the second part, and the fatigue level of a third part (the above-mentioned lower back pain level) based on the load amount calculated for the third part.

[0095] In the example shown in the figure, the degree of stiff shoulders is estimated from the load on the trapezius muscle, the degree of back pain is estimated from the fatigue of the latissimus dorsi muscle, and the degree of lower back pain is estimated from the load on the lumbar paraspinal muscles. In this way, one fatigue level may be estimated from the load on one muscle and / or joint, but one fatigue level may also be estimated from the combined load on multiple muscles and / or joints. For example, the degree of stiff shoulders (i.e., one fatigue level of the shoulder) may be estimated from the average value of the loads on the trapezius muscle, levator scapulae muscle, rhomboid muscle, and deltoid muscle. In addition, in estimating the fatigue level, a more realistic estimation of the fatigue level may be performed by weighting the load on the muscle and / or joint that has a particularly large effect on the fatigue level of the body part in accordance with the magnitude of the effect, rather than using a simple average value.

[0096] The fatigue level estimated in this manner may be shown as a relative position on a reference meter with a minimum value of 0 and a maximum value of 100, as shown in the figure. Here, a reference value is set at a predetermined position on the reference meter. Such a reference value is set to a relative position (or around it, etc.) of a fatigue level that is quantified in advance by an epidemiological survey or the like and that may cause subjective symptoms such as pain in a typical subject 11. Therefore, the reference value may be set to a different value depending on the fatigue level of each body part.

[0097] Furthermore, when the estimated fatigue level of the subject 11 reaches a reference value, the display device 205 may display a warning to the subject 11 as an estimated result. The reference value here is an example of a first threshold value. In the figure, as an example of such a warning, "The degree of stiff shoulders exceeds the reference value" is displayed at the bottom of the display device 205. In addition, in relation to such a warning, the display device 205 may display a specific method of dealing with the problem, such as "We recommend taking a break" as also shown in the figure.

[0098] In addition to the above-mentioned configuration in which the estimation result is displayed to the subject 11 to encourage the subject 11 to deal with the accumulated fatigue level, a configuration in which the fatigue estimation system 200 actively recovers the fatigue level of the subject 11 may be considered. Specifically, the fatigue level of the subject 11 is recovered by the operation of the recovery device 206 shown in FIG. 2. The specific configuration of the recovery device 206 is as described above, so an explanation is omitted. When the estimated fatigue level of the subject 11 reaches a reference value, the recovery device 206 operates to change the load on at least one of the muscles and joints of the subject 11 and promote blood circulation to reduce the fatigue level of the subject. The reference value here is an example of a third threshold value, and may be the same as or different from either the first threshold value or the second threshold value.

[0099] [Effects, etc.] As described above, the fatigue estimation system 200 in this embodiment comprises a plurality of imaging devices 201, each of which outputs images capturing different parts of the body of the subject 11, a posture estimation device (posture estimation unit 105) that estimates the posture of the subject 11 based on the multiple images output by each of the multiple imaging devices 201, and a fatigue estimation device 100 that estimates and outputs the fatigue level of the subject 11 based on the estimation result of the posture of the subject 11.

[0100] In such a fatigue estimation system 200, in a situation where an image showing the whole body of the subject 11 in one image cannot be obtained due to conditions such as the placement position of the imaging device 201, making it difficult to estimate the posture of the subject 11 from one image, the posture estimation unit 105 can appropriately estimate the posture of the subject 11 by combining and using another image. Then, based on the posture appropriately estimated by the posture estimation unit 105, it becomes possible to more appropriately estimate the fatigue level of the subject 11.

[0101] Also, for example, the posture estimation unit 105 may estimate a joint position 11c in one body part based on one image 90a in which one body part is captured among the multiple images, and estimate a joint position 11d in another body part based on another image 90b in which another body part that shares at least one joint with the one body part is captured among the multiple images, and may combine the estimated joint position 11c in the one body part and the estimated joint position 11d in the other body part to estimate the joint position 11a of the body part including the one body part and the other body part as the posture of the subject 11.

[0102] According to this, it is possible to estimate the joint position 11a of the body part by using one image 90a in which one body part is captured among the multiple images, and another image 90b in which another body part is captured. Here, it is possible to estimate the joint position 11a of the body part including the one body part and the other body part by estimating the joint position 11c of the one body part from the one image 90a, estimating the joint position 11d of the other body part from the other image 90b, and combining these. In this way, it is possible to estimate the joint position 11a including all of the one body part not included in the one image 90a, the other body part, and the one body part not included in the other image 90b.

[0103] Furthermore, for example, in synthesizing an estimated joint position 11c in one body part and an estimated joint position 11d in another body part, at least one of a relative posture and a relative scale between the joint position 11c in one body part and the joint position 11d in the other body part may be determined based on the relative position of a shared joint, which is a joint shared between one body part and the other body part, and a reference coordinate shared within a plurality of images, and the shared joint and the reference coordinate may be superimposed in a state in which at least one of the determined relative posture and relative scale between the joint position 11c in one body part and the joint position 11d in the other body part is applied.

[0104] According to this, by superimposing the relative positions of the shared joint and the reference coordinates, it is possible to synthesize the joint position 11c in one body part from one image 90a, and the joint position 11d in another body part from another image 90b. At this time, depending on the arrangement position of the imaging device 201, a situation may occur in which at least one of the posture and the scale of the joint position 11c in one body part and the joint position 11d in the other body part is different between the two images. In the above aspect, the above synthesis can be performed after adjusting the posture and the scale, etc., which are different between the two images, by applying the relative posture (e.g., the amount of rotation for eliminating the difference in the relative orientation) and the relative scale (e.g., the ratio of enlargement / reduction for eliminating the difference in the relative scale) between the joint position 11c in one body part and the joint position 11d in the other body part.

[0105] Also, for example, the reference coordinate may be the joint position of a joint other than the shared joint.

[0106] According to this, by using the joint position of a joint other than the shared joint as a reference coordinate and superimposing the relative position of the shared joint and the reference coordinate, it is possible to synthesize a joint position 11c of one body part from one image 90a, and a joint position 11d of another body part from another image 90b.

[0107] Furthermore, for example, the reference coordinates may be the spatial coordinates of a marker placed at a position that is captured in all of the multiple images.

[0108] According to this, by using the spatial coordinates of a marker placed at a position that is captured in all of the multiple images as reference coordinates and superimposing the relative positions of the shared joint and the reference coordinates, it is possible to synthesize a joint position 11c in one body part from one image 90a, and a joint position 11d in another body part from another image 90b.

[0109] For example, a discrimination unit 102 may further be provided that discriminates the target person 11 from other people 11z staying in the same space as the target person 11 based on the positional relationship between the marker and the target person 11.

[0110] According to this, based on the positional relationship between the marker and the subject 11, the subject 11 can be distinguished from other persons 11z staying in the same space as the subject 11, and the posture of only the subject 11 can be estimated. As a result, even if other persons 11z are present, the fatigue level of only the subject 11 can be estimated.

[0111] Also, for example, when the posture estimation unit 105 fails to estimate the posture of the subject 11, it outputs an estimation result indicating that estimation of the posture of the subject 11 has failed, and the fatigue estimation system 200 may further include an output unit 109 that outputs notification information to notify the presence of a subject 11 whose posture estimation has failed, based on the estimation result indicating that estimation of the posture of the subject 11 has failed.

[0112] According to this, it is possible to know the presence of a target person 11 whose posture estimation has failed based on the notified notification information.

[0113] Also, for example, the posture estimation unit 105 may estimate joint positions of some body parts of the subject 11 based on multiple images, input the estimated joint positions of some body parts of the subject 11 into a machine learning model in which the joint positions of a human's entire body have been learned, and estimate the joint positions 11a of body parts of the subject 11, including invisible parts that are not captured in the multiple images, as the posture of the subject 11.

[0114] According to this, by using a machine learning model that has learned the joint positions of the entire human body, it is possible to estimate joint positions 11a for body parts, including invisible parts of the subject 11, that cannot be estimated using any combination of multiple images.

[0115] Furthermore, for example, the posture estimation unit 105 may further obtain the number of detection targets within the area, and calculate the number of detection targets whose posture can be estimated from only one image output from one of the imaging devices 201 that captures the image within the area, and when the number of detection targets and the number of posture-estimateable targets differ, the posture estimation unit 105 may estimate the posture of the detection targets 11 by treating the detection targets whose posture is not estimable as the target 11.

[0116] According to this, only when the number of detection subjects and the number of posture-estimable subjects in an area are different, that is, only when the detection subjects include a subject 11 whose posture cannot be estimated without combining multiple images, can the posture estimation described above be performed using multiple images. This makes it possible to reduce the processing resources for posture estimation, and realize the fatigue estimation system 200 with a simple configuration.

[0117] Moreover, the posture estimation device in this embodiment is posture estimation section 105 described above.

[0118] According to this, in a situation where an image showing the entire body of the subject 11 cannot be obtained in a single image due to conditions such as the placement position of the imaging device 201, and it is difficult to estimate the posture of the subject 11 from a single image, the posture of the subject 11 can be appropriately estimated by combining and using another image.

[0119] In addition, the fatigue estimation method in this embodiment is a fatigue estimation method executed by the fatigue estimation device 100, which acquires images from each of a plurality of imaging devices 201, each of which outputs images capturing a different part of a body part of the subject 11, and estimates a joint position 11a of a body part including the one body part and the other body part as the posture of the subject 11 based on one image 90a of the plurality of images capturing one body part and another image 90b of the plurality of images capturing another body part that shares at least one joint with the one body part, and estimates a fatigue level of the subject 11 based on the estimated posture of the subject 11 and outputs the result to an output device connected to the fatigue estimation device 100.

[0120] This makes it possible to achieve the same effects as the fatigue estimation system 200 described above.

[0121] Moreover, the program in this embodiment is a program for causing a computer to execute the fatigue estimation method described above.

[0122] According to this, it is possible to achieve the same effects as those of the fatigue estimation system 200 described above using a computer.

[0123] (Other embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0124] For example, in the above embodiment, the process executed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0125] Furthermore, the fatigue estimation system or posture estimation device of the present disclosure may be realized by multiple devices each having some of the multiple components, or may be realized by a single device having all of the multiple components. Also, some of the functions of a component may be realized as the functions of another component, and each function may be distributed in any manner among the components. As long as the configuration has all the functions that can substantially realize the fatigue estimation system or posture estimation device of the present disclosure, it is included in the present disclosure.

[0126] In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0127] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or an integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit, or a dedicated circuit.

[0128] In addition, the general or specific aspects of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0129] In the above embodiment, the posture of the subject is estimated from an image using a rigid link model generated by image recognition, the load is calculated from the estimated posture of the subject, and the fatigue level of the subject is estimated based on the load and duration, but the method of estimating the fatigue level is not limited to this. Any existing method may be used as a method of estimating the posture of the subject from an image, and any existing method may be used as a method of estimating the load from the posture of the subject.

[0130] In the above embodiment, the increasing function and the decreasing function are described as linear first-order functions, but are not limited thereto. The increasing function may be a curved function as long as the fatigue level increases over time. The decreasing function may be a curved function as long as the fatigue level decreases over time.

[0131] In addition, the above-described estimation device estimates the subject's fatigue level using estimated values ​​of the load on the muscles, the load on the joints, and the blood flow rate estimated from the subject's posture, but it is also possible to achieve a more accurate estimation of the fatigue level by correcting the estimated values ​​using values ​​measured using a measuring device. Specifically, the estimation device obtains measurement values ​​that correspond to the estimated values ​​and are based on the measurement results of measuring the subject using a measuring device.

[0132] The detection device may be, for example, an electromyograph, a muscle hardness meter, a pressure meter, or a near-infrared spectrometer, and may obtain measurement values ​​related to the load on a muscle, the load on a joint, and the blood flow rate by measurement. For example, an electromyograph may estimate the muscle movement corresponding to the potential measured by potential measurement. That is, the estimated muscle movement may be obtained as a measurement value. The estimated muscle movement may be converted into the load on a muscle, and therefore the estimated muscle load may be corrected by the measurement value. The correction may be, for example, taking the average value of the estimated value and the measurement value, selecting either the estimated value or the measurement value, or substituting the estimated value into the correlation function between the estimated value and the measurement value.

[0133] The muscle hardness meter can estimate muscle hardness based on the stress when pressure is applied to the muscle. The estimated muscle hardness can be converted into the amount of load on the muscle, so it can be used to correct the estimated value in the same way as above.

[0134] The pressure gauge can obtain measurements of pressure applied to the subject's body parts. Such pressure parameters can be input to the musculoskeletal model. Inputting additional parameters such as pressure improves the estimation accuracy of the musculoskeletal model, and the estimated values ​​estimated using the musculoskeletal model can be corrected with higher accuracy.

[0135] The near-infrared spectrometer can obtain a measurement value obtained by spectroscopically measuring the blood flow rate of the subject. As in the above embodiment, when the estimated value does not include the blood flow rate, the estimated value may be corrected by combining the blood flow rate measured by the infrared spectrometer. Even when the estimated value includes the blood flow rate, the measured blood flow rate may be used when the reliability of the estimated value of the blood flow rate is low.

[0136] In this way, by using measurement values ​​corresponding to estimates obtained from a different perspective and making corrections to improve the accuracy of the estimates, it is possible to more accurately estimate the subject's level of fatigue.

[0137] Furthermore, a fatigue factor identification system for identifying the factor of fatigue of a subject may be configured using the fatigue estimation system described in the above embodiment. Conventionally, with devices or systems that estimate the degree of fatigue as "shoulder stiffness level" and "lower back pain level", it has been difficult to identify the way of using muscles and joints that are the factors of such "shoulder stiffness level" and "lower back pain level" (i.e., the posture that is the cause). Therefore, by using the fatigue estimation system in the present disclosure, the above problem can be addressed.

[0138] That is, in the fatigue factor identification system of the present disclosure, a body part where fatigue is likely to accumulate in a static posture taken by a subject (a body part with a large estimated amount of promoting various types of fatigue) is identified as a fatigue factor part. Furthermore, the fatigue factor identification system may simply identify a fatigue factor part in one static posture taken by the subject, or may identify a fatigue factor posture with the largest estimated amount in the fatigue factor part from among multiple static postures taken by the subject. In addition, a recommended posture to replace the identified fatigue factor posture may be presented, or a fatigue recovery operation using a recovery device may be performed on the fatigue factor part in the fatigue factor posture.

[0139] The fatigue factor identification system includes the fatigue estimation system described in the above embodiment and a storage device for storing information related to the estimated fatigue level. Such a storage device is realized using, for example, a semiconductor memory, and may be one of the main storage units constituting the fatigue estimation system, or a new storage device may be provided that is communicatively connected to the estimation device.

[0140] The present disclosure may also be realized as a fatigue estimation method executed by a fatigue estimation system or an estimation device. The present disclosure may also be realized as a program for causing a computer to execute such a fatigue estimation method, or as a computer-readable non-transitory recording medium having such a program recorded thereon.

[0141] In addition, the present disclosure also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art may conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the spirit of the present disclosure. [Explanation of symbols]

[0142] 11. Target Audience 11a Joint position 11c Joint position in one body part 11d. Joint positions in other body parts 11z Other people 90a First image 90b Other images 100 Fatigue Estimation Device 101 Acquisition Department 102 Identification unit 105 Posture estimation unit (posture estimation device) 109 Output section 200 Fatigue Estimation System 201 Imaging device Mc, Md spatial coordinates

Claims

1. A plurality of image capturing devices each outputting an image capturing a different part of a body part of a subject; a posture estimation device that estimates a posture of the subject based on a plurality of the images output from each of the plurality of imaging devices; a fatigue estimation device that estimates and outputs a fatigue level of the subject based on an estimation result of the posture of the subject, The plurality of images include an image in which one body part is captured, and in which a first part that is a part of another body part sharing at least one joint with the one body part is not captured, and another image in which the other body part is captured, and in which a second part that is a part of the one body part is not captured, The posture estimation device includes: estimating a first joint position in the one body part including a joint of the second part based on the one image of the plurality of images; estimating a second joint position in the other body part including a joint of the first part based on the other image of the plurality of images; combining the estimated first joint position and the estimated second joint position to estimate joint positions of body parts including the one body part and the other body part as a posture of the subject; In the synthesis of the estimated first joint position and the estimated second joint position, determining at least one of a relative orientation and a relative scale between the first joint position and the second joint position based on a relative position between a shared joint, which is a joint shared between the one body part and the other body part, and a reference coordinate, which is a joint position of a joint different from the shared joint, which is a reference coordinate shared among a plurality of the images; By superimposing the shared joint and the reference coordinates in a state in which at least one of a relative posture and a relative scale between the determined first joint position and the second joint position is applied, the joint of the second part and the joint of the first part are made to complement each other. Fatigue estimation system.

2. when the posture estimation device fails to estimate the posture of the subject, outputting the estimation result indicating that estimation of the posture of the subject has failed; The fatigue estimation system further includes an output unit that outputs notification information for notifying the presence of the subject whose posture estimation has failed based on the estimation result indicating that estimation of the posture of the subject has failed. The fatigue estimation system according to claim 1 .

3. The posture estimation device includes: Estimating joint positions in a body part of the subject based on the plurality of images; The estimated joint positions of the part of the body of the subject are input into a machine learning model that has learned the joint positions of the entire human body; The joint positions of the body parts of the subject, including invisible parts that are not captured in the plurality of images, are estimated as the posture of the subject. The fatigue estimation system according to claim 1 .

4. The posture estimation device further comprises: Obtain the number of people to be detected in the area, Calculating the number of posture-estimable persons whose postures can be estimated from only one image output from one of the imaging devices that captures images within the area, among the detection targets; When the number of the detection targets is different from the number of the posture-estimateable persons, the detection targets are those whose postures are not estimable, and the postures of the targets are estimated. The fatigue estimation system according to claim 1 .

5. The posture estimation device according to claim 1 .

6. A fatigue estimation method executed by a fatigue estimation device, comprising: acquiring images from a plurality of image capture devices each outputting an image of a different part of a body part of a subject; based on one image of the plurality of images in which one body part is captured and which does not include a first part that is a part of another body part sharing at least one joint with the one body part, and another image of the plurality of images in which the other body part is captured and which does not include a second part that is a part of the one body part, joint positions of the body parts including the one body part and the other body part are estimated as a posture of the subject, A fatigue level of the subject is estimated based on the result of estimating the posture of the subject, and the fatigue level is output to an output device connected to the fatigue estimation device. In the estimation of the posture of the subject, estimating a first joint position in the one body part including a joint of the second part based on the one image of the plurality of images; estimating a second joint position in the other body part including a joint of the first part based on the other image among the plurality of images; combining the estimated first joint position and the estimated second joint position to estimate joint positions of body parts including the one body part and the other body part as a posture of the subject; In the synthesis of the estimated first joint position and the estimated second joint position, determining at least one of a relative orientation and a relative scale between the first joint position and the second joint position based on a relative position between a shared joint, which is a joint shared between the one body part and the other body part, and a reference coordinate, which is a joint position of a joint different from the shared joint, which is a reference coordinate shared among a plurality of the images; By superimposing the shared joint and the reference coordinates in a state in which at least one of a relative posture and a relative scale between the determined first joint position and the second joint position is applied, the joint of the second part and the joint of the first part are made to complement each other. Fatigue estimation methods.

7. A method for causing a computer to execute the fatigue estimation method according to claim 6. program.

Images