Calculation device

Abstract

This calculation device 800 comprises: a component calculation unit 831 which calculates, on the basis of image data, the inter-portion lengths between portions that are present in a target of interest of a person in the image data; a comparison length calculation unit 832 which calculates, on the basis of the image data, a comparison length to be compared; and an index value calculation unit 833 which calculates an index value that indicates a state of the target of interest on the basis of the inter-portion lengths calculated by the component calculation unit, and the comparison length calculated by the comparison length calculation unit.

Description

[Technical field]

[0001] The present invention relates to a calculation device, a calculation method, and a recording medium. [Background technology]

[0002] It is known to perform a gait analysis of a user based on the data.

[0003] An example of a document describing human gait analysis is Patent Document 1. Patent Document 1 describes a gait analysis device having a data acquisition unit that acquires two types of image data from a depth sensor, a skeletal information creation unit that creates skeletal information based on the image data acquired by the data acquisition unit, a correction processing unit that corrects the skeletal information created by the skeletal information creation unit, and an analysis processing unit that analyzes the user's gait using the corrected skeletal information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 170832 Summary of the Invention [Problem to be solved by the invention]

[0005] An example of an index to be calculated when performing gait analysis is an index value corresponding to the state of an object of interest, such as the lift of the toes, the direction of the toes, etc. However, when performing analysis based on image data without using a depth sensor such as that described in Patent Document 1, for example, it is difficult to calculate an index value indicating the state of the object of interest because information in the depth direction cannot be obtained.

[0006] Therefore, an object of the present invention is to provide a determination device, a determination method, and a program that solve the problem that it is difficult to calculate an index value according to the state of an object of interest based on image data. [Means for solving the problem]

[0007] In order to achieve this object, a calculation device according to one embodiment of the present disclosure includes: a component calculation unit that calculates a length between parts of a person in the image data that exists within an object of interest based on the image data; a comparison length calculation unit that calculates a comparison length to be compared based on the image data; an index value calculation unit that calculates an index value indicating a state of the target object based on the inter-part length calculated by the component calculation unit and the comparison length calculated by the comparison length calculation unit; have The structure is as follows.

[0008] In addition, a calculation method according to another embodiment of the present disclosure includes: An information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; Calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length. The structure is as follows.

[0009] In addition, a recording medium according to another embodiment of the present disclosure includes: In the information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; Calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length. It is a computer-readable recording medium on which a program for implementing processing is recorded. Effect of the Invention

[0010] According to each of the above-mentioned configurations, it is possible to calculate an index value according to the state of an object of interest based on image data. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a configuration example of a determination system according to a first embodiment of the present disclosure. [Diagram 2] 1 is a diagram for explaining an example of image data acquired in a determination system; [Diagram 3] 2 is a block diagram showing an example of the configuration of the determination device shown in FIG. 1. [Figure 4] FIG. 11 is a diagram showing an example of skeletal information. [Diagram 5] FIG. 11 is a diagram showing an example of ground contact determination information. [Figure 6] 10A and 10B are diagrams for explaining examples of determination made by a ground contact determining unit; [Figure 7] 11A and 11B are diagrams for explaining an example of determination by a ground contact timing determination unit. [Figure 8] 10 is a flowchart showing an example of a process performed by a ground contact determination unit. [Figure 9] 10 is a flowchart showing an example of a process performed by a ground contact timing determination unit. [Figure 10] FIG. 11 is a diagram illustrating a configuration example of a calculation system according to a second embodiment of the present disclosure. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of the calculation device shown in FIG. [Figure 12] FIG. 13 is a diagram showing an example of hand length and height. [Figure 13] 13 is a flowchart illustrating an example of the operation of the calculation device. [Figure 14] FIG. 13 is a diagram illustrating an example of the configuration of a calculation system according to a third embodiment of the present disclosure. [Figure 15] FIG. 15 is a block diagram showing an example of the configuration of the calculation device shown in FIG. [Figure 16] FIG. 13 is a diagram showing an example of height. [Figure 17] FIG. 1 is a diagram showing an example of a component. [Figure 18] FIG. 13 is a diagram showing another example of a component. [Figure 19] 13 is a flowchart illustrating an example of the operation of the calculation device. [Figure 20] FIG. 13 is a diagram illustrating an example of a hardware configuration of a calculation device according to a fourth embodiment of the present disclosure. [Figure 21] FIG. 2 is a block diagram showing an example of the configuration of a calculation device. [Figure 22] FIG. 13 is a block diagram showing another example of the configuration of the calculation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [First embodiment] A first embodiment of the present disclosure will be described with reference to Figs. 1 to 9. Fig. 1 is a diagram showing an example of a configuration of a determination system 100. Fig. 2 is a diagram for explaining an example of image data acquired in the determination system 100. Fig. 3 is a block diagram showing an example of a configuration of a determination device 300. Fig. 4 is a diagram showing an example of skeletal information 323. Fig. 5 is a diagram showing an example of ground contact determination information 324. Fig. 6 is a diagram for explaining an example of determination by the ground contact determination unit 333. Fig. 7 is a diagram for explaining an example of determination by the ground contact timing determination unit 334. Fig. 8 is a flowchart showing an example of processing by the ground contact determination unit 333. Fig. 9 is a flowchart showing an example of processing by the ground contact timing determination unit 334.

[0013] In the first embodiment of the present disclosure, a determination system 100 that determines the timing when the feet of a walking person land on the ground based on image data acquired using an imaging device such as a smartphone 200 will be described. The determination system 100 acquires time-series image data showing a person walking from the back side to the front side of the screen. As will be described later, the determination system 100 recognizes the position of the skeleton from each of the multiple image data. Then, the determination system 100 determines the timing when the feet land on the ground based on the degree of change in the recognized positions.

[0014] The timing of landing determined by the determination system 100 can be used, for example, when performing various gait analyses such as indices such as the angle of each joint at the time when the foot touches the floor, whether the toes are lifted from the floor, pitch, stride, stride cycle, etc. The timing of landing determined by the determination system 100 may be used for purposes other than those exemplified above.

[0015] Fig. 1 shows a configuration example of a determination system 100. Referring to Fig. 1, the determination system 100 includes, for example, a smartphone 200 and a determination device 300. As shown in Fig. 1, the smartphone 200 and the determination device 300 are connected to each other, for example, wirelessly or by wire, so as to be able to communicate with each other.

[0016] The smartphone 200 functions as an imaging device that captures an image of a person walking. The smartphone 200 may be a smartphone having general functions such as a camera function for acquiring image data, a touch panel for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.

[0017] In this embodiment, as shown in Fig. 2, the smartphone 200 captures an image of a person walking in a forward-backward direction, such as from the back of the screen to the front. In other words, the smartphone 200 acquires time-series image data showing a person walking in a forward-backward direction. The smartphone 200 also transmits the acquired image data to the determination device 300. The smartphone 200 may associate information indicating the date and time when the image data was acquired with the image data, and transmit the associated data to the determination device 300.

[0018] The determination device 300 is an information processing device that determines the timing when the feet of a walking person touch the ground based on image data acquired by the smartphone 200. For example, the determination device 300 is a server device. The determination device 300 may be a single information processing device, or may be realized, for example, on a cloud.

[0019] Fig. 3 shows an example of the configuration of the determination device 300. Referring to Fig. 3, the determination device 300 has, as main components, for example, a communication I / F unit 310, a storage unit 320, and an arithmetic processing unit 330.

[0020] The communication I / F unit 310 includes a data communication circuit and performs data communication with an external device, the smartphone 200, or the like connected via a communication line.

[0021] The storage unit 320 is a storage device such as a hard disk or a memory. The storage unit 320 stores processing information and a program 326 required for various processes in the calculation processing unit 330. The program 326 is read into the calculation processing unit 330 and executed to realize various processing units. The program 326 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 310, and is stored in the storage unit 320. Main information stored in the storage unit 320 includes, for example, a trained model 321, image information 322, skeletal information 323, ground contact determination information 324, and ground contact timing information 325.

[0022] The trained model 321 is a trained model used by the skeleton recognition unit 332 when performing skeleton recognition. The trained model 321 is generated in advance, for example, in an external device or the like by performing machine learning using teacher data such as image data containing skeleton coordinates, and is acquired from an external device or the like via the communication I / F unit 310 or the like and stored in the storage unit 320. The trained model 321 may be generated using a known method. The trained model 321 may be updated by a re-learning process using additional teacher data or the like.

[0023] The image information 322 includes time-series image data captured by a camera included in the smartphone 200. The image information 322 is generated and updated when the image acquisition unit 331 acquires image data, for example.

[0024] For example, image data is associated with identification information for identifying the image data in the image information 322. The image information 322 may include information other than the above examples, such as information indicating the date and time when the smartphone 200 acquired the image data.

[0025] The skeleton information 323 includes information indicating the coordinates of each part of the person recognized by the skeleton recognition unit 332. The skeleton information 323 includes, for example, information indicating the coordinates of each part of each piece of time-series image data. For example, the skeleton information 323 is generated / updated as a result of processing by the skeleton recognition unit 332.

[0026] Fig. 4 shows an example of skeletal information 323. Referring to Fig. 4, in skeletal information 323, for example, identification information and position information of each part are associated with each other. Here, the identification information is information indicating image data used when recognizing the skeletal information, etc. The identification information may correspond to identification information for identifying image data in image information 322. Furthermore, the position information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

[0027] The parts included in the position information of each part correspond to the trained model 321. For example, in FIG. 4, the parts are pelvis, center of spine, ..., right knee, left knee, ..., right ankle, left ankle, ... are shown as examples. The position information of each part can include about 30 parts such as the right shoulder, . . ., the left elbow, . . . (other parts than those exemplified may also be included). In this embodiment, the position information of each part includes information indicating the coordinates of at least the right ankle and the left ankle. The parts included in the position information of each part may be other than those exemplified in FIG. 4 and the like.

[0028] The ground contact determination information 324 includes information indicating the result of the determination made by the ground contact determination unit 333. For example, the ground contact determination information 324 is generated and updated as the result of the determination made by the ground contact determination unit 333.

[0029] FIG. 5 shows an example of the ground contact judgment information 324. As shown in FIG. 5, for example, in the ground contact judgment information 324, identification information and ground contact judgment results are associated with each other. Here, the identification information is information indicating image data for which ground contact judgment has been performed, etc. The identification information may correspond to identification information for identifying image data in the image information 322. Furthermore, the ground contact judgment result indicates the result of the ground contact judgment. For example, the ground contact judgment result includes "left foot" indicating the result of judging that the left foot is on the ground, "right foot" indicating the result of judging that the right foot is on the ground, "-" indicating that it has been judged that the foot is excluded from the ground contact judgment, and the like. The ground contact judgment result may indicate other than the above examples.

[0030] The ground contact timing information 325 includes information indicating the result of the determination by the ground contact timing determination unit 334. For example, the ground contact timing information 325 is generated and updated as the result of the determination by the ground contact timing determination unit 334.

[0031] For example, the ground contact timing information 325 includes information indicating the timing when the foot touched the ground. Here, the information indicating the timing when the foot touched the ground is, for example, at least one of identification information indicating image data determined to have touched the ground, information indicating the acquisition date and time of the image data determined to have touched the ground, etc. The ground contact timing information 325 may be, for example, information indicating the foot touching the ground and information indicating the timing when the foot touched the ground in association with each other.

[0032] The arithmetic processing unit 330 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 330 reads and executes the program 326 from the storage unit 320, thereby implementing various processing units by making the above hardware and the program 326 work together. Major processing units implemented by the arithmetic processing unit 330 include, for example, an image acquisition unit 331, a skeleton recognition unit 332, a ground contact determination unit 333, a ground contact timing determination unit 334, and an output unit 335.

[0033] The image acquisition unit 331 acquires image data acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 310. For example, the image acquisition unit 331 acquires time-series image data from the smartphone 200. In addition, the image acquisition unit 331 stores the acquired image data in the storage unit 320 as image information 322.

[0034] The skeleton recognition unit 332 uses the trained model 321 to recognize the skeleton of the person in the image data who is the target of the walking posture measurement. For example, the skeleton recognition unit 332 recognizes the skeleton for each piece of image data in time series. The skeleton recognition unit 332 may be configured to recognize the skeleton for image data extracted from the time series image data based on a predetermined condition. For example, the skeleton recognition unit 332 recognizes each part such as the upper part of the spine, the right shoulder, the left shoulder, the right elbow, the left elbow, the right wrist, the left wrist, the right hand, the left hand, ...... In addition, the skeleton recognition unit 332 calculates the coordinates in the screen data of each recognized part. Then, the skeleton recognition unit 332 associates the recognition and calculation results with identification information for identifying the image data, and stores them in the storage unit 320 as skeleton information 323. In this way, the skeleton recognition unit 332 acquires information indicating the position of each part of the person in the image data.

[0035] The parts recognized by the skeleton recognition unit 332 correspond to the trained model 321 (teaching data used when training the trained model 321). Therefore, the skeleton recognition unit 332 may recognize parts other than those exemplified above according to the trained model 321.

[0036] The ground contact determination unit 333 determines which foot is on the ground based on the skeleton of the person indicated by the skeleton information 323. For example, the ground contact determination unit 333 determines which foot, the right foot or the left foot, of a walking person is on the ground based on the change in the coordinates of the right ankle and the left ankle included in the skeleton information 323. For example, the ground contact determination unit 333 acquires information indicating the position, such as the coordinates of the ankle, by referring to the skeleton information 323. Then, when the change in the coordinates of the ankle satisfies a predetermined condition, the ground contact determination unit 333 determines that the foot having the ankle that satisfies the condition is on the ground. In addition, the ground contact determination unit 333 stores information indicating the result of the determination in the storage unit 320 as ground contact determination information 324.

[0037] FIG. 6 shows an example of changes in the Y coordinates (the vertical coordinates of the image data) of the right and left ankles. For example, in FIG. 6, the X axis shows values ​​according to time, and the Y axis shows the vertical coordinates (Y coordinates) in the image data. For example, in the case of FIG. 6, the larger the Y axis value, the lower the right or left ankle is located in the image data (i.e., the closer the right or left ankle is located to the front of the screen).

[0038] Generally, when a person walks, the following motions are repeated alternately: one of the right and left feet remains on the ground while the other moves forward, both the right and left feet touch the ground, one of the right and left feet moves forward while the other remains on the ground, both the right and left feet touch the ground. In other words, in the case of this embodiment, it is assumed that the following states are repeated: the Y coordinate of the right ankle is constant and the Y coordinate of the left ankle increases, the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant, the Y coordinate of the left ankle is constant and the Y coordinate of the right ankle increases, and the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant.

[0039] Therefore, the ground contact determination unit 333 determines which foot is on the ground based on the change in the Y coordinate of the right ankle and the left ankle. In other words, when it can be evaluated that the Y coordinate of the ankle is constant, the ground contact determination unit 333 determines that the foot having the ankle is on the ground. Specifically, for example, the ground contact determination unit 333 compares the Y coordinate of the ankle in the image data to be determined with the Y coordinate of the ankle in the image data immediately preceding the determination target in the time series image data. Then, when the displacement (amount of change) from the Y coordinate of the ankle in the immediately preceding image data is equal to or less than a predetermined value, the ground contact determination unit 333 determines that the Y coordinate is constant and that the foot having the ankle with the constant Y coordinate is on the ground. Note that the ground contact determination unit 333 may determine that the Y coordinate is constant by a method other than the above example. Also, the range of displacement in which the ground contact determination unit 333 determines that the Y coordinate is constant may be set arbitrarily.

[0040] In addition, the ground contact determination unit 333 may be configured to determine that the foot other than the foot determined to be on the ground starts to move based on the Y coordinate of the ankle, and to exclude the foot from the ground contact determination. For example, in the case of walking, the right foot generally touches the ground, and then the left foot starts to move. The ground contact determination unit 333 may determine that the left foot starts to move when the Y coordinate of the left ankle starts to increase after the Y coordinate of the right ankle becomes constant. Specifically, for example, the ground contact determination unit 333 can determine that the left foot starts to move and exclude the left ankle from the ground contact determination when the displacement of the Y coordinate of the left ankle from the previous image data in the time series image data exceeds the displacement of the Y coordinate of the right ankle from the previous image data after the Y coordinate of the right ankle becomes constant.

[0041] Generally, when a person walks, the foot that will newly touch the ground moves forward more than the foot that has already touched the ground. That is, in this embodiment, the Y coordinate of the ankle of the foot that will newly touch the ground is larger than the Y coordinate of the ankle of the foot that has already touched the ground. Therefore, the ground contact determination unit 333 may be configured to determine whether or not the ankle is touching the ground by focusing only on the ankle with the larger Y coordinate out of the right ankle and the left ankle.

[0042] Furthermore, the ground contact determination unit 333 may be configured to refer to the coordinates of the knee in addition to the coordinates of the ankle when making a ground contact determination. For example, the ground contact determination unit 333 may be configured to determine that the foot, which is to be determined whether or not it is on the ground, is excluded from the ground contact determination when the change in the coordinate of the knee satisfies a predetermined condition. Specifically, for example, the ground contact determination unit 333 may be configured to determine that the foot, which is to be determined whether or not it is on the ground, is excluded from the ground contact determination when the displacement of the Y coordinate of the knee from the previous image data in the time series image data is 0 or less.

[0043] For example, as described above, the ground contact determination unit 333 performs the ground contact determination based on the change in the Y coordinate of the ankle. Note that the ground contact determination unit 333 may be configured to perform the ground contact determination based on the change in the skeleton other than the above-mentioned examples, such as using the Y coordinate of the toe.

[0044] The ground contact timing determination unit 334 determines the timing when the foot touches the ground based on the foot placement state determined based on the person's skeleton indicated by the skeleton information 323. For example, the ground contact timing determination unit 334 determines the timing when the foot touches the ground based on a change in the foot touching the ground indicated by the ground contact determination information 324. Then, the ground contact timing determination unit 334 stores the determination result in the storage unit 320 as ground contact timing information 325.

[0045] For example, the contact timing determination unit 334 determines that the foot has touched the ground at the timing when the contact foot indicated by the contact determination result is switched to another foot. For example, referring to FIG. 7, the "left foot" is contacted at identification information "124" and "125", and is excluded from contact determination at identification information "126" and "127". After that, it is determined that the "right foot" is contacted at identification information "128". That is, in the case of FIG. 7, the contact foot is switched from the left foot to the right foot at the timing of identification information "128". Therefore, the contact timing determination unit 334 determines that the right foot has touched the ground at the timing of identification information "128".

[0046] The output unit 335 outputs the ground contact timing information 325, the ground contact determination information 324, etc. For example, the output unit 335 can output at least one of the above information to an external device, the smartphone 200, or the like.

[0047] The above is an example of the configuration of the determination device 300. Next, the operation of the determination device 300 will be described with reference to Figs.

[0048] First, an example of the operation of the ground contact determination unit 333 will be described with reference to Fig. 8. Referring to Fig. 8, the ground contact determination unit 333 compares the Y coordinate of the knee in the image data to be determined with the Y coordinate of the knee in the image data immediately preceding the determination target in the time series image data (step S101). If the displacement from the Y coordinate of the knee in the immediately preceding image data exceeds 0 (step S101, Yes), the ground contact determination unit 333 checks whether the displacement of the ankle satisfies a condition (step S102).

[0049] If the ankle displacement satisfies the condition (step S102, Yes), the ground contact determination unit 333 determines that the foot that satisfies the condition is on the ground (step S103). On the other hand, if the displacement of the Y coordinate of the knee is 0 or less (step S101, No) or if the ankle displacement does not satisfy the condition (step S102, No), the ground contact determination unit 333 determines that the foot that does not satisfy the condition is not on the ground.

[0050] The above is an example of the operation of the ground contact determination unit 333. The ground contact determination unit 333 can determine that the displacement of the ankle satisfies the condition, for example, when the Y coordinate of the ankle can be evaluated as being constant based on the change in the Y coordinate, when it is not determined that the foot other than the foot determined to be on the ground has started to move, or when the change in the knee coordinate satisfies the condition. The ground contact determination unit 333 may be configured to identify an ankle that focuses on the Y coordinate of the ankle, and to perform the above-mentioned check on the identified ankle.

[0051] Next, an example of the operation of the ground contact timing determination unit 334 will be described with reference to Fig. 9. Referring to Fig. 9, the ground contact timing determination unit 334 checks whether the ground contact foot indicated by the ground contact determination result has changed by referring to the ground contact timing information 325 (step S201).

[0052] If the foot has changed (step S201, Yes), the ground contact timing determination unit 334 determines that the changed timing is the ground contact timing (step S202). On the other hand, if the foot on the ground shown by the determination result has not changed (step S201), the ground contact timing determination unit 334 does not perform the above determination.

[0053] The above is an example of the processing performed by the contact timing determination unit 334.

[0054] Thus, the determination device 300 has the skeleton recognition unit 332, the ground contact determination unit 333, and the ground contact timing determination unit 334. With this configuration, the ground contact timing determination unit 334 can determine the timing at which the foot touches the ground based on the determination result of the ground contact determination unit 333 using the coordinates of the skeleton recognized by the skeleton recognition unit 332. In other words, with the above configuration, it becomes possible to grasp the timing at which the foot touches the ground based on the image data.

[0055] The determination device 300 may be configured to determine the landing timing based on the results of skeleton recognition by an external device that performs skeleton recognition. When configured in this way, the determination device 300 does not need to have the function of the skeleton recognition unit 332.

[0056] [Second embodiment] Next, a second embodiment of the present disclosure will be described with reference to Fig. 10 to Fig. 13. Fig. 10 is a diagram showing a configuration example of a calculation system 400. Fig. 11 is a block diagram showing a configuration example of a calculation device 500. Fig. 12 is a diagram showing an example of effort length and height. Fig. 13 is a flowchart showing an operation example of the calculation device 500.

[0057] In the second embodiment of the present disclosure, a calculation system 400 is described that calculates an index value indicating the bending of the waist of a person in image data based on image data acquired using an imaging device such as a smartphone 200. As in the first embodiment, the calculation system 400 acquires image data showing a person walking from the back side of the screen to the front side, and recognizes the position of the skeleton from the image data. Then, the calculation system 400 calculates an index value indicating the bending of the waist based on the recognition result.

[0058] As described later, the calculation system 400 described in this embodiment acquires the hand length, which is the sum of the distances in the image of the skeletal lengths connecting each part present in the left and right hands, and the height, which is the distance in the image between the head and the toes. Then, the calculation system 400 calculates an index value indicating the bending of the waist based on the acquired hand length and height. If both the hand length and the height are assumed to be height, the height changes as the waist is bent, while the hand length does not change even if the waist is bent. In addition, the length of both hands when spread out to the left and right corresponds to the height. The calculation system 400 calculates an index value indicating the bending of the waist by utilizing the above. It is known that the ratio of the hand length to the height changes depending on the age, height, etc. Therefore, the calculation system 400 may be configured to add a predetermined correction value to the hand length according to the attributes of the person such as age, height, and sex, and then calculate an index value indicating the bending of the waist based on the corrected hand length and height.

[0059] The calculation system 400 described in this embodiment can have the function of the determination system 100 described in the first embodiment. In other words, the calculation system 400 may have a configuration for determining the contact timing.

[0060] Fig. 10 shows a configuration example of the calculation system 400. Referring to Fig. 10, the calculation system 400 includes, for example, a smartphone 200 and a calculation device 500. As shown in Fig. 10, the smartphone 200 and the calculation device 500 are connected to each other, for example, wirelessly or by wire, so as to be able to communicate with each other.

[0061] The configuration of the smartphone 200 is the same as that of the first embodiment, so a description of the smartphone 200 will be omitted.

[0062] The calculation device 500 is an information processing device that calculates an index value indicating bending of the waist based on image data acquired by the smartphone 200. For example, the calculation device 500 is a server device. The calculation device 500 may be a single information processing device, or may be realized on a cloud, for example.

[0063] Fig. 11 shows an example of the configuration of the calculation device 500. Referring to Fig. 11, the calculation device 500 has, as main components, for example, a communication I / F unit 510, a storage unit 520, and an arithmetic processing unit 530.

[0064] The communication I / F unit 510 includes a data communication circuit and performs data communication with an external device, the smartphone 200, or the like connected via a communication line.

[0065] The storage unit 520 is a storage device such as a hard disk or a memory. The storage unit 520 stores processing information and a program 527 required for various processes in the calculation processing unit 530. The program 527 is read into the calculation processing unit 530 and executed to realize various processing units. The program 527 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 510, and is stored in the storage unit 520. Main information stored in the storage unit 520 includes, for example, a trained model 521, image information 522, skeletal information 523, hand length information 524, height information 525, and waist bending value information 526.

[0066] The trained model 521, image information 522, and skeletal information 523 are similar to the trained model 321, image information 322, and skeletal information 323 described in the first embodiment.

[0067] The effort length information 524 indicates the effort length, which is the sum of distances in the image of the skeletal lengths connecting the parts between the left and right hands, such as the palm, wrist, elbow, and shoulder. The effort length information 524 is calculated in advance based on image data acquired with the hand extended downward so that the effort length is the longest, and is acquired from an external device via the communication I / F unit 510 and stored in the storage unit 520. The effort length information 524 may be generated or updated as a result of a calculation process by the effort length calculation unit 533.

[0068] For example, the effort length information 524 includes information indicating the effort length. The effort length information 524 may include information indicating the effort length acquired for each time-series image data. For example, the effort length information 524 may be information in which identification information indicating the image data used when calculating the effort length is associated with information indicating the effort length.

[0069] The height information 525 indicates the height of a person in the image data, which is the distance in the image connecting the head (forehead) and the toes (for example, the lower of the left and right). The height information 525 is generated and updated, for example, as a result of a calculation process by the height calculation unit 534.

[0070] For example, the height information 525 includes information indicating the height acquired for each time series image data. The height information 525 may be information in which identification information indicating the image data used in calculating the height is associated with information indicating the height. The height information 525 may also be information in which identification information indicating image data satisfying a predetermined condition is associated with information indicating the height.

[0071] The waist bending value information 526 indicates a waist bending value which is an index value indicating the bending of the waist. The waist bending value information 526 is generated / updated as a result of a calculation process by the bending calculation unit 535, for example.

[0072] For example, waist bending value information 526 includes information indicating a waist bending value. Waist bending value information 526 may also include information indicating a waist bending value acquired for each time-series image data. Waist bending value information 526 may be, for example, a combination of identification information indicating image data used in calculating the waist bending value and information indicating the waist bending value. Waist bending value information 526 may also be, for example, a combination of identification information indicating image data satisfying a predetermined condition and information indicating the waist bending value.

[0073] The arithmetic processing unit 530 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 530 reads and executes a program 527 from the storage unit 520, thereby implementing various processing units by making the above hardware and the program 527 work together. Major processing units implemented by the arithmetic processing unit 530 include, for example, an image acquisition unit 531, a skeleton recognition unit 532, a hand-to-hand length calculation unit 533, a height calculation unit 534, a bend calculation unit 535, and an output unit 536.

[0074] The image acquisition unit 531 and the skeleton recognition unit 532 can perform the same processes as the image acquisition unit 331 and the skeleton recognition unit 332 described in the first embodiment.

[0075] The effort length calculation unit 533 calculates the effort length, which is the sum of the in-image distances of the skeleton lengths connecting the left and right efforts, based on the skeleton information 523. Then, the effort length calculation unit 533 stores the calculated effort length in the storage unit 520 as effort length information 524.

[0076] For example, Fig. 12 shows an example of the effort length. Referring to Fig. 12, the effort length calculation unit 533 acquires information indicating the positions of each part between the left and right hands, such as the palm, wrist, elbow, and shoulder, by referring to the skeletal information 523. Then, based on the acquired information, the effort length calculation unit 533 calculates the sum of the distances between each part when adjacent parts are connected by a straight line. For example, as described above, the effort length calculation unit 533 calculates the effort length based on the positions of each part between the left and right hands.

[0077] The effort length calculation unit 533 can perform the above-mentioned calculation process of the effort length for each image data in the time series. That is, the effort length calculation unit 533 may be configured to calculate the effort length for each image data. The effort length calculation unit 533 may be configured to calculate the effort length for image data extracted under any condition from the image data in the time series.

[0078] The effort length calculation unit 533 may calculate the effort length using a portion other than the above-mentioned examples. Information indicating the effort length can be calculated and acquired in advance. When information indicating the effort length is acquired in advance, the calculation device 500 does not need to include the effort length calculation unit 533.

[0079] The height calculation unit 534 calculates the height, which is the distance from the head to the toes in the image, based on the skeleton information 523. Then, the height calculation unit 534 stores the calculated height in the storage unit 520 as height information 525.

[0080] For example, FIG. 12 shows an example of height. Referring to FIG. 12, the height calculation unit 534 acquires information indicating the positions of the forehead, which is the highest part, and the left and right toes by referring to the skeletal information 523. Then, the height calculation unit 534 calculates the distance between the forehead and the lower of the left and right toes with a straight line based on the acquired information. Alternatively, the height calculation unit 534 calculates the difference in the Y-axis direction (the up and down direction in FIG. 12) between the forehead and the lower of the left and right toes based on the acquired information. For example, as described above, the height calculation unit 534 calculates the height based on the positions of the head and the feet.

[0081] The height calculation unit 534 can perform the above-mentioned height calculation process for each piece of image data in the time series, similar to the hand length calculation unit 533. That is, the height calculation unit 534 may be configured to calculate the height in each piece of image data. The height calculation unit 534 may be configured to calculate the height in image data extracted under any condition from the time series image data.

[0082] The height calculation unit 534 may calculate the height using a part other than the above-mentioned examples. For example, the height calculation unit 534 may calculate the length between parts including the waist between the neck and the ankle as the height.

[0083] The bending calculation unit 535 calculates a waist bending value, which is an index value indicating the bending of the waist, based on the hand-to-hand length indicated by the hand-to-hand length information 524 and the height indicated by the height information 525. The bending calculation unit 535 then stores the calculated waist bending value in the storage unit 520 as waist bending value information 526.

[0084] For example, the curvature calculation unit 535 calculates the waist curvature value by dividing the hand length by the height. That is, the curvature calculation unit 535 calculates the waist curvature value by calculating (hand length) / (height). For example, the curvature calculation unit 535 can calculate the waist curvature value by using the hand length and height calculated for each image data.

[0085] It is assumed that the effort length is shortened by waving hands while walking. Therefore, the bending calculation unit 535 may calculate the bending value of the waist using an effort length selected from the effort length information 524 according to a predetermined criterion, such as using the longest effort length among the effort lengths included in the effort length information 524. In other words, the effort length used by the bending calculation unit 535 when calculating the bending value of the waist may be an effort length selected according to a predetermined criterion among the effort lengths included in the effort length information 524. The selection criterion may be set arbitrarily, such as using the longest one. In addition, the effort length information 524 may include only one effort length acquired in advance. Therefore, the effort length used by the bending calculation unit 535 when calculating the bending value of the waist may be, for example, a constant value.

[0086] The bending calculation unit 535 may also specify image data that is the calculation target of the waist bending value, using the result of the determination of the landing timing described in the first embodiment. For example, the bending calculation unit 535 may be configured to calculate the waist bending value using the height acquired based on the image data determined as the landing timing. The bending calculation unit 535 may also be configured to calculate the waist bending value using the height acquired based on the image data of a predetermined number of frames from the image data determined as the landing timing. In this way, by configuring the bending value of the waist to be calculated based on the image data specified based on the determination result of the landing timing (based on the height), it is possible to calculate a waist bending value that is more appropriate as an index to be compared with other people. The bending calculation unit 535 may also be configured to calculate the waist bending value for each image data in the time series, and to calculate the average value of the calculated waist bending values.

[0087] The output unit 536 outputs hand-to-hand length information 524, height information 525, waist bending value information 526, etc. For example, the output unit 536 can output at least one of the above information to an external device, the smartphone 200, etc.

[0088] The above is an example of the configuration of the calculation device 500. Next, an example of the operation of the calculation device 500 will be described with reference to FIG.

[0089] Referring to FIG. 13, the bend calculation unit 535 acquires the hand length indicated by the hand length information 524 and the height indicated by the height information 525 (step S301).

[0090] The bending calculation unit 535 calculates a bending value of the waist, which is an index value indicating bending of the waist, based on the hand length and the height (step S302). For example, the bending calculation unit 535 calculates the bending value of the waist by dividing the hand length by the height.

[0091] The above is an example of the operation of the calculation device 500.

[0092] In this way, the calculation device 500 has the bending calculation unit 535. With this configuration, the calculation device 500 can calculate the bending value of the waist based on the acquired hand length and height. In other words, with the above configuration, it is possible to calculate an index value corresponding to the bending of the waist based on the image data.

[0093] The calculation device 500 can have the function of the determination device 300 described in the first embodiment. The calculation device 500 can have the same modified example as the determination device 300 described in the first embodiment.

[0094] [Third embodiment] Next, a third embodiment of the present disclosure will be described with reference to Fig. 14 to Fig. 19. Fig. 14 is a diagram showing a configuration example of a calculation system 600. Fig. 15 is a block diagram showing a configuration example of a calculation device 700. Fig. 16 is a diagram showing an example of height. Figs. 17 and 18 are diagrams showing examples of foot components. Fig. 19 is a flowchart showing an operation example of the calculation device 700.

[0095] In the third embodiment of the present disclosure, a calculation system 600 is described that calculates an index value indicating the state of an object of interest based on image data acquired using an imaging device such as a smartphone 200. As in the first and second embodiments, the calculation system 600 acquires image data showing a person walking from the back of the screen to the front, and recognizes the position of the skeleton from the image data. Then, the calculation system 600 calculates an index value indicating the state of the object based on the recognition result.

[0096] For example, the calculation system 600 described in this embodiment calculates an index value indicating the state of the toes, such as the rise and direction of the toes, focusing on the foot. Specifically, for example, the calculation system 600 acquires the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the toes, which are parts present in the foot of interest, and a comparison length to be compared. Then, the calculation system 600 calculates an index value indicating the direction of the toes based on the difference in the X-axis direction and the comparison length, and calculates an index value indicating the rise of the toes based on the difference in the Y-axis direction and the comparison length. The closer a person is to the camera, which is an imaging device, the larger the person is captured. Therefore, even if the toes are raised in the same way or point in the same direction, the length in the image data differs depending on the proximity of the person to the imaging device. The calculation system 600 makes it possible to calculate an index value that solves the above problem by comparing the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the toes with the comparison length, which is the length to be compared.

[0097] In this embodiment, the case where the foot is focused on as the target of interest is illustrated. However, the target of interest may be any part other than the foot. That is, the calculation system 600 may use the difference in the X-axis direction or the difference in the Y-axis direction between the positions of parts existing in the target of interest other than the ankle and the toes. The comparison length refers to the length between parts whose length in the image data varies depending on the proximity of the person to the camera, which is the imaging device, when the image data is acquired, but whose length is constant for the person. The comparison length may be, for example, one of the height (without considering the bending of the waist) which is the distance in the image from the head to the toes described in the second embodiment, the hand length, the height which is the distance in the image from the head to the pelvis, and the like. The comparison length may be the length between parts other than those illustrated above.

[0098] Furthermore, the calculation system 600 described in this embodiment can have the functions of the determination system 100 described in the first embodiment and the calculation system 400 described in the second embodiment. In other words, the calculation system 600 may have a configuration for determining the ground contact timing and a configuration for calculating the waist bending value.

[0099] Fig. 14 illustrates a configuration example of a calculation system 600. Referring to Fig. 14, the calculation system 600 includes, for example, a smartphone 200 and a calculation device 700. As illustrated in Fig. 14, the smartphone 200 and the calculation device 700 are connected to each other, for example, wirelessly or by wire, so as to be able to communicate with each other.

[0100] The configuration of the smartphone 200 is the same as that of the first embodiment, so a description of the smartphone 200 will be omitted.

[0101] The calculation device 700 is an information processing device that calculates an index value indicating the lift of the toes and the direction of the toes based on image data acquired by the smartphone 200. For example, the calculation device 700 is a server device. The calculation device 700 may be a single information processing device, or may be realized, for example, on the cloud. Note that the calculation device 700 may calculate only one of the index value indicating the lift of the toes and the index value indicating the direction of the toes.

[0102] Fig. 15 shows an example of the configuration of the calculation device 700. Referring to Fig. 15, the calculation device 700 has, as main components, for example, a communication I / F unit 710, a storage unit 720, and an arithmetic processing unit 730.

[0103] The communication I / F unit 710 includes a data communication circuit and performs data communication with an external device, the smartphone 200, or the like connected via a communication line.

[0104] The storage unit 720 is a storage device such as a hard disk or a memory. The storage unit 720 stores processing information and a program 727 required for various processes in the calculation processing unit 730. The program 727 is read into the calculation processing unit 730 and executed to realize various processing units. The program 727 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 710, and is stored in the storage unit 720. Main information stored in the storage unit 720 includes, for example, a trained model 721, image information 722, skeletal information 723, comparative length information 724, foot component information 725, and toe data information 726.

[0105] The trained model 721, image information 722, and skeletal information 723 are similar to the trained model 321, image information 322, and skeletal information 323 described in the first embodiment.

[0106] The comparative length information 724 indicates a comparative length, which is a length between parts of a person whose length is constant for that person, although the length in the image data varies depending on the proximity of the person to the camera, which is an imaging device, when the image data is acquired. For example, the comparative length information 724 indicates one of a height, which is the distance in the image from the head to the toes, a hand length, a second height, which is the distance in the image from the head to the pelvis, and the like. The comparative length information 724 is generated and updated, for example, as a result of a calculation process by the comparative length calculation unit 733.

[0107] For example, the comparison length information 724 includes information indicating a comparison length obtained for each time-series image data. For example, the comparison length information 724 may be information in which identification information indicating the image data used in calculating the comparison length is associated with information indicating the comparison length.

[0108] The foot component information 725 indicates inter-part lengths such as an X component which is the difference in the X-axis direction between the positions of the ankle and the toes, and a Y component which is the difference in the Y-axis direction between the positions of the ankle and the toes. The foot component information 725 is generated and updated, for example, as a result of a calculation process by the foot component calculation unit 734. Note that the foot component information 725 may include only one of the X component and the Y component.

[0109] For example, the foot component information 725 includes information indicating the X component and the Y component acquired for each time-series image data. For example, the comparison length information 724 may be information that associates identification information indicating the image data used to calculate the X component and the Y component with information indicating the X component and the Y component.

[0110] The toe data information 726 indicates an index value indicating the lift of the toes and an index value indicating the direction of the toes. The toe data information 726 is generated and updated as a result of a calculation process by the toe data calculation unit 735, for example.

[0111] For example, the toe data information 726 includes an index value indicating the rise of the toes and an index value indicating the direction of the toes. The toe data information 726 includes an index value indicating the rise of the toes and an index value indicating the direction of the toes acquired for each time-series image data. The height information 525 may be, for example, a combination of identification information indicating the image data used to calculate the index value and an index value indicating the rise of the toes and an index value indicating the direction of the toes. The toe data information 726 may also be, for example, a combination of identification information indicating image data satisfying a predetermined condition and an index value indicating the rise of the toes and an index value indicating the direction of the toes.

[0112] The arithmetic processing unit 730 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 730 reads and executes a program 727 from the storage unit 720, thereby implementing various processing units by making the above hardware and the program 727 work together. Major processing units implemented by the arithmetic processing unit 730 include, for example, an image acquisition unit 731, a skeleton recognition unit 732, a comparative length calculation unit 733, a foot component calculation unit 734, a toe data calculation unit 735, and an output unit 736.

[0113] The image acquisition unit 731 and the skeleton recognition unit 732 can perform the same processes as the image acquisition unit 331 and the skeleton recognition unit 332 described in the first embodiment.

[0114] The comparative length calculation unit 733 calculates a comparative length, which is one of a height that is a distance in an image from the head to the toes, a hand length, a second height that is a distance in an image from the head to the pelvis, etc., based on the skeletal information 723. Then, the comparative length calculation unit 733 stores the calculated comparative length in the storage unit 720 as comparative length information 724.

[0115] For example, FIG. 16 shows an example of height, which is an example of the comparative length. Referring to FIG. 16, the comparative length calculation unit 733 acquires information indicating the positions of the forehead, which is the highest part, and the left and right toes by referring to the skeletal information 523. Then, the comparative length calculation unit 733 calculates the distance between the forehead and the lower one of the left and right toes with a straight line based on the acquired information. Alternatively, the comparative length calculation unit 733 calculates the difference in the Y-axis direction (up and down direction in FIG. 12) between the forehead and the lower one of the left and right toes based on the acquired information. For example, as described above, the comparative length calculation unit 733 calculates the height based on the positions of the head and the feet. Note that the comparative length calculation unit 733 may be configured to calculate the hand length or the second height, which is the in-image distance from the head to the pelvis, as the comparative length by similar processing.

[0116] The foot component calculation unit 734 calculates at least one of inter-part lengths such as an X component that is the difference in the X-axis direction between the positions of the ankle and the toes, and a Y component that is the difference in the Y-axis direction between the positions of the ankle and the toes, by referring to the skeleton information 523. Then, the foot component calculation unit 734 stores the calculated foot components in the storage unit 720 as foot component information 725.

[0117] For example, Fig. 17 shows an example of the Y component. Referring to Fig. 17, the foot component calculation unit 734 acquires information indicating the positions of the ankle and the toes by referring to the skeleton information 523. Then, based on the acquired information, the foot component calculation unit 734 calculates the difference between the ankle and the toes in the Y-axis direction (the up-down direction in Fig. 17). For example, as described above, the foot component calculation unit 734 calculates the Y component based on the positions of the ankle and the toes.

[0118] Moreover, Fig. 18 shows an example of the X component. Referring to Fig. 18, the foot component calculation unit 734 acquires information indicating the positions of the ankle and the toes by referring to the skeleton information 523. Then, the foot component calculation unit 734 calculates the difference between the ankle and the toes in the X-axis direction (the left-right direction in Fig. 18) based on the acquired information. For example, as described above, the foot component calculation unit 734 calculates the X component based on the positions of the ankle and the toes.

[0119] The foot component calculation unit 734 may be configured to calculate the Y and X components of each of the left and right feet, or may be configured to calculate the Y and X components of only the foot that is located lower among the left and right feet. The foot component calculation unit 734 may be configured to determine whether or not to calculate the foot components using the result of the determination of the contact timing described in the first embodiment. For example, the foot component calculation unit 734 can be configured to calculate the Y and X components based on image data determined to be contact timing. The foot component calculation unit 734 may be configured to calculate the Y and X components based on a predetermined number of frames of image data from the image data determined to be contact timing.

[0120] The toe data calculation unit 735 calculates an index value indicating the rise of the toes based on the comparison length indicated by the comparison length 724 and the Y component indicated by the foot component information 725. The toe data calculation unit 735 also calculates an index value indicating the direction of the toes based on the comparison length indicated by the comparison length 724 and the X component indicated by the foot component information 725. The toe data calculation unit 735 then stores the calculated index value indicating the rise of the toes and the index value indicating the direction of the toes in the storage unit 720 as toe data information 726.

[0121] For example, the toe data calculation unit 735 calculates an index value indicating the rise of the toes by dividing the Y component by the comparison length. That is, the toe data calculation unit 735 calculates an index value indicating the rise of the toes by calculating (Y component) / (comparison length). For example, the toe data calculation unit 735 can calculate an index value indicating the rise of the toes by using the Y component calculated for each image data and the comparison length.

[0122] Also, for example, the toe data calculation unit 735 calculates an index value indicating the direction of the toes by dividing the X component by the comparison length. That is, the toe data calculation unit 735 calculates an index value indicating the direction of the toes by calculating (X component) / (comparison length). For example, the toe data calculation unit 735 can calculate an index value indicating the direction of the toes by using the X component calculated for each image data and the comparison length.

[0123] The toe data calculation unit 735 may use the result of the determination of the ground contact timing described in the first embodiment to specify image data to be used for calculating the index value indicating the toe lift and the index value indicating the toe direction. For example, the toe data calculation unit 735 may be configured to calculate the index value indicating the toe lift and the index value indicating the toe direction using the Y component and X component acquired based on the image data determined as the ground contact timing and the comparison length. The toe data calculation unit 735 may be configured to calculate the index value indicating the toe lift and the index value indicating the toe direction using the Y component and X component acquired based on the image data of a predetermined number of frames from the image data determined as the ground contact timing and the comparison length. In this way, by performing calculations based on the image data specified based on the result of the determination of the ground contact timing, it becomes possible to evaluate the toe lift and the toe direction when the foot touches the ground.

[0124] The output unit 736 outputs the comparative length information 724, the foot component information 725, the toe data information 726, etc. For example, the output unit 736 can output at least one of the above information to an external device, the smartphone 200, etc.

[0125] The above is an example of the configuration of the calculation device 700. Next, an example of the operation of the calculation device 700 will be described with reference to FIG.

[0126] Referring to FIG. 19, the toe data calculation unit 735 acquires the comparative length indicated by the comparative length information and the Y component and the X component indicated by the foot component information 725 (step S401).

[0127] The toe data calculation unit 735 calculates toe data based on the comparison length and the Y component and the X component. For example, the toe data calculation unit 735 calculates an index value indicating the lift of the toes based on the comparison length and the Y component. Also, for example, the toe data calculation unit 735 calculates an index value indicating the direction of the toes based on the comparison length and the X component.

[0128] The above is an example of the operation of the calculation device 700.

[0129] In this way, the calculation device 700 has a toe data calculation unit 735. With this configuration, the calculation device 700 can calculate an index value indicating the lift of the toes and an index value indicating the direction of the toes based on the acquired comparison length and the Y component and X component. In other words, with the above configuration, it is possible to calculate an index value indicating the state of the lift and direction of the toes based on the image data.

[0130] When the target object is an arbitrary part other than the foot, the calculation device 700 may be configured to calculate, as the Y component and the X component, the length between parts existing within the target object other than the ankle and the toe.

[0131] Moreover, the calculation device 700 can have the function of the determination device 300 described in the first embodiment. Moreover, the calculation device 700 can have the same modified example as the determination device 300 described in the first embodiment. Similarly, the calculation device 700 can have the function of the calculation device 500 described in the second embodiment. Moreover, the calculation device 700 can have the same modified example as the calculation device 500 described in the second embodiment.

[0132] [Fourth embodiment] Next, a fourth embodiment of the present disclosure will be described with reference to Fig. 20 to Fig. 22. In the fourth embodiment of the present disclosure, an overview of the configuration of a calculation device 800 that is an information processing device will be described.

[0133] Fig. 20 shows an example of the hardware configuration of the calculation device 800. Referring to Fig. 20, the calculation device 800 has, as an example, the following hardware configuration. ·CPU (Central Processing Unit) 801 (computing unit) ROM (Read Only Memory) 802 (storage device) RAM (Random Access Memory) 803 (storage device) · Programs 804 loaded into RAM 803 A storage device 805 for storing a group of programs 804 A drive device 806 for reading and writing data from a recording medium 810 outside the information processing device A communication interface 807 for connecting to a communication network 811 outside the information processing device Input / output interface 808 for inputting and outputting data Bus 809 connecting each component

[0134] 21. The program group 804 is stored in advance in, for example, the storage device 805 or the ROM 802, and is loaded into, for example, the RAM 803 by the CPU 801 when necessary, and executed by the CPU 801. The program group 804 may be supplied to the CPU 8801 via the communication network 811, or may be stored in advance in the recording medium 810, and the drive device 806 may read out the program and supply it to the CPU 801.

[0135] 20 shows an example of the hardware configuration of the calculation device 800. The hardware configuration of the calculation device 800 is not limited to the above. For example, the calculation device 800 may be configured with a part of the above configuration, such as not having the drive device 806.

[0136] The height calculation unit 821 calculates the height of a person in the image data based on the image data.

[0137] The bending calculation unit 822 acquires a hand length that is a length connecting the left and right hands of the person in the image data. The bending calculation unit 822 also calculates an index value indicating the bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit 821.

[0138] In this way, the calculation device 800 has a height calculation unit 821 and a curvature calculation unit 822. The curvature calculation unit 822 is configured to acquire a hand length. With this configuration, the curvature calculation unit 822 can calculate an index value indicating the curvature of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit 821. As a result, it becomes possible to calculate an index value corresponding to the curvature of the waist based on the image data.

[0139] An information processing device such as the calculation device 800 described above can be realized by incorporating a predetermined program into the information processing device. Specifically, a program according to another embodiment of the present invention is a program for causing an information processing device to realize a process of calculating the height of a person in image data based on the image data, acquiring a wrist length that is a length connecting the left and right wrists of the person in the image data, and calculating an index value indicating the bending of the waist of the person in the image data based on the acquired wrist length and the calculated height.

[0140] In addition, the calculation method executed by the above-mentioned information processing device is such that the information processing device calculates the height of the person in the image data based on the image data, obtains a wrist length which is the length connecting the left and right wrists of the person in the image data, and calculates an index value indicating the curvature of the waist of the person in the image data based on the obtained wrist length and the calculated height.

[0141] Even if the invention is a program (or a recording medium) or a calculation method having the above-mentioned configuration, it has the same action and effect as the above-mentioned case, and therefore can achieve the same object as the calculation device 800 having the above-mentioned curvature calculation unit 822.

[0142] In addition, the calculation device 800 may be configured to realize the functions of a component calculation unit 831, a comparison length calculation unit 832, and an index value calculation unit 833 shown in FIG. 22 by the CPU 801 acquiring the program group 804 and executing it.

[0143] Based on the image data, the component calculation section 831 calculates the part-to-part length between parts that exist within the target object of interest of the person in the image data.

[0144] The comparative length calculation unit 832 calculates a comparative length to be compared based on the image data.

[0145] The index value calculation section 833 calculates an index value indicating the state of the target object, based on the inter-part length calculated by the component calculation section 831 and the comparison length calculated by the comparison length calculation section 832.

[0146] In this way, the calculation device 800 can have a component calculation section 831, a comparison length calculation section 832, and an index value calculation section 833. With this configuration, the index value calculation section 833 can calculate an index value indicating the state of the target of interest based on the inter-part length calculated by the component calculation section 831 and the comparison length calculated by the comparison length calculation section 832. As a result, it becomes possible to calculate an index value according to the state of the target of interest based on the image data.

[0147] An information processing device such as the calculation device 800 described above can be realized by incorporating a predetermined program into the information processing device. Specifically, a program according to another aspect of the present invention is a program for causing an information processing device to realize a process of calculating, based on image data, a part-to-part length between parts present in a target object of interest for a person in the image data, calculating, based on the image data, a comparison length to be compared, and calculating an index value indicating a state of the target object of interest based on the calculated part-to-part length and the calculated comparison length.

[0148] In addition, the calculation method executed by the above-mentioned information processing device is such that the information processing device calculates, based on image data, the length between parts that exist within the target object of interest for the person in the image data, calculates a comparison length to be compared based on the image data, and calculates an index value indicating the state of the target object of interest based on the calculated length between parts and the calculated comparison length.

[0149] Even if the invention is a program (or a recording medium) or a calculation method having the above-mentioned configuration, it has the same action and effect as the above-mentioned case, and therefore can achieve the same object as the calculation device 800 having the above-mentioned index value calculation unit 833.

[0150] <Additional Notes> A part or all of the above-described embodiments can be described as follows: A calculation device and the like according to the present invention will be outlined below. However, the present invention is not limited to the following configuration.

[0151] (Appendix 1) a component calculation unit that calculates a length between parts of a person in the image data that exists within an object of interest based on the image data; a comparison length calculation unit that calculates a comparison length to be compared based on the image data; an index value calculation unit that calculates an index value indicating a state of the target object based on the inter-part length calculated by the component calculation unit and the comparison length calculated by the comparison length calculation unit; have Calculation device. (Appendix 2) The index value calculation unit calculates the index value by dividing the inter-site length by the comparison length. 2. The computing device of claim 1. (Appendix 3) The component calculation unit calculates a difference in an X-axis direction between parts present in the target object as the inter-part length, The index value calculation unit calculates an index value indicating a state of the target object based on the inter-part length calculated by the component calculation unit and the comparison length calculated by the comparison length calculation unit. 3. The computing device of claim 1 or 2. (Appendix 4) The component calculation unit calculates a difference in a Y-axis direction between parts present in the target object as the inter-part length, The index value calculation unit calculates an index value indicating a state of the target object based on the inter-part length calculated by the component calculation unit and the comparison length calculated by the comparison length calculation unit. 4. The computing device according to claim 1, (Appendix 5) The component calculation unit calculates the inter-part length based on information indicating coordinates of each part of the person in the image data recognized based on the image data. 5. The computing device according to claim 1, (Appendix 6) The comparative length calculation unit calculates the comparative length based on information indicating coordinates of each part of the person in the image data recognized based on the image data. 6. The computing device according to claim 1, (Appendix 7) The index value calculation unit acquires ground contact timing information indicating a timing at which the foot of the person in the image data touches the ground, and calculates the index value based on the part-to-part length and the comparison length calculated based on the image data specified based on the ground contact timing information. 7. The computing device according to claim 1, (Appendix 8) The comparative length calculation unit calculates the height from the head to the toes of the person in the image data as the comparative length. 8. The computing device according to claim 1, (Appendix 9) An information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; Calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length. Calculation method. (Appendix 10) In the information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; Calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length. A program for implementing processing.

[0152] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0153] Although the present invention has been described with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0154] In addition, the present invention claims the benefit of priority based on patent application No. 2021-055314, filed in Japan on March 29, 2021, and all contents described in that patent application are incorporated herein by reference. [Explanation of symbols]

[0155] 100 Judgment System 200 Smartphones 300 Judgment device 310 Communication I / F section 320 Storage section 321 trained models 322 Image information 323 Skeletal Information 324 Grounding judgment information 325 Touchdown Timing Information 326 Programs 330 Processing Unit 331 Image Acquisition Unit 332 Skeleton Recognition Department 333 Grounding judgment section 334 Touchdown Timing Judgment Unit 335 Output section 400 Calculation System 500 Calculation Device 510 Communication I / F section 520 Storage section 521 trained models 522 Image information 523 Skeletal information 524 Hand length information 525 Height Information 526 Waist bending value information 527 Program 530 Processing Unit 531 Image Acquisition Unit 532 Skeleton Recognition Department 533 Labor length calculation unit 534 Height Calculation Unit 535 Curve Calculation Unit 536 Output section 600 Calculation System 700 Calculation Device 710 Communication I / F section 720 Storage section 721 trained models 722 Image information 723 Skeletal information 724 Comparison length information 725 Foot Ingredients Information 726 Toe Data Information 727 Program 730 Processing Unit 731 Image Acquisition Unit 732 Skeleton Recognition Department 733 Comparison length calculation unit 734 Foot Component Calculation Unit 735 Toe Data Calculation Unit 736 Output section 800 Calculation Device 801 CPU 802 ROM 803 RAM 804 Programs 805 Storage device 806 Drive device 807 Communication Interface 808 Input / Output Interface 809 Bus 810 Recording media 811 Communication Network 821 Height Calculation Unit 822 Curve Calculation Unit 831 Component Calculation Unit 832 Comparison length calculation unit 833 Index value calculation unit

Claims

1. a component calculation unit that calculates a length between parts of a person in the image data that exists within an object of interest based on the image data; a comparison length calculation unit that calculates a comparison length to be compared based on the image data; an index value calculation unit that calculates an index value indicating a state of the target object based on the inter-part length calculated by the component calculation unit and the comparison length calculated by the comparison length calculation unit; having The index value calculation unit acquires ground contact timing information indicating a timing at which the foot of the person in the image data touches the ground, and calculates the index value based on the part-to-part length and the comparison length calculated based on the image data specified based on the ground contact timing information. Calculation device.

2. The index value calculation unit calculates the index value by dividing the inter-site length by the comparison length. The computing device of claim 1 .

3. The component calculation unit calculates the inter-part length based on information indicating coordinates of each part of the person in the image data recognized based on the image data. The computing device of claim 1 .

4. The comparative length calculation unit calculates the comparative length based on information indicating coordinates of each part of the person in the image data recognized based on the image data. The computing device of claim 1 .

5. The comparative length calculation unit calculates the height from the head to the toes of the person in the image data as the comparative length. The computing device of claim 1 .

6. An information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length; When calculating the index value, ground contact timing information indicating the timing at which the foot of the person in the image data touches the ground is acquired, and the index value is calculated based on the part-to-part length and the comparison length calculated based on the image data specified based on the ground contact timing information. Calculation method.

7. In the information processing device, Calculating an inter-part length between parts of a person in the image data that exists within an object of interest, based on the image data; Calculating a comparison length to be compared based on the image data; calculating an index value indicating a state of the target object based on the calculated inter-part length and the calculated comparison length; When calculating the index value, ground contact timing information indicating the timing at which the foot of the person in the image data touches the ground is acquired, and the index value is calculated based on the part-to-part length and the comparison length calculated based on the image data specified based on the ground contact timing information. A program for implementing processing.

8. A component calculation unit that calculates a length between the parts of an ankle and a toe, the ankle and the toe being parts of a foot of interest of a person in the image data, based on the image data; a comparison length calculation unit that calculates a comparison length to be compared based on the image data; an index value calculation unit that calculates an index value indicating a state of the toe of the target foot based on the inter-part length calculated by the component calculation unit and the comparative length calculated by the comparative length calculation unit; have Calculation device.

Images