Determination device

Abstract

A determination device 400 has a determination unit 421 that acquires information acquired from each piece of time-series image data and indicating the position where a predetermined portion exists, and determines a timing when the foot contacts the ground on the basis of a change in the acquired position at which the portion exists.

Description

[Technical field]

[0001] The present invention relates to a determination device, a determination method, and a recording medium. [Background technology]

[0002] It is known to perform a gait analysis of a user based on the data.

[0003] An example of a document describing human gait analysis is Patent Document 1. Patent Document 1 describes a gait analysis device having a data acquisition unit that acquires two types of image data from a depth sensor, a skeletal information creation unit that creates skeletal information based on the image data acquired by the data acquisition unit, a correction processing unit that corrects the skeletal information created by the skeletal information creation unit, and an analysis processing unit that analyzes the user's gait using the corrected skeletal information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 170832 Summary of the Invention [Problem to be solved by the invention]

[0005] When performing various gait analyses, it is necessary to grasp the timing when the foot touches the ground. However, for example, when performing an analysis based on image data without using a depth sensor as described in Patent Document 1, it is difficult to grasp the timing of the foot touching the ground because information in the depth direction cannot be obtained. Specifically, for example, in Patent Document 1, the frame when the displacement of the Y value of the thoracic and lumbar part captured as a three-dimensional position changes from a negative value to a positive value is set as the touchdown timing. However, when performing an analysis based on image data acquired from one direction without using a depth sensor, it is not possible to obtain information in the depth direction, so it is not possible to capture the skeleton three-dimensionally. As a result, for example, in the case of image data of a person photographed from the side, the arms and shoulders on the camera side of the thoracic and lumbar part get in the way, making it difficult to recognize the thoracic and lumbar part, and in the case of image data of a person photographed from the front, a person walking from the back of the screen to the front changes in size on the screen between the back and the front, so it is not possible to use the method using the displacement of the thoracic and lumbar part as described in Patent Document 1 as it is.

[0006] As a method for determining the touchdown timing, for example, there is a method of determining the touchdown timing by laying a pressure gauge on the floor. However, in this case, the measurable section is limited by the size of the pressure gauge. Therefore, it is not possible to determine the touchdown timing without limiting the measurable section.

[0007] SUMMARY OF THE PRESENTLY PREFERRED EMBODIMENTS Accordingly, an object of the present invention is to provide a determination device, a determination method, and a recording medium that solve the problem that it is difficult to grasp the timing at which the foot touches the ground based on image data. [Means for solving the problem]

[0008] In order to achieve this object, a determination device according to one embodiment of the present disclosure includes: The device has a determination unit that obtains information indicating the position of a predetermined part from each piece of time-series image data, and determines the timing when the foot touches the ground based on the change in the position of the obtained part. The structure is as follows.

[0009] In addition, a determination method according to another aspect of the present disclosure includes: An information processing device, Information indicating the position of a predetermined part is obtained from each piece of time-series image data, and the timing at which the foot touches the ground is determined based on the change in the position of the obtained part. The structure is as follows.

[0010] In addition, a recording medium according to another embodiment of the present disclosure includes: In the information processing device, Information indicating the position of a predetermined part is obtained from each piece of time-series image data, and the timing at which the foot touches the ground is determined based on the change in the position of the obtained part. It is a computer-readable recording medium on which a program for implementing processing is recorded. Effect of the Invention

[0011] According to the above-mentioned configurations, it is possible to grasp the timing when the foot touches the ground based on the image data. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a configuration example of a determination system according to a first embodiment of the present disclosure. [Diagram 2] 1 is a diagram for explaining an example of image data acquired in a determination system; [Diagram 3] 2 is a block diagram showing an example of the configuration of the determination device shown in FIG. 1. [Figure 4] FIG. 11 is a diagram showing an example of skeletal information. [Diagram 5] FIG. 11 is a diagram showing an example of ground contact determination information. [Figure 6] 11A and 11B are diagrams for explaining examples of determination made by a ground contact determining unit; [Figure 7] 11A and 11B are diagrams for explaining an example of determination by a ground contact timing determination unit. [Figure 8]10 is a flowchart illustrating an example of a process performed by a ground contact determination unit. [Figure 9] 10 is a flowchart showing an example of a process performed by a ground contact timing determination unit. [Figure 10] FIG. 11 is a diagram illustrating an example of a hardware configuration of a determination device according to a second embodiment of the present disclosure. [Figure 11] FIG. 2 is a block diagram showing an example of the configuration of a determination device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] [First embodiment] A first embodiment of the present disclosure will be described with reference to Figs. 1 to 9. Fig. 1 is a diagram showing an example of a configuration of a determination system 100. Fig. 2 is a diagram for explaining an example of image data acquired in the determination system 100. Fig. 3 is a block diagram showing an example of a configuration of a determination device 300. Fig. 4 is a diagram showing an example of skeletal information 323. Fig. 5 is a diagram showing an example of ground contact determination information 324. Fig. 6 is a diagram for explaining an example of determination by the ground contact determination unit 333. Fig. 7 is a diagram for explaining an example of determination by the ground contact timing determination unit 334. Fig. 8 is a flowchart showing an example of processing by the ground contact determination unit 333. Fig. 9 is a flowchart showing an example of processing by the ground contact timing determination unit 334.

[0014] In the first embodiment of the present disclosure, a determination system 100 that determines the timing when the feet of a walking person land on the ground based on image data acquired using an imaging device such as a smartphone 200 will be described. The determination system 100 acquires time-series image data showing a person walking from the back side to the front side of the screen. As will be described later, the determination system 100 recognizes the position of the skeleton from each of the multiple image data. Then, the determination system 100 determines the timing when the feet land on the ground based on the degree of change in the recognized positions.

[0015] The timing of landing determined by the determination system 100 can be used, for example, when performing various gait analyses such as indices such as the angle of each joint at the time when the foot touches the floor, whether the toes are lifted from the floor, pitch, stride, stride cycle, etc. The timing of landing determined by the determination system 100 may be used for purposes other than those exemplified above.

[0016] Fig. 1 shows a configuration example of a determination system 100. Referring to Fig. 1, the determination system 100 includes, for example, a smartphone 200 and a determination device 300. As shown in Fig. 1, the smartphone 200 and the determination device 300 are connected to each other, for example, wirelessly or by wire, so as to be able to communicate with each other.

[0017] The smartphone 200 functions as an imaging device that captures an image of a person walking. The smartphone 200 may be a smartphone having general functions such as a camera function for acquiring image data, a touch panel for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.

[0018] In this embodiment, as shown in Fig. 2, the smartphone 200 captures an image of a person walking in a forward-backward direction, such as from the back of the screen to the front. In other words, the smartphone 200 acquires time-series image data showing a person walking in a forward-backward direction. The smartphone 200 also transmits the acquired image data to the determination device 300. The smartphone 200 may associate information indicating the date and time when the image data was acquired with the image data, and transmit the associated data to the determination device 300.

[0019] The determination device 300 is an information processing device that determines the timing when the feet of a walking person touch the ground based on image data acquired by the smartphone 200. For example, the determination device 300 is a server device. The determination device 300 may be a single information processing device, or may be realized, for example, on a cloud.

[0020] Fig. 3 shows an example of the configuration of the determination device 300. Referring to Fig. 3, the determination device 300 has, as main components, for example, a communication I / F unit 310, a storage unit 320, and an arithmetic processing unit 330.

[0021] The communication I / F unit 310 includes a data communication circuit and performs data communication with an external device, the smartphone 200, or the like connected via a communication line.

[0022] The storage unit 320 is a storage device such as a hard disk or a memory. The storage unit 320 stores processing information and a program 326 required for various processes in the calculation processing unit 330. The program 326 is read into the calculation processing unit 330 and executed to realize various processing units. The program 326 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 310, and is stored in the storage unit 320. Main information stored in the storage unit 320 includes, for example, a trained model 321, image information 322, skeletal information 323, ground contact determination information 324, and ground contact timing information 325.

[0023] The trained model 321 is a trained model used by the skeleton recognition unit 332 when performing skeleton recognition. The trained model 321 is generated in advance, for example, in an external device or the like by performing machine learning using teacher data such as image data containing skeleton coordinates, and is acquired from an external device or the like via the communication I / F unit 310 or the like and stored in the storage unit 320. The trained model 321 may be generated using a known method. The trained model 321 may be updated by a re-learning process using additional teacher data or the like.

[0024] The image information 322 includes time-series image data captured by a camera included in the smartphone 200. The image information 322 is generated and updated when the image acquisition unit 331 acquires image data, for example.

[0025] For example, image data is associated with identification information for identifying the image data in the image information 322. The image information 322 may include information other than the above examples, such as information indicating the date and time when the smartphone 200 acquired the image data.

[0026] The skeleton information 323 includes information indicating the coordinates (i.e., the position) of each part of the person recognized by the skeleton recognition unit 332. The skeleton information 323 includes, for example, information indicating the coordinates of each part of each piece of time-series image data. For example, the skeleton information 323 is generated / updated as a result of processing by the skeleton recognition unit 332.

[0027] Fig. 4 shows an example of skeletal information 323. Referring to Fig. 4, in skeletal information 323, for example, identification information and position information of each part are associated with each other. Here, the identification information is information indicating image data used when recognizing the skeletal information, etc. The identification information may correspond to identification information for identifying image data in image information 322. Furthermore, the position information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

[0028] The parts included in the position information of each part correspond to the trained model 321. For example, in FIG. 4, the parts are pelvis, center of spine, ..., right knee, left knee, ..., right ankle, left ankle, ... are shown as examples. The position information of each part can include about 30 parts such as the right shoulder, . . ., the left elbow, . . . (other parts than those exemplified may also be included). In this embodiment, the position information of each part includes information indicating the coordinates of at least the right ankle and the left ankle. The parts included in the position information of each part may be other than those exemplified in FIG. 4 and the like.

[0029] The ground contact determination information 324 includes information indicating the result of the determination made by the ground contact determination unit 333. For example, the ground contact determination information 324 is generated and updated as the result of the determination made by the ground contact determination unit 333.

[0030] FIG. 5 shows an example of the ground contact judgment information 324. As shown in FIG. 5, for example, in the ground contact judgment information 324, identification information and ground contact judgment results are associated with each other. Here, the identification information is information indicating image data for which ground contact judgment has been performed, etc. The identification information may correspond to identification information for identifying image data in the image information 322. Furthermore, the ground contact judgment result indicates the result of the ground contact judgment. For example, the ground contact judgment result includes "left foot" indicating the result of judging that the left foot is on the ground, "right foot" indicating the result of judging that the right foot is on the ground, "-" indicating that it has been judged that the foot is excluded from the ground contact judgment, and the like. The ground contact judgment result may indicate other than the above examples.

[0031] The ground contact timing information 325 includes information indicating the result of the determination by the ground contact timing determination unit 334. For example, the ground contact timing information 325 is generated and updated as the result of the determination by the ground contact timing determination unit 334.

[0032] For example, the ground contact timing information 325 includes information indicating the timing when the foot touched the ground. Here, the information indicating the timing when the foot touched the ground is, for example, at least one of identification information indicating image data determined to have touched the ground, information indicating the acquisition date and time of the image data determined to have touched the ground, etc. The ground contact timing information 325 may be, for example, information indicating the foot touching the ground and information indicating the timing when the foot touched the ground in association with each other.

[0033] The arithmetic processing unit 330 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 330 reads and executes the program 326 from the storage unit 320, thereby implementing various processing units by making the above hardware and the program 326 work together. Major processing units implemented by the arithmetic processing unit 330 include, for example, an image acquisition unit 331, a skeleton recognition unit 332, a ground contact determination unit 333, a ground contact timing determination unit 334, and an output unit 335.

[0034] The image acquisition unit 331 acquires image data acquired by the smartphone 200 from the smartphone 200 via the communication I / F unit 310. For example, the image acquisition unit 331 acquires time-series image data from the smartphone 200. In addition, the image acquisition unit 331 stores the acquired image data in the storage unit 320 as image information 322.

[0035] The skeleton recognition unit 332 recognizes the skeleton of the person to be measured for walking posture in the image data by using the trained model 321. For example, the skeleton recognition unit 332 recognizes the skeleton for each piece of time-series image data. The skeleton recognition unit 332 may be configured to recognize the skeleton for image data extracted from the time-series image data based on a predetermined condition. For example, the skeleton recognition unit 332 recognizes each part such as the upper part of the spine, the right shoulder, the left shoulder, the right elbow, the left elbow, the right wrist, the left wrist, the right hand, the left hand, ...... In addition, the skeleton recognition unit 332 calculates the coordinates in the screen data of each recognized part. Then, the skeleton recognition unit 332 associates the recognition and calculation results with identification information for identifying the image data, and stores them in the storage unit 320 as skeleton information 323.

[0036] The parts recognized by the skeleton recognition unit 332 correspond to the trained model 321 (teaching data used when training the trained model 321). Therefore, the skeleton recognition unit 332 may recognize parts other than those exemplified above according to the trained model 321.

[0037] The ground contact determination unit 333 determines which foot is on the ground based on the skeleton of the person indicated by the skeleton information 323. For example, the ground contact determination unit 333 determines which foot, the right foot or the left foot, of a walking person is on the ground based on the change in the coordinates of the right ankle and the left ankle included in the skeleton information 323. For example, the ground contact determination unit 333 acquires information indicating the coordinates of the ankle by referring to the skeleton information 323. Then, when the change in the coordinates of the ankle satisfies a predetermined condition, the ground contact determination unit 333 determines that the foot having the ankle that satisfies the condition is on the ground. In addition, the ground contact determination unit 333 stores information indicating the result of the determination in the storage unit 320 as ground contact determination information 324.

[0038] FIG. 6 shows an example of changes in the Y coordinates (the vertical coordinates of the image data) of the right and left ankles. For example, in FIG. 6, the X axis shows values ​​according to time, and the Y axis shows the vertical coordinates (Y coordinates) in the image data. For example, in the case of FIG. 6, the larger the Y axis value, the lower the right or left ankle is located in the image data (i.e., the closer the right or left ankle is located to the front of the screen).

[0039] Generally, when a person walks, the following motions are repeated alternately: one of the right and left feet remains on the ground while the other moves forward, both the right and left feet touch the ground, one of the right and left feet moves forward while the other remains on the ground, both the right and left feet touch the ground. In other words, in the case of this embodiment, it is assumed that the following states are repeated: the Y coordinate of the right ankle is constant and the Y coordinate of the left ankle increases, the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant, the Y coordinate of the left ankle is constant and the Y coordinate of the right ankle increases, and the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant.

[0040] Therefore, the ground contact determination unit 333 determines which foot is on the ground based on the change in the Y coordinate of the right ankle and the left ankle. In other words, when it can be evaluated that the Y coordinate of the ankle is constant, the ground contact determination unit 333 determines that the foot having the ankle is on the ground. Specifically, for example, the ground contact determination unit 333 compares the Y coordinate of the ankle in the image data to be determined with the Y coordinate of the ankle in the image data immediately preceding the determination target in the time series image data. Then, when the displacement (amount of change) from the Y coordinate of the ankle in the immediately preceding image data is equal to or less than a predetermined value, the ground contact determination unit 333 determines that the Y coordinate is constant and that the foot having the ankle with the constant Y coordinate is on the ground. Note that the ground contact determination unit 333 may determine that the Y coordinate is constant by a method other than the above example. Also, the range of displacement in which the ground contact determination unit 333 determines that the Y coordinate is constant may be set arbitrarily.

[0041] In addition, the ground contact determination unit 333 may be configured to determine that the foot other than the foot determined to be on the ground starts to move based on the Y coordinate of the ankle, and to exclude the foot from the ground contact determination. For example, in the case of walking, the right foot generally touches the ground, and then the left foot starts to move. The ground contact determination unit 333 may determine that the left foot starts to move when the Y coordinate of the left ankle starts to increase after the Y coordinate of the right ankle becomes constant. Specifically, for example, the ground contact determination unit 333 can determine that the left foot starts to move and exclude the left ankle from the ground contact determination when the displacement of the Y coordinate of the left ankle from the previous image data in the time series image data exceeds the displacement of the Y coordinate of the right ankle from the previous image data after the Y coordinate of the right ankle becomes constant.

[0042] Generally, when a person walks, the foot that will newly touch the ground moves forward more than the foot that has already touched the ground. That is, in this embodiment, the Y coordinate of the ankle of the foot that will newly touch the ground is larger than the Y coordinate of the ankle of the foot that has already touched the ground. Therefore, the ground contact determination unit 333 may be configured to determine whether or not the ankle is touching the ground by focusing only on the ankle with the larger Y coordinate out of the right ankle and the left ankle.

[0043] Furthermore, the ground contact determination unit 333 may be configured to refer to the coordinates of the knee in addition to the coordinates of the ankle when making a ground contact determination. For example, the ground contact determination unit 333 may be configured to determine that the foot, which is to be determined whether or not it is on the ground, is excluded from the ground contact determination when the change in the coordinate of the knee satisfies a predetermined condition. Specifically, for example, the ground contact determination unit 333 may be configured to determine that the foot, which is to be determined whether or not it is on the ground, is excluded from the ground contact determination when the displacement of the Y coordinate of the knee from the previous image data in the time series image data is 0 or less.

[0044] For example, as described above, the ground contact determination unit 333 performs the ground contact determination based on the change in the Y coordinate of the ankle. Note that the ground contact determination unit 333 may be configured to perform the ground contact determination based on the change in the skeleton other than the above-mentioned examples, such as using the Y coordinate of the toe.

[0045] The contact timing determination unit 334 determines the timing when the foot touches the ground based on the contact state of the foot determined based on the skeleton of the person indicated by the skeleton information 323. For example, the contact timing determination unit 334 determines the timing when the foot touches the ground based on the change in the foot touching the ground indicated by the contact determination information 324. Then, the contact timing determination unit 334 stores the determination result in the storage unit 320 as contact timing information 325.

[0046] For example, the contact timing determination unit 334 determines that the foot has touched the ground at the timing when the contact foot indicated by the contact determination result is switched to another foot. For example, referring to FIG. 7, the "left foot" is contacted at identification information "124" and "125", and is excluded from contact determination at identification information "126" and "127". After that, it is determined that the "right foot" is contacted at identification information "128". That is, in the case of FIG. 7, the contact foot is switched from the left foot to the right foot at the timing of identification information "128". Therefore, the contact timing determination unit 334 determines that the right foot has touched the ground at the timing of identification information "128".

[0047] The output unit 335 outputs the ground contact timing information 325, the ground contact determination information 324, etc. For example, the output unit 335 can output at least one of the above information to an external device, the smartphone 200, or the like.

[0048] The above is an example of the configuration of the determination device 300. Next, the operation of the determination device 300 will be described with reference to Figs.

[0049] First, an example of the operation of the ground contact determination unit 333 will be described with reference to Fig. 8. Referring to Fig. 8, the ground contact determination unit 333 compares the Y coordinate of the knee in the image data to be determined with the Y coordinate of the knee in the image data immediately preceding the determination target in the time series image data (step S101). If the displacement from the Y coordinate of the knee in the immediately preceding image data exceeds 0 (step S101, Yes), the ground contact determination unit 333 checks whether the displacement of the ankle satisfies a condition (step S102).

[0050] If the ankle displacement satisfies the condition (step S102, Yes), the ground contact determination unit 333 determines that the foot that satisfies the condition is on the ground (step S103). On the other hand, if the displacement of the Y coordinate of the knee is 0 or less (step S101, No) or if the ankle displacement does not satisfy the condition (step S102, No), the ground contact determination unit 333 determines that the foot that does not satisfy the condition is not on the ground.

[0051] The above is an example of the operation of the ground contact determination unit 333. The ground contact determination unit 333 can determine that the displacement of the ankle satisfies the condition, for example, when the Y coordinate of the ankle can be evaluated as being constant based on the change in the Y coordinate, when it is not determined that the foot other than the foot determined to be on the ground has started to move, or when the change in the knee coordinate satisfies the condition. The ground contact determination unit 333 may be configured to identify an ankle that focuses on the Y coordinate of the ankle, and to perform the above-mentioned check on the identified ankle.

[0052] Next, an example of the operation of the ground contact timing determination unit 334 will be described with reference to Fig. 9. Referring to Fig. 9, the ground contact timing determination unit 334 checks whether the ground contact foot indicated by the ground contact determination result has changed by referring to the ground contact timing information 325 (step S201).

[0053] If the foot has changed (step S201, Yes), the ground contact timing determination unit 334 determines that the changed timing is the ground contact timing (step S202). On the other hand, if the foot on the ground shown by the determination result has not changed (step S201), the ground contact timing determination unit 334 does not perform the above determination.

[0054] The above is an example of the processing performed by the contact timing determination unit 334.

[0055] Thus, the determination device 300 has the skeleton recognition unit 332, the ground contact determination unit 333, and the ground contact timing determination unit 334. With this configuration, the ground contact timing determination unit 334 can determine the timing at which the foot touches the ground based on the determination result of the ground contact determination unit 333 using the coordinates of the skeleton recognized by the skeleton recognition unit 332. In other words, with the above configuration, it becomes possible to grasp the timing at which the foot touches the ground based on the image data.

[0056] In addition, the determination device 300 determines the landing timing by focusing on the part close to the foot that is actually on the ground. Therefore, the determination device 300 can perform a more realistic determination compared to the method using the displacement of the thoracic and lumbar region.

[0057] The determination device 300 may be configured to determine the landing timing based on the results of skeleton recognition by an external device that performs skeleton recognition. When configured in this way, the determination device 300 does not need to have the function of the skeleton recognition unit 332.

[0058] [Second embodiment] Next, a second embodiment of the present disclosure will be described with reference to Fig. 10 and Fig. 11. In the second embodiment of the present disclosure, an overview of the configuration of a determination device 400, which is an information processing device, will be described.

[0059] Fig. 10 shows an example of a hardware configuration of the determination device 400. Referring to Fig. 10, the determination device 400 has, as an example, the following hardware configuration. ·CPU(Central Processing Unit)401(Arithmetic unit) ROM (Read Only Memory) 402 (storage device) RAM (Random Access Memory) 403 (storage device) · Programs 404 loaded into RAM 403 A storage device 405 for storing the program group 404 A drive device 406 for reading and writing data from a recording medium 410 outside the information processing device A communication interface 407 for connecting to a communication network 411 outside the information processing device Input / output interface 408 for inputting and outputting data Bus 409 connecting each component

[0060] 11 by the CPU 401 acquiring the program group 404 and executing it. The program group 404 is stored in advance in the storage device 405 or the ROM 402, for example, and the CPU 401 loads the program group 404 into the RAM 403 or the like and executes it as necessary. The program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in advance in the recording medium 410, and the drive device 406 may read out the program and supply it to the CPU 401.

[0061] 10 shows an example of a hardware configuration of the determination device 400. The hardware configuration of the determination device 400 is not limited to the above. For example, the determination device 400 may be configured with a part of the above configuration, such as not having the drive device 406.

[0062] The determination unit 421 acquires information indicating the position of a predetermined part, which is acquired from each piece of time-series image data, and determines the timing when the foot touches the ground based on the change in the position of the acquired part.

[0063] In this way, the determination device 400 has the determination unit 421. With this configuration, the determination unit 421 can determine the timing when the foot touches the ground based on the change in the position of the part. This makes it possible to grasp the timing when the foot touches the ground based on the image data.

[0064] An information processing device such as the above-mentioned determination device 400 can be realized by incorporating a predetermined program into the information processing device. Specifically, the program according to another embodiment of the present invention is a program for implementing a process in which the information processing device acquires information indicating the position of a predetermined part, acquired from each piece of time-series image data, and determines the timing at which the foot touches the ground based on the change in the position of the acquired part.

[0065] In addition, the determination method executed by the above-mentioned information processing device is such that the information processing device acquires information indicating the position of a predetermined part, obtained from each piece of image data in a time series, and determines the timing when the foot touches the ground based on the degree of change in the position of the acquired part.

[0066] Even if the invention is a program (or a recording medium) or a determination method having the above-mentioned configuration, it has the same action and effect as the above-mentioned case, and therefore can achieve the above-mentioned object of the present invention.

[0067] <Additional Notes> A part or all of the above-described embodiments can be described as follows: The following provides an overview of the determination device and the like in the present invention. However, the present invention is not limited to the following configuration.

[0068] (Appendix 1) The device has a determination unit that obtains information indicating the position of a predetermined part from each piece of time-series image data, and determines the timing when the foot touches the ground based on the change in the position of the obtained part. Judgment device. (Appendix 2) a determination unit for determining whether a foot is on the ground based on a change in the position of the part; The determination unit determines a timing when the foot touches the ground based on a result of the determination by the determination unit. 2. The determination device according to claim 1. (Appendix 3) The determination unit determines that a foot has landed on the ground at a timing when it is determined that the foot on the ground has been switched based on a result of the determination made by the determination unit. 3. The determination device according to claim 2. (Appendix 4) The information indicating the position of the part includes information indicating the position of the ankle, which is determined by recognizing the skeleton of the person in the image data; The determining unit determines which foot is on the ground based on the change in the position of the ankle. 4. The determination device according to claim 2 or 3. (Appendix 5) The determination unit determines that the foot having the ankle is on the ground when it can be evaluated that the position of the ankle is constant based on the change in the position of the ankle. 5. The determination device according to claim 4. (Appendix 6) The determination unit determines whether a foot other than the foot determined to be on the ground based on a change in the ankle position has started to move and, when such a foot has started to move, the determination unit determines whether the foot is excluded from the ground contact determination. 6. The determination device according to claim 4 or 5. (Appendix 7) The determination unit determines a foot to be focused on based on the position of the ankle, and determines whether the foot to be focused on is in contact with the ground. 7. The determination device according to claim 4, (Appendix 8) The information indicating the position of the part includes information indicating the position of the knee determined by recognizing the skeleton of the person in the image data, The determining unit determines which foot is on the ground based on the change in the position of the ankle and the change in the position of the knee. 8. The determination device according to claim 4, (Appendix 9) An information processing device, Information indicating the position of a predetermined part is obtained from each piece of time-series image data, and the timing at which the foot touches the ground is determined based on the change in the position of the obtained part. Judgment method. (Appendix 10) In the information processing device, Information indicating the position of a predetermined part is obtained from each piece of time-series image data, and the timing at which the foot touches the ground is determined based on the change in the position of the obtained part. A program for implementing processing.

[0069] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0070] Although the present invention has been described with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0071] In addition, the present invention claims the benefit of priority based on patent application No. 2021-055312, filed in Japan on March 29, 2021, and all contents described in that patent application are incorporated herein by reference. [Explanation of symbols]

[0072] 100 Judgment System 200 Smartphones 300 Judgment device 310 Communication I / F section 320 Storage section 321 trained models 322 Image information 323 Skeletal Information 324 Grounding judgment information 325 Touchdown Timing Information 326 Programs 330 Processing Unit 331 Image Acquisition Unit 332 Skeleton Recognition Department 333 Grounding judgment section 334 Touchdown Timing Judgment Unit 335 Output section 400 Judgment device 401 CPU 402 ROM 403 RAM 404 Programs 405 Storage device 406 Drive Unit 407 Communication Interface 408 Input / Output Interface 409 Bus 410 Recording media 411 Communication Network 421 Judgment section

Claims

1. A determination unit that obtains information indicating the position of a predetermined part, the ankle, obtained according to the result of skeletal recognition using a trained model for each piece of time-series image data, and determines which foot is on the ground based on the change in the position of the obtained part; a determination unit that determines a timing when the foot touches the ground based on a result of the determination by the determination unit; having The determination unit determines that the foot has touched the ground at a timing when the foot that is on the ground is switched as a result of the ground contact determination by the determination unit. Judgment device.

2. The determination unit determines that the foot having the ankle is on the ground when it can be evaluated that the position of the ankle is constant based on the change in the position of the ankle. The determination device according to claim 1 .

3. The determination unit determines whether a foot other than the foot determined to be on the ground based on a change in the ankle position has started to move and, when such a foot has started to move, the determination unit determines whether the foot is excluded from the ground contact determination. The determination device according to claim 1 .

4. The determination unit determines a foot to be focused on based on the position of the ankle, and determines whether the foot to be focused on is in contact with the ground. The determination device according to claim 1 .

5. The information indicating the position of the part includes information indicating the position of the knee determined by recognizing the skeleton of the person in the image data, The determining unit determines which foot is on the ground based on the change in the position of the ankle and the change in the position of the knee. The determination device according to claim 1 .

6. An information processing device, acquiring information indicating the position of the ankle, which is a predetermined part, according to the result of skeletal recognition using the trained model for each piece of time-series image data, and determining which foot is in contact with the ground based on the change in the position of the acquired part; Based on the result of the judgment, the timing of the foot contact is determined, When making the judgment, it is judged that the foot has touched the ground at the timing when the foot that is on the ground changes as indicated by the result of the ground contact judgment. Judgment method.

7. In the information processing device, acquiring information indicating the position of the ankle, which is a predetermined part, according to the result of skeletal recognition using the trained model for each piece of time-series image data, and determining which foot is in contact with the ground based on the change in the position of the acquired part; Based on the result of the judgment, the timing of the foot contact is determined, When making the judgment, it is judged that the foot has touched the ground at the timing when the foot that is on the ground changes as indicated by the result of the ground contact judgment. A program for implementing processing.

Images