A device for creating training data, a method for creating training data, and a program.

Abstract

To provide learning data for accurately estimating a posture even when a person wears a garment making it difficult to grasp his or her joint positions.SOLUTION: A learning data creation device comprises: an input part that receives, as an input, a moving image frame a, obtained by imaging motion of a subject X wearing a garment 3 attached with light sources 2 at predetermined portions and transmitting at least partially light beams from the light sources 2, the moving image frame a, being obtained by single imaging during lighting control of the light source 2; an image creation part 20 that creates a lighting-time moving image M1 and a non-lighting-time moving image M2 of the light sources 2 from the moving image frame a; a coordinate acquisition part 30 that acquires time-series coordinate data Z for each image frame, from the lighting-time moving image M1; and a data set creation part 40 that creates learning data by using the lighting-time moving image M1, the non-lighting-time moving image M2, and the coordinate data Z.SELECTED DRAWING: Figure 3

Claims

1. A training data creation device for creating training data for creating a posture estimation model, A video frame recording the movements of a subject wearing clothing that allows at least a portion of the light from the light sources to pass through, covering each of the light sources, wherein the video frame is received as input during the control of turning the light sources on and off, and the input unit receives the video frame acquired in one recording as input. An image creation unit that creates an on-camera video of the light source when it is lit and an off-camera video of the light source when it is not lit from the aforementioned video frames, A coordinate acquisition unit that acquires time-series coordinate data for each image frame from the video of the light source recorded in the aforementioned video of the light being lit, A dataset creation unit creates the training data, which is a dataset, using the aforementioned video of the lights on, the aforementioned video of the lights off, and the coordinate data. A training data creation device having [a certain feature].

2. The light source is a visible light LED, The aforementioned garment transmits at least visible light, The control of the on / off state of the light source is a control that causes each of the light sources to blink at predetermined intervals. The aforementioned image creation unit, An image classification unit extracts still images from the aforementioned video frames and classifies them into "on-light" images, which are still images of the light source when it is turned on, and "off-light" images, which are still images of the light source when it is turned off. The system consists of a video creation unit that collects each of the aforementioned images of the lights on in chronological order to create the aforementioned video of the lights on, and a video creation unit that collects each of the aforementioned images of the lights off in chronological order to create the aforementioned video of the lights off. A learning data creation device according to claim 1.

3. The aforementioned light source is an infrared LED, The aforementioned garment is made of a material that does not transmit visible light but transmits infrared light. The control of the light source being switched on and off means that the light source is kept on at all times, at least while the subject's movements are being filmed. The aforementioned video frame consists of a first video frame in which the light source appears to be lit, captured by an infrared light camera, and a second video frame in which the light source appears to be off, captured by an RGB camera through an infrared light cut filter. The aforementioned image creation unit, An image discrimination unit that distinguishes between the first video frame and the second video frame, It consists of a video creation unit that creates the video of the light being on from the first video frame and the video of the light being off from the second video frame, A learning data creation device according to claim 1.

4. The aforementioned light source is an infrared LED, The aforementioned garment is made of a material that does not transmit visible light but transmits infrared light. The control of the on / off state of the light source is a control that causes each of the light sources to blink at predetermined intervals. The aforementioned video frames were recorded using an event camera capable of capturing images up to the near-infrared range and which creates video frames by detecting changes in the brightness of the subject being filmed. The aforementioned image creation unit, A lighting video creation unit extracts multiple still images from the aforementioned video frames, and accumulates only the images taken while the infrared light LED is lit in chronological order to create the lighting video. The system comprises a unit for creating a video of the LED being turned off, which excludes images taken while the infrared LED is lit from the plurality of still images, and accumulates the remaining still images in chronological order to create the video of the LED being turned off. A learning data creation device according to claim 1.

5. A method for creating training data using a training data creation device for creating training data for creating a posture estimation model, A video frame recording the movements of a subject wearing clothing that allows at least a portion of the light from each of the light sources to pass through, wherein the video frame acquired in one recording while the on / off control of the light sources is being used is received as input by the input unit. The image creation unit creates a video of the light source being lit from the video frames, and a video of the light source being off. The coordinate acquisition unit acquires time-series coordinate data for each image frame from the video of the light source recorded in the video of the light being lit. The dataset creation unit creates the training data, which is a dataset, using the video of the lights on, the video of the lights off, and the coordinate data. Method for creating training data.

6. A program for causing a computer to operate the learning data creation device described in any one of claims 1 to 4.

Images