Motion capture method and apparatus based on an artificial neural network model.

Abstract

This invention provides a motion capture method and apparatus using an artificial neural network model. [Solution] The motion capture method includes the steps of: S110 inputting camera footage into a first artificial neural network model; S130 generating event information relating to the movement of one or more objects in the motion capture space based on the process of utilizing the first artificial neural network model; and S150 generating first motion capture data relating to one or more objects relating to the camera footage based on the event information.

Claims

1. A method performed in a computing device for motion capture using an artificial neural network model, The stage of inputting camera images into the first artificial neural network model, Based on the process of utilizing the first artificial neural network model, the process involves generating event information relating to the changes of one or more objects in the motion capture space, Based on the event information, a step is to generate first motion capture data relating to one or more objects related to the camera image, Includes, The aforementioned event information is, The first event information relating to the first object that newly appeared in the motion capture space, The second event information relating to the second object that has left the motion capture space includes one or more of the following: Based on the process of utilizing the first artificial neural network model, the step of generating event information relating to the changes of one or more objects in the motion capture space is: When the first object newly appears in the motion capture space, the operation of generating first event information is performed, The process includes one or more steps of the following: when the second object leaves the motion capture space, an action to generate second event information; Based on the event information, the step of generating first motion capture data relating to one or more objects in the camera image is: Based on the event information, the step of determining whether or not to activate the skeleton asset corresponding to each of one or more objects in the motion capture space, The process includes the step of generating first motion capture data relating to one or more objects related to the camera image, based on the camera image and the currently activated skeleton asset. Based on the event information, the step of determining whether or not to activate each of the skeleton assets corresponding to one or more objects in the motion capture space is as follows: If the event information includes first event information, the first skeleton asset, which is the skeleton asset corresponding to the first object, is activated. If the event information includes a second event information, the process includes the step of deactivating the second skeleton asset, which is the skeleton asset corresponding to the second object. method.

2. In the method according to claim 1, The first artificial neural network model is an artificial neural network model that has been trained to generate event information when an object included in the motion capture space changes based on object detection. method.

3. In the method of claim 2, The first artificial neural network model is, The process involves generating a training dataset based on video data from a reference camera capturing the rotational movement of each object, and corresponding label data for each object. The steps include training the first artificial neural network model to identify and classify each object in the reference camera image based on the aforementioned training dataset, and the trained neural network model is... method.

4. In the method according to claim 1, Based on the event information, the step of generating first motion capture data relating to one or more objects in the camera image is: A step of generating first verification data based on a process of verifying the position of a marker for motion capture that corresponds to an object included in the aforementioned camera image, The steps include: if there is an abnormality in the first verification data, generating alarm information; method.

5. In the method according to claim 1, Furthermore, the process includes a step of generating second motion capture data, which is corrected motion capture data, based on the step of inputting the first motion capture data into the second artificial neural network model. method.

6. In the method according to claim 5, Based on the step of inputting the first motion capture data into the second artificial neural network model, the step of generating second motion capture data, which is corrected motion capture data, is: The first motion capture data includes a step of filtering the data relating to the contact points of the object, Based on the process of verifying the validity of the data relating to the contact points of the aforementioned object, a step is made to generate second verification data, The process includes the step of generating the second motion capture data based on the second verification data, method.

7. In the method according to claim 6, Based on the step of inputting the first motion capture data into the second artificial neural network model, the step of generating the second motion capture data, which is corrected motion capture data, is: A step of generating virtual motion capture data for one or more third objects included in the first motion capture data based on reference motion capture data, The process includes a step of generating second motion capture data based on a step of inputting virtual motion capture data of one or more third objects and the first motion capture data into a second artificial neural network model, method.

8. In the method according to claim 7, The step of generating virtual motion capture data for one or more third objects included in the first motion capture data, based on reference motion capture data, The steps include identifying one or more third objects included in the first motion capture data, The process includes a step of generating virtual motion capture data for one or more third objects based on a step of comparing the skeleton of a reference object included in the reference motion capture data with the skeleton of one or more third objects, method.

9. A computer program stored on a computer-readable storage medium that causes a computing device to perform actions for motion capture using an artificial neural network model, The aforementioned operation is, The operation of inputting camera images into the first artificial neural network model, Based on the process of utilizing the first artificial neural network model, an operation is performed to generate event information relating to the changes of one or more objects in the motion capture space, Based on the aforementioned event information, an operation is performed to generate first motion capture data relating to one or more objects in the camera image, Includes, The aforementioned event information is, The first event information relating to the first object that newly appeared in the motion capture space, The information includes one or more of the following: second event information relating to a second object exiting the motion capture space, Based on the process of utilizing the first artificial neural network model, the operation of generating event information related to the changes of one or more objects in the motion capture space is: When the first object newly appears in the motion capture space, the operation of generating first event information is performed, When the second object leaves the motion capture space, the operation includes one or more of the following: an operation to generate second event information, Based on the aforementioned event information, the operation to generate first motion capture data relating to one or more objects in the camera image is as follows: Based on the aforementioned event information, an action is taken to determine whether or not to activate the skeleton asset corresponding to each of one or more objects in the motion capture space, This includes the operation of generating first motion capture data relating to one or more objects related to the camera image, based on the camera image and the currently activated skeleton asset, Based on the aforementioned event information, the operation of determining whether or not to activate each of the skeleton assets corresponding to one or more objects in the motion capture space is as follows: If the event information includes the first event information, the first skeleton asset, which is the skeleton asset corresponding to the first object, is activated. If the event information includes a second event, the operation includes deactivating the second skeleton asset, which is the skeleton asset corresponding to the second object. A computer program stored on a computer-readable storage medium.

10. A computing device for motion capture using an artificial neural network model, One or more processors, memory, Includes, The one or more processors described above are The camera image is input into the first artificial neural network model. Based on the process of utilizing the first artificial neural network model, event information relating to the changes of one or more objects in the motion capture space is generated. Based on the aforementioned event information, first motion capture data relating to one or more objects in the camera image is generated. The aforementioned event information is, The first event information relating to the first object that newly appeared in the motion capture space, The information includes one or more of the following: second event information relating to a second object exiting the motion capture space, Based on the process of utilizing the first artificial neural network model, generating event information related to the changes of one or more objects in the motion capture space is: When the first object newly appears in the motion capture space, the operation of generating first event information is performed, The operation includes one or more of the following: when the second object leaves the motion capture space, an operation to generate second event information; Based on the aforementioned event information, generating first motion capture data relating to one or more objects in the camera image is: Based on the aforementioned event information, it is determined whether or not to activate the skeleton asset corresponding to each of one or more objects in the motion capture space. This includes generating first motion capture data relating to one or more objects related to the camera image based on the camera image and the currently activated skeleton asset, Based on the aforementioned event information, determining whether or not to activate the skeleton asset corresponding to each of one or more objects in the motion capture space is: If the event information includes the first event information, the first skeleton asset, which is the skeleton asset corresponding to the first object, is activated. If the event information includes a second event, the second skeleton asset, which is the skeleton asset corresponding to the second object, is deactivated. Computing device.

Images