Posture analysis device, posture analysis method, and program

Abstract

The pose analysis device (2000) acquires a target image (10) and estimates the pose of each person. The target image (10) includes a plurality of persons imaged by a camera. The persons perform arbitrary actions such as playing, exercising, and musical instrument performance. The pose analysis device (2000) classifies the persons into a plurality of pose groups based on the pose of the persons, and outputs group information (20) including information regarding at least one pose group.

Description

Technical Field

[0001] The present invention generally relates to a posture analysis device, a posture analysis method, and a non-transitory computer-readable storage medium.

Background Art

[0002] There is a technology for analyzing images of people. Patent Document 1 discloses a system that analyzes images of students in a class to determine the current class situation such as concentration. The class situation is determined by comparing features (for example, postures) obtained from pre-stored class situation sample images with features of the students in the class imaged in the image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 does not disclose a technique for handling images of multiple people. The object of the present disclosure is to provide a novel technique for analyzing the postures of people using images of multiple people.

Means for Solving the Problems

[0005] The posture analysis device provided by the present disclosure includes at least one storage element configured to store instructions, and at least one processor. The at least one processor is configured to execute the instructions to acquire a target image in which two or more people are imaged, estimate the posture of each person, classify the people into two or more posture groups based on the posture of the people, and output group information including information regarding at least one of the plurality of posture groups.

[0006] The posture analysis method further provided by the present disclosure is executed by a computer. The posture analysis method includes obtaining a target image in which two or more persons are imaged, estimating the posture of each person, classifying the persons into two or more posture groups based on the posture of the persons, and outputting group information including information regarding at least one of the plurality of posture groups.

[0007] The non-transitory computer-readable medium further provided by the present disclosure stores a program. The program causes a computer to obtain a target image in which two or more persons are imaged, estimate the posture of each person, classify the persons into two or more posture groups based on the posture of the persons, and output group information including information regarding at least one of the plurality of posture groups.

Advantages of the Invention

[0008] According to the present invention, a novel technique for analyzing the postures of persons using an image in which a plurality of persons are imaged is provided.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0010] Embodiments according to the present disclosure will be described below with reference to the drawings. The same reference numerals are assigned to the same elements throughout the drawings, and redundant descriptions will be omitted as necessary. Also, predetermined information (for example, a predetermined value or a predetermined threshold) is stored in advance in a storage device accessible to a computer that uses the information, unless otherwise specified.

[0011] <Overview> FIG. 1 is a diagram showing an overview of the posture analysis device 2000 according to the embodiment. Note that the overview shown in FIG. 1 shows an example of the operation of the posture analysis device 2000 for easy understanding of the posture analysis device 2000, and does not limit or narrow the scope of the operation of the posture analysis device 2000.

[0012] The posture analysis device 2000 is configured to classify the persons imaged in the target image 10 into groups according to the postures of the persons, and output information (group information 20) regarding the classification result. The target image 10 is image data that visibly includes a plurality of persons, for example, an RGB image or a grayscale image.

[0013] The persons imaged in the target image 10 perform arbitrary actions. For example, the person performs a performance such as figure skating or dance. In other examples, the person performs an exercise such as yoga. In other examples, the person plays an instrument such as a guitar or a piano. In other examples, the person participates in a school class. In other examples, the person performs an operation such as assembling parts in a factory or patrolling inside a building.

[0014] To classify the people captured in the target image 10, the pose analysis device 2000 can operate as follows. The pose analysis device 2000 acquires the target image 10 and estimates the pose of each person captured in the target image 10. Next, the pose analysis device 2000 classifies the people into groups called "pose groups" based on the estimated poses of the people. Then, the pose analysis device 2000 outputs group information 20 including information about one or more pose groups.

[0015] Note that the pose analysis device 2000 may handle a plurality of target images 10 including different people in parallel. In this case, a plurality of cameras may be installed to image different areas (for example, different areas of a lesson room where people are receiving performance lessons), and each camera may be configured to generate a target image 10. The pose analysis device 2000 may analyze each of these target images 10 to detect people and classify the detected people into pose groups based on their poses.

[0016] Note that, for the sake of simplicity, unless otherwise specified, the pose analysis device 2000 is assumed to handle a single target image 10. The pose analysis device 2000 that handles a plurality of target images 10 can operate in the same manner as the pose analysis device 2000 that handles a single target image 10 unless otherwise specified.

[0017] <Example of effects> According to the pose analysis device 2000 of the present embodiment, the poses of the people captured in the target image 10 are estimated, and the people are classified into pose groups based on the poses. Thereby, a new technique for analyzing the poses of people using an image in which a plurality of people are captured is provided.

[0018] In addition, the posture analysis device 2000 outputs group information 20 indicating information regarding at least one posture group. The information regarding the posture group is valid and useful in various meanings. Briefly speaking, the viewer of the group information 20 can distinguish the person belonging to the posture group from other persons, and thereby can find a group of persons having some characteristics with each other.

[0019] For example, although details will be described later, the posture analysis device 2000 can classify persons based on the quality of postures (for example, the degree of similarity to an ideal posture), and output the group information 20 indicating the posture group with the lowest quality of postures. With this group information 20, the viewer of the group information 20 can recognize a person whose performance quality is lower than that of other persons. Suppose that the viewer of the group information 20 is a performance trainer, and the person imaged in the target image 10 is a person receiving training. In this case, the posture group with the lowest quality of performance can be handled by the trainer as a group of persons to whom the trainer should pay attention and give detailed feedback.

[0020] Other usefulness and effectiveness of the group information 20 will be described later.

[0021] Hereinafter, a more detailed description of the posture analysis device 2000 will be given.

[0022] <Example of functional configuration> FIG. 2 is a block diagram showing an example of the functional configuration of the posture analysis device 2000 according to the present embodiment. The posture analysis device 2000 includes an acquisition unit 2020, an estimation unit 2040, a classification unit 2060, and an output unit 2080. The acquisition unit 2020 acquires the target image 10. The estimation unit 2040 estimates the posture of each person imaged in the target image 10. The classification unit 2060 classifies persons into posture groups based on the estimated postures of the persons. The output unit 2080 outputs the group information 20.

[0023] <Example of hardware configuration> The posture analysis device 2000 may be implemented by one or more computers. Each of the one or more computers may be a dedicated computer manufactured to implement the posture analysis device 2000, or may be a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0024] The posture analysis device 2000 may also be implemented by installing an application on one or more computers. The application is realized by a program for causing the one or more computers to function as the posture analysis device 2000. In other words, the program is one in which the functional components of the posture analysis device 2000 illustrated in FIG. 2 are implemented.

[0025] FIG. 3 is a block diagram showing an example of the hardware configuration of a computer 1000 that realizes the posture analysis device 2000. In FIG. 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output (I / O) interface 1100, and a network interface 1120.

[0026] Bus 1020 is a data transmission path for the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 to send and receive data from each other. The processor 1040 is a processor such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), or FPGA (Field-Programmable Gate Array). The memory 1060 is a main memory element such as RAM (Random Access Memory) or ROM (Read Only Memory). The storage device 1080 is an auxiliary storage element such as a hard disk, SSD (Solid State Drive), or memory card. The input / output interface 1100 is an interface between the computer 1000 and peripheral devices (such as a keyboard, mouse, or display device). The network interface 1120 is an interface between the computer 1000 and a network. The network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0027] The hardware configuration of the computer 1000 is not limited to that shown in FIG. 3. For example, as described above, the posture analysis device 2000 may be implemented by a plurality of computers. In this case, those computers may be connected to each other via a network.

[0028] <Flow of processing> FIG. 4 is a flowchart showing an example of the flow of processing executed by the posture analysis device 2000 of the embodiment. The acquisition unit 2020 acquires the target image 10 (S102). The estimation unit 2040 estimates the posture of the person captured in the target image 10 (S104). The classification unit 2060 classifies the person into posture groups based on the posture (S106). The output unit 2080 outputs the group information 20 (S108).

[0029] <Acquisition of target image 10: S102> The acquisition unit 2020 acquires the target image 10 (S102). As described above, the target image 10 includes one or more persons. The persons captured in the target image 10 are doing anything. For example, the person performs a performance such as figure skating or dance. In other examples, the person performs exercises such as yoga. In other examples, the person plays musical instruments such as guitar or piano. In other examples, the person attends school classes. In other examples, the person performs tasks such as assembling parts in a factory or patrolling inside a building.

[0030] In some embodiments, the target image 10 is one (video frame) of a plurality of time-series images that constitute video data called "target video". In this case, the acquisition unit 2020 can acquire one or more video frames that constitute the target video and use the acquired video frames as the target image 10. Note that it is not necessary to use all the video frames of the target video as the target image 10. For example, the acquisition unit 2020 acquires video frames (for example, every 10 frames) at a predetermined interval from the target video as the target image 10.

[0031] Also, the acquisition unit 2020 may divide the target video into a plurality of sections and acquire one or more video frames from each section as the target image 10. The target video may be divided into sections based on the length of time. Specifically, the target video can be divided into sections of a predetermined time length. In other examples, the acquisition unit 2020 recognizes a plurality of scenes captured in the target video and divides the target video into sections representing the recognized scenes. Suppose a figure skating performance is captured in the target video. In this case, since the target video may include scenes such as jumps, spins, and steps, the acquisition unit 2020 divides the target video into sections such as jumps, spins, and steps. Note that there are various techniques for recognizing scenes from video data, and any of these techniques can be applied to the acquisition unit 2020 to recognize scenes from the target video.

[0032] There are various methods for acquiring the target image 10. In some embodiments, the target image 10 is pre-stored in a storage device in a state that can be acquired by the posture analysis device 2000. In this case, the acquisition unit 2020 can access the storage device to acquire the target image 10. In other embodiments, the target image 10 may be transmitted from another computer such as a camera that generates the target image 10. In this case, the acquisition unit 2020 can receive and acquire the target image 10.

[0033] When the acquisition unit 2020 acquires a target video, the target video may be acquired in the same manner as the target image 10. In another example, the acquisition unit 2020 may acquire a target video generated in real time. Specifically, the video camera that generates the target video can repeatedly execute "imaging the surrounding scenery to generate video frames of the target video and outputting the generated video frames to the posture analysis device 2000". In this case, the acquisition unit 2020 receives the video frames sequentially transmitted from the video camera, and sets the time series of the received video frames as the target video.

[0034] <Posture Estimation: S104> The estimation unit 2040 estimates the posture of each person imaged in the target image 10 (S104). There are various methods for posture estimation, and any of these methods can be applied to the estimation unit 2040. For example, the estimation unit 2040 detects the positions of characteristic parts (such as the neck, eyes, shoulders, etc.) of the human body from the target image 10 as key points. Then, the estimation unit 2040 divides the key points into groups (referred to as key point groups) that include key points belonging to the same person, and estimates the posture of each person based on the key point group corresponding to that person.

[0035] A person's pose can be classified into any of a predetermined type of pose (such as a figure skating jump, spin, or step). In this case, a specific person's pose is represented by a pair of the person's keypoint group and a type label indicating the type of the person's pose. To recognize the type of a person's pose, the estimation unit 2040 can include a classification model configured to obtain a set of the person's keypoints (i.e., the keypoint group) and output a type label indicating the type of the person's pose. The classification model can be implemented by a machine learning-based model such as a neural network.

[0036] As described later, the classification unit 2060 may classify a person into a pose group using the time series of the person's poses instead of a single pose of the person. In this case, the estimation unit 2040 obtains the time series of the poses of each person by estimating the pose of the person from each target image 10 using the time series of the target image 10. Note that the time series of poses can also be called "motion". Therefore, when classifying a person using the time series of the person's poses, it can be said that the pose analysis device 2000 classifies the person based on the motion of the person.

[0037] <Classification of Persons Based on Poses: S106> The classification unit 2060 classifies a person into a pose group based on the pose of the person (S106). Hereinafter, an example of classifying a person based on a pose will be described.

[0038] <<Example 1>> In some embodiments, a person can be classified based on the degree of similarity of the person's pose to a predefined reference pose. The reference pose can be defined by a series of keypoints representing an ideal pose. In this case, the higher the similarity of the person's pose to the reference pose, the higher the quality of the pose.

[0039] To represent how similar a person's pose is to the reference pose, the classification unit 2060 can calculate a similarity score for each person. The similarity score of a specific person can be a value representing the similarity between the person's pose and the reference pose.

[0040] There are various methods for quantifying the similarity between two postures, and one of these methods can be applied to the classification unit 2060 to calculate a similarity score. Briefly speaking, the similarity between a person's posture and a reference posture can be represented by the similarity between the spatial arrangement of the keypoints within the keypoint group of that person and the spatial arrangement of the keypoints of the reference posture.

[0041] In some embodiments, the classification unit 2060 includes a machine learning-based feature extractor such as a neural network configured to obtain a keypoint group as an input and output a feature quantity (for example, a feature quantity of the spatial arrangement of the keypoints in the keypoint group) of the posture represented by the keypoint group. In this case, the classification unit 2060 inputs the keypoint group of the person into the feature extractor to obtain the feature quantity of the person's posture. Also, the classification unit 2060 inputs the keypoint group of the reference posture into the feature extractor to obtain the feature quantity of the reference posture. Then, the classification unit 2060 calculates, as a similarity score, a value representing the similarity between the feature quantity of the person's posture and the feature quantity of the reference posture.

[0042] As described above, in some embodiments, the classification unit 2060 can classify a person into a posture group using the time series (motion) of the person's posture. In this case, a time series of a reference posture called a "reference motion" is prepared in advance. The reference motion can be represented by a time series of keypoint groups each representing a reference posture at a certain point in time. The classification unit 2060 calculates a similarity score representing the similarity between the motion of the person and the reference motion for each person.

[0043] There are various methods for quantifying the similarity between two motions, and one of these methods can be applied to the classification unit 2060. Briefly speaking, the similarity between a person's motion and a reference motion can be represented by the similarity between the time series of the spatial arrangement of the keypoints of that person and the time series of the spatial arrangement of the keypoints of the reference motion.

[0044] In some embodiments, the classification unit 2060 includes a machine learning-based feature extractor such as a neural network configured to obtain a time series of keypoint groups as input and output feature amounts of the actions represented by the keypoint groups. In this case, the classification unit 2060 inputs a plurality of keypoint groups of a person into the feature extractor to obtain the feature amounts of the actions of the person. Further, the classification unit 2060 inputs a plurality of keypoint groups of a reference action into the feature extractor to obtain the feature amounts of the reference action. Then, the classification unit 2060 calculates, as a similarity score, a value representing the degree of similarity between the feature amounts of the person's action and the feature amounts of the reference action.

[0045] Based on the similarity score of the person, the person is classified into a posture group. For example, the classification unit 2060 can generate a predetermined number of posture groups initialized to be empty. Each posture group is associated with a range of similarity scores called a "score range". The score ranges are defined so as not to overlap with each other.

[0046] Assume that the entire range of the similarity score S is 0 <= S < 100 and two posture groups GP1 and GP2 are defined. In this case, the posture groups GP1 and GP2 can be defined as a posture group GP1 with a score range of 0 <= S < 50 and a posture group GP2 with a score range of 50 <= S <= 100.

[0047] The classification unit 2060 determines, for each person, a score range including the similarity score of that person, and assigns that person to a posture group corresponding to the determined score range. Suppose there are the above two posture groups GP1 and GP2. Further, suppose there are person P1 with a similarity score of 20, person P2 with a similarity score of 70, person P3 with a similarity score of 60, person P4 with a similarity score of 45, and person P5 with a similarity score of 10. In this case, persons P1, P4, and P5 are assigned to the posture group GP1 because their similarity scores are within the score range of 0 <= S < 50, while persons P2 and P3 are assigned to the posture group GP2 because their similarity scores are within the score range of 50 < S <= 100.

[0048] When the person captured in the target image 10 is a trainee for performances such as dance or figure skating, it can be said that the posture group GP1 is a group of trainees whose performance quality is lower than that of the trainees in the posture group GP2. Therefore, the trainers of these trainees can grasp that they should pay more attention to the persons in the posture group GP1 than to the persons in the posture group GP2 in order to give detailed feedback to the persons in the posture group GP1.

[0049] <<Example 2>> In some embodiments, based on the similarity of each other's postures, people can be divided into posture groups. This means assigning people with similar postures to the same posture group and assigning people with dissimilar postures to different posture groups.

[0050] Therefore, the classification unit 2060 can execute clustering such as k-means clustering on the keypoint group to divide the keypoint group into a plurality of clusters. Each cluster represents a group of people taking similar poses. Thus, each cluster can be treated as a pose group. Note that the keypoint group can be represented by multi-dimensional data (for example, an array of the positions of body parts), and there are various methods for executing clustering on a set of multi-dimensional data. Therefore, any of these methods can be applied to the classification unit 2060 to execute clustering on the set of keypoint groups. Note that the number of clusters (that is, the number of pose groups) may be defined in advance or may be dynamically determined as a result of clustering.

[0051] As a result of such classification, the quality of the poses of each pose group may be different. Also, assume that the person imaged in the target image 10 is a trainee, and the classification unit 2060 generates three pose groups as a result of the above clustering. In this case, these pose groups may include a first pose group with a high level of performance, a second pose group with a medium level of performance, and a third pose group with a low level of performance. In this way, the pose analysis device 2000 can make it easier for the trainer of the trainee to identify the performance level of the trainee and can make it easier for the trainer to recognize the trainee who should be paid attention to.

[0052] From another perspective, as a result of classification, the pose group may include people who make similar pose mistakes. That is, the pose group can represent a group of trainees for whom the trainer can give common advice. Therefore, the pose analysis device 2000 can improve the working efficiency of the trainer.

[0053] As described above, in some embodiments, the classification unit 2060 can classify a person into a posture group using the time series (motion) of the person's posture. In this case, the classification unit 2060 can perform clustering on a series of actions of a person. The actions of a specific person can be represented by a time series of a group of keypoints, each of which is multi-dimensional data (for example, an array of the positions of body parts). There are various methods for performing clustering on a set of time series of multiple multi-dimensional data, and one of these methods can be applied to the classification unit 2060 to perform clustering on a series of actions of a person.

[0054] <<Consideration of Posture Types>> When a plurality of different types of postures are captured in the target image 10, the classification unit 2060 may classify the person's posture based on the type of posture. For example, assume that a performance lesson is being taken in which different types of postures are taken simultaneously. This means that there are groups taking different postures. In this case, the comparison of the quality of the postures should be performed for each type of posture. Therefore, it is preferable to generate a set of posture groups for each type of posture.

[0055] In this case, the estimation unit 2040 estimates the person's posture by generating a group of keypoints of the person and determining the person type label. Then, the classification unit 2060 classifies the person into a group called a type group (that is, the type of posture) based on the person type label. The type groups are generated for each type of posture. A specific type group of postures includes persons having a type label representing the type of posture corresponding to the type group. The classification unit 2060 classifies the persons into posture groups as described above for each type group.

[0056] FIG. 5 shows the classification of persons considering the type of posture. First, the set 30 of persons captured in the target image 10 is classified into type groups 40 based on the type of posture of each person. Then, each type group 40 is classified into a plurality of posture groups 50 based on the postures of the persons within the type group.

[0057] When classifying a person into a pose group based on a reference pose (i.e., in the case of Example 1), since the ideal pose differs for each type of pose, a reference pose is prepared for each type of pose. The classification unit 2060 operates as follows for each type group. First, the classification unit 2060 calculates a similarity score of the person, which represents the similarity between the person's pose and the reference pose corresponding to the type group. Then, based on the similarity score, the classification unit 2060 assigns each person within the type group to one of the pose groups.

[0058] Suppose the target image 10 includes ten people P1 to P10, where P1 to P4 are in the pose of type T1, and P5 to P10 are in the pose of type T2. In this case, the classification unit 2060 assigns people P1 to P4 to the type group GT1 corresponding to type T1. On the other hand, the classification unit 2060 assigns people P5 to P10 to the type group GT2 corresponding to type T2.

[0059] In this example, two pose groups are prepared for each type group, such as pose groups GP1 and GP2 for type group GT1, and pose groups GP3 and GP4 for type group GT2. The classification unit 2060 assigns people P1 to P4 to pose group GP1 or GP2 based on the similarity of their poses to the reference pose RP1 representing the ideal pose of type T1. On the other hand, the classification unit 2060 assigns people P5 to P10 to pose group GP3 or GP4 based on the similarity of their poses to the reference pose RP2 representing the ideal pose of type T2.

[0060] When classifying a person into a posture group based on the similarity between postures (i.e., in the case of Example 2), the classification unit 2060 can divide each type group into a plurality of posture groups by performing clustering for each type group. Assume that the target image 10 includes the above-described persons P1 to P10. In this case, the classification unit 2060 divides the persons P1 to P4 into a plurality of posture groups by performing clustering on the type group GT1. Similarly, the classification unit 2060 divides the persons P5 to P10 into a plurality of posture groups by performing clustering on the type group GT2.

[0061] When classifying a person based on the person's motion, a type label representing the type of the person's motion is determined, and a type group including persons performing the type of motion corresponding to the type group is generated. Also, when classifying a person into a posture group based on a reference motion (i.e., in the case of Example 1), since there is an individual ideal motion for each type of motion, a reference motion is prepared for each type of motion.

[0062] <Output of group information 20: S108> The output unit 2080 outputs the group information 20 (S108). The group information 20 includes one or more pieces of information regarding one or more posture groups. In some embodiments, the output unit 2080 changes the target image 10 so that a viewer of the changed target image 10 can grasp one or more posture groups, and includes the changed target image 10 (hereinafter referred to as an "output image") in the group information 20. For example, the output image includes a common mark (such as a boundary box of the same color) above or around persons belonging to the same posture group.

[0063] The mark may be added to a single posture group or a plurality of posture groups. In the former case, the mark is used, for example, by a viewer such as a trainer to highlight a posture group of a person to be noted. For example, a posture group with the lowest performance level (e.g., the smallest similarity score) can be highlighted.

[0064] When adding marks for multiple pose groups, different types of marks can be used for each pose group. For example, the color, shape, or line of the mark is defined for each pose group.

[0065] FIG. 6 shows an example of an output image. In the example shown in FIG. 6, it is assumed that two pose groups GP1 and GP2 are generated. The output image 60 includes marks 70-1 to 70-3 indicating the persons included in the pose group GP1 and marks 80-1 to 80-3 indicating the persons included in the pose group GP2. The mark 70 is a solid-line bounding box, and the mark 80 is a dotted-line bounding box. Since the types of lines are different, the viewer of the output image 60 can easily and naturally recognize that the persons captured by the camera are divided into two groups and which person belongs to which group.

[0066] Also, the output image 60 may include information indicating one or more features of each pose group. Thereby, the viewer of the output image 60 can easily understand the features of each pose group.

[0067] For example, the feature of the pose group may include the start, end, or both of the score range of the pose group. In another example, the feature of the pose group may include the rank of the pose group. In this case, the pose groups can be ranked by the score range. Assuming that three pose groups are generated based on the similarity scores of each person. In this case, these pose groups can be ranked as high quality, medium quality, and low quality, respectively. Therefore, the output image 60 may include information indicating which mark represents which rank.

[0068] Furthermore, or alternatively, the output image 60 can include information indicating one or more features (e.g., similarity scores) of each person. If marks are not added to all pose groups, the output image 60 can show only the features of the persons to whom marks are added.

[0069] Furthermore, or alternatively, the group information 20 can include statistics regarding the pose groups. An example of statistics regarding the pose groups is the proportion of people within each pose group. There are two pose groups GP1 and GP2. GP1 is a pose group with a high quality of performance and includes 6 people. GP2 is a pose group with a low quality of performance and includes 14 people. In this case, the proportions of the pose groups GP1 and GP2 are 30% and 70% respectively. With this information, the viewer of the group information 20 can easily know to what extent the proportion of a specific pose group is, for example, to what extent the proportion of people with a low quality of performance is.

[0070] When generating pose groups by clustering (i.e., Example 2 of classification), the proportion of a specific pose group can represent the proportion of people taking similar poses to each other. This information is useful, for example, when the people captured in the target image 10 are performing a performance that requires everyone to take the same pose as each other, such as line dance or artistic swimming. In this case, the proportion of the pose group in which people take the correct pose can be used as an indicator of the quality of the overall performance. Thereby, the viewer of the group information can easily recognize the quality of the overall performance.

[0071] There are various ways to output the group information 20. In some embodiments, the group information 20 may be stored in a storage device, may be displayed on a display device, or may be transmitted to other computers such as the user's PC or smartphone of the pose analysis device 2000.

[0072] In some embodiments, the posture analysis device 2000 acquires the target image 10 that constitutes the target video and outputs the group information 20 in real time. In this case, the viewer can easily grasp the posture groups in real time. For example, assume that one or more cameras are installed in a lesson room where a plurality of trainees are receiving performance lessons, and the target image 10 that constitutes the target video is generated and transmitted to the posture analysis device 2000. Further, the posture analysis device 2000 receives the target image 10, classifies the people in the target image 10 into posture groups, and outputs a series of output videos called "output videos" to the display device in real time. In this case, the trainer of the trainee can easily grasp the posture groups in real time by looking at the output videos displayed on the display device. Thereby, the trainer can, for example, feel the performance level of each trainee. Thereby, the trainer can easily give appropriate feedback to the trainees. In particular, as described above, since the trainer can recognize the group of trainees with a lower quality of performance than other trainees, the trainer can pay attention to those trainees and give detailed feedback.

[0073] The program can be stored using various types of non-transitory computer readable media and provided to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM, CD-R, CD-R / W, semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM). Also, the program may be provided to a computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can supply the program to a computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0074] Although the present disclosure has been described with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the invention.

Description of Reference Numerals

[0075] 10 Target Image 20 Group Information 30 Set 40 Type Group 50 Pose Group 60 Output Image 70 Mark 80 Mark 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage Device 1100 Input / Output Interface 1120 Network Interface 2000 Posture Analysis Device 2020 Acquisition Unit 2040 Estimation Unit 2060 Classification Unit 2080 Output Unit

Claims

1. at least one storage element configured to store commands; and at least one processor, wherein the at least one processor executes the commands to obtain a target image in which two or more persons are imaged, estimate the posture of each person, classify the persons into two or more posture groups based on the postures of the persons, and output group information including information about at least one of the plurality of posture groups. A posture analysis device configured as such.

2. The classification of the persons includes calculating a similarity score representing the similarity between a reference posture and the posture of each of the plurality of persons, and assigning each person to the posture group corresponding to the similarity score of the person. The plurality of posture groups are assigned to different ranges of the similarity scores. The posture analysis device according to claim 1.

3. The classification of the persons includes performing clustering based on postures on the plurality of persons to classify the plurality of persons into two or more clusters, thereby obtaining the plurality of clusters as the plurality of posture groups. The posture analysis device according to claim 1.

4. The classification of the persons includes classifying the plurality of persons into two or more type groups based on the types of postures of the persons. The plurality of type groups are associated with different types of postures, and for each of the plurality of type groups, classifying the plurality of persons within the type group into the posture groups. The posture analysis device according to any one of claims 1 to 3.

5. The group information includes an output image generated by changing the target image so as to display a common mark for a plurality of persons belonging to the same posture group. The posture analysis device according to any one of claims 1 to 3.

6. obtaining a target image in which two or more persons are imaged, estimating the posture of each person, classifying the persons into two or more posture groups based on the postures of the persons, and outputting group information including information about at least one of the plurality of posture groups. A posture analysis method executed by a computer, including the above steps.

7. The classification of the persons includes Calculating, for each of the plurality of persons, a similarity score representing the similarity between the reference posture and the posture of the person; Assigning each person to the posture group corresponding to the similarity score of the person, wherein the plurality of posture groups are assigned to different ranges of the similarity score, the posture analysis method according to claim 6.

8. The classification of the person includes obtaining a plurality of posture groups as a plurality of clusters by performing posture-based clustering on the plurality of persons to classify the plurality of persons into two or more clusters, the posture analysis method according to claim 6.

9. The classification of the person includes classifying the plurality of persons into two or more type groups based on the types of postures of the persons, wherein the plurality of type groups are associated with different types of postures, for each of the plurality of type groups, classifying the plurality of persons within the type group into the posture group, the posture analysis method according to any one of claims 6 to 8.

10. The group information includes an output image generated by changing the target image to display a common mark for a plurality of persons belonging to the same posture group, the posture analysis apparatus according to any one of claims 6 to 8.

11. Obtaining a target image in which two or more persons are imaged; Estimating the posture of each person; Classifying the persons into two or more posture groups based on the postures of the persons; A non-transitory computer-readable storage medium storing a program for causing a computer to output group information including information about at least one of the plurality of posture groups.

12. The classification of the person includes Calculating, for each of the plurality of persons, a similarity score representing the similarity between the reference posture and the posture of the person; Assigning each person to the posture group corresponding to the similarity score of the person, wherein the plurality of posture groups are assigned to different ranges of the similarity score, the storage medium according to claim 11.

13. The classification of the person includes obtaining a plurality of clusters as a plurality of the posture groups by performing clustering based on postures on the plurality of persons and classifying the plurality of persons into two or more clusters. The storage medium according to claim 11.

14. The classification of the person includes classifying the plurality of persons into two or more type groups based on types of postures of the persons, and the plurality of type groups are associated with different types of postures from each other. The storage medium according to any one of claims 11 to 13, including classifying the plurality of persons within each of the plurality of type groups into the posture groups.

15. The group information includes an output image generated by changing the target image so as to display a common mark for the plurality of persons belonging to the same posture group. The storage medium according to any one of claims 11 to 13.

Images