Posture analysis device, posture analysis method and program

Abstract

The pose analysis device (2000) acquires a target image (10) and estimates the pose of each person. The target image (10) includes a plurality of persons imaged by a camera. The persons perform arbitrary actions such as playing, exercising, and musical instrument performance. The pose analysis device (2000) classifies the persons into a plurality of pose groups based on the pose of the persons, and outputs group information (20) including information regarding at least one pose group.

Description

[Technical Field]

[0001] The present invention generally relates to a posture analysis apparatus, a posture analysis method, and a non-transitory computer-readable storage medium. [Background technology]

[0002] There are technologies for analyzing images of people. Patent Document 1 discloses a system that analyzes images of students in a class to determine the current situation of the class, such as their concentration level. The situation of the class is determined by comparing characteristics (e.g., posture) obtained from sample images of the class situation stored in advance with the characteristics of the students in the class captured in the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent Application Publication No. 2020 / 0126444 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 does not disclose a technology for handling an image in which multiple people are captured. The purpose of the present disclosure is to provide a novel technology for analyzing the postures of people using an image in which multiple people are captured. [Means for solving the problem]

[0005] The posture analysis device provided by the present disclosure includes at least one memory element configured to store instructions and at least one processor. The at least one processor is configured to execute the instructions to acquire a target image in which two or more people are captured, estimate a pose of each of the people, classify the people into two or more pose groups based on the poses of the people, and output group information including information about at least one of the pose groups.

[0006] The present disclosure further provides a posture analysis method implemented by a computer. The posture analysis method includes acquiring a target image in which two or more people are captured, estimating the posture of each of the people, classifying the people into two or more posture groups based on the posture of the people, and outputting group information including information about at least one of the posture groups.

[0007] The present disclosure further provides a non-transitory computer-readable medium for storing a program. The program causes a computer to acquire a target image in which two or more people are captured, estimate the posture of each of the people, classify the people into two or more posture groups based on the posture of the people, and output group information including information about at least one of the posture groups. [Effects of the Invention]

[0008] According to the present invention, a novel technique for analyzing the posture of a person using an image in which a plurality of people are captured is provided. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an outline of a posture analysis device. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of the posture analysis device. [Figure 3] FIG. 3 is a block diagram illustrating an example of a hardware configuration of the posture analysis device. [Figure 4] FIG. 4 is a flowchart showing an example of the flow of processing executed by the posture analysis device. [Figure 5] FIG. 5 is a diagram showing classification of people taking into consideration types of postures. [Figure 6] FIG. 6 is a diagram showing an example of an output image. DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiments according to the present disclosure will be described below with reference to the drawings. The same elements are assigned the same reference numerals throughout the drawings, and redundant descriptions will be omitted as necessary. Furthermore, unless otherwise specified, predetermined information (e.g., predetermined values ​​or predetermined threshold values) is pre-stored in a storage device accessible by a computer that uses the information.

[0011] <Summary> Fig. 1 is a diagram showing an overview of a posture analysis device 2000 according to an embodiment. Note that the overview shown in Fig. 1 shows an example of the operation of the posture analysis device 2000 to make it easier to understand the posture analysis device 2000, and is not intended to limit or narrow the scope of the operation of the posture analysis device 2000.

[0012] The posture analysis device 2000 is configured to classify people captured in a target image 10 into groups according to the postures of the people, and output information on the classification results (group information 20). The target image 10 is image data, such as an RGB image or a grayscale image, that visually includes multiple people.

[0013] The person captured in target image 10 is doing anything. For example, the person is performing a sport such as figure skating or dancing. In another example, the person is exercising, such as yoga. In another example, the person is playing a musical instrument, such as guitar or piano. In another example, the person is attending a class at school. In another example, the person is assembling parts in a factory or patrolling a building.

[0014] To classify people captured in the target image 10, the posture analysis device 2000 may operate as follows. The posture analysis device 2000 acquires the target image 10 and estimates the posture of each person captured in the target image 10. Next, the posture analysis device 2000 classifies people into groups called "pose groups" based on the estimated postures of the people. The posture analysis device 2000 then outputs group information 20 containing information about one or more posture groups.

[0015] The posture analysis device 2000 may handle multiple target images 10 containing different people in parallel. In this case, multiple cameras may be installed to capture different regions (for example, different areas of a lesson room where people are taking performance lessons), and each camera may be configured to generate a target image 10. The posture analysis device 2000 may analyze each of these target images 10 to detect people, and classify the detected people into posture groups based on their postures.

[0016] For the sake of simplicity, unless otherwise specified, posture analysis device 2000 handles a single target image 10. A posture analysis device 2000 that handles multiple target images 10 can operate in the same manner as a posture analysis device 2000 that handles a single target image 10, unless otherwise specified.

[0017] <Examples of effects> According to the posture analysis device 2000 of this embodiment, the postures of people captured in the target image 10 are estimated, and the people are classified into posture groups based on the postures. This provides a new technique for analyzing the postures of people using an image in which multiple people are captured.

[0018] Furthermore, the posture analysis device 2000 outputs group information 20 indicating information about at least one posture group. The information about posture groups is effective and useful in various ways. In short, a viewer of the group information 20 can distinguish people who belong to that posture group from other people, thereby finding groups of people who share certain characteristics with each other.

[0019] For example, as will be described in detail later, the posture analysis device 2000 may classify people based on the quality of their posture (e.g., similarity to an ideal posture) and output group information 20 indicating the posture group with the lowest posture quality. This group information 20 allows a viewer of the group information 20 to recognize people whose performance quality is lower than other people. Suppose that the viewer of the group information 20 is a performance trainer, and the people captured in the target image 10 are people receiving training. In this case, the posture group with the lowest performance quality can be treated by the trainer as a group of people to whom the trainer should pay attention and provide detailed feedback.

[0020] Other usefulness and effectiveness of the group information 20 will be described later.

[0021] The posture analysis device 2000 will be described in more detail below.

[0022] <Example of functional configuration> 2 is a block diagram showing an example of the functional configuration of a posture analysis device 2000 according to this embodiment. The posture analysis device 2000 includes an acquisition unit 2020, an estimation unit 2040, a classification unit 2060, and an output unit 2080. The acquisition unit 2020 acquires a target image 10. The estimation unit 2040 estimates the posture of each person captured in the target image 10. The classification unit 2060 classifies people into posture groups based on the estimated postures of the people. The output unit 2080 outputs group information 20.

[0023] <Example of hardware configuration> The posture analysis device 2000 may be realized by one or more computers. Each of the one or more computers may be a dedicated computer manufactured for realizing the posture analysis device 2000, or may be a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0024] The posture analysis device 2000 may be realized by installing an application in one or more computers. The application is realized by a program that causes one or more computers to function as the posture analysis device 2000. In other words, the program is an implementation of the functional components of the posture analysis device 2000 illustrated in FIG. 2.

[0025] 3 is a block diagram showing an example of the hardware configuration of a computer 1000 that realizes the posture analysis device 2000. In FIG. 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output (I / O) interface 1100, and a network interface 1120.

[0026] The bus 1020 is a data transmission path through which the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 transmit and receive data to and from each other. The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA). The memory 1060 is a main memory element such as a random access memory (RAM) or a read-only memory (ROM). The storage device 1080 is an auxiliary memory element such as a hard disk, a solid state drive (SSD), or a memory card. The input / output interface 1100 is an interface between the computer 1000 and peripheral devices (such as a keyboard, a mouse, or a display device). The network interface 1120 is an interface between the computer 1000 and a network. The network may be a local area network (LAN) or a wide area network (WAN).

[0027] The hardware configuration of the computer 1000 is not limited to that shown in Fig. 3. For example, as described above, the posture analysis device 2000 may be realized by a plurality of computers. In this case, the computers may be connected to each other via a network.

[0028] <Processing flow> 4 is a flowchart showing an example of the flow of processing executed by the posture analysis device 2000 of the embodiment. The acquisition unit 2020 acquires the target image 10 (S102). The estimation unit 2040 estimates the posture of the person captured in the target image 10 (S104). The classification unit 2060 classifies the person into posture groups based on the posture (S106). The output unit 2080 outputs the group information 20 (S108).

[0029] <Acquisition of target image 10: S102> The acquisition unit 2020 acquires a target image 10 (S102). As described above, the target image 10 includes one or more people. The people captured in the target image 10 are doing anything. For example, the people perform a performance such as figure skating or dancing. In another example, the people exercise such as yoga. In another example, the people play a musical instrument such as guitar or piano. In another example, the people attend class at school. In another example, the people perform a task such as assembling parts in a factory or patrolling a building.

[0030] In some embodiments, the target image 10 is one of a plurality of images (video frames) in a time sequence that constitutes video data called a "target video." In this case, the acquisition unit 2020 may acquire one or more video frames that constitute the target video and use the acquired video frames as the target image 10. Note that it is not necessary to use all video frames of the target video as the target image 10. For example, the acquisition unit 2020 may acquire every predetermined number of video frames (e.g., every 10 frames) from the target video as the target image 10.

[0031] Alternatively, the acquisition unit 2020 may divide the target video into multiple segments and acquire one or more video frames from each segment as the target image 10. The target video may be divided into segments based on the length of time. Specifically, the target video may be divided into segments of a predetermined length of time. In another example, the acquisition unit 2020 recognizes multiple scenes captured in the target video and divides the target video into segments representing the recognized scenes. Assume that the target video includes a figure skating performance. In this case, the target video may include scenes such as jumps, spins, and steps, so the acquisition unit 2020 divides the target video into segments corresponding to jumps, spins, and steps. Note that there are various techniques for recognizing scenes from video data, and any of these techniques can be applied to the acquisition unit 2020 to recognize scenes from the target video.

[0032] There are various methods for acquiring the target image 10. In some embodiments, the target image 10 is pre-stored in a storage device in a state that allows the posture analysis device 2000 to acquire it. In this case, the acquisition unit 2020 may acquire the target image 10 by accessing the storage device. In other embodiments, the target image 10 may be transmitted from another computer, such as a camera that generates the target image 10. In this case, the acquisition unit 2020 may receive and acquire the target image 10.

[0033] When the acquisition unit 2020 acquires the target video, the target video may be acquired in the same manner as the target image 10. In another example, the acquisition unit 2020 may acquire a target video generated in real time. Specifically, a video camera that generates the target video may repeatedly capture images of the surrounding scenery to generate video frames of the target video, and output the generated video frames to the posture analysis device 2000. In this case, the acquisition unit 2020 receives video frames sequentially transmitted from the video camera, and defines the time series of the received video frames as the target video.

[0034] <Posture estimation: S104> The estimation unit 2040 estimates the posture of each person captured in the target image 10 (S104). There are various methods for posture estimation, and the estimation unit 2040 can apply any of these methods. For example, the estimation unit 2040 detects the positions of characteristic parts of the human body (neck, eyes, shoulders, etc.) as key points from the target image 10. Then, the estimation unit 2040 divides the key points into groups (called key point groups) that include key points that belong to the same person, and estimates the posture of each person based on the key point group corresponding to that person.

[0035] A person's posture can be classified into one of predetermined posture types (such as a jump, spin, or step in figure skating). In this case, a specific person's posture is represented by a pair of a keypoint group of the person and a type label indicating the type of the person's posture. To recognize the type of the person's posture, the estimator 2040 may include a classification model configured to obtain a set of keypoints (i.e., keypoint groups) of the person and output a type label indicating the type of the person's posture. The classification model may be implemented by a machine learning-based model such as a neural network.

[0036] As will be described later, the classification unit 2060 may classify people into pose groups using a time series of the person's poses, rather than a single pose of the person. In this case, the estimation unit 2040 obtains a time series of poses for each person by estimating the person's pose from each target image 10 using the time series of the target images 10. Note that the time series of poses can also be called "movements." Therefore, when classifying people using the time series of the person's poses, it can be said that the posture analysis device 2000 classifies people based on the person's movements.

[0037] <Person Classification Based on Posture: S106> The classification unit 2060 classifies people into posture groups based on the postures of the people (S106). An example of classifying people based on their postures will be described below.

[0038] <<Example 1>> In some implementations, a person can be classified based on the similarity of their pose to a predefined reference pose, which can be defined by a set of keypoints that represent an ideal pose, where the more similar the person's pose is to the reference pose, the higher the quality of the pose.

[0039] To represent how similar a person's pose is to the reference pose, the classifier 2060 may calculate a similarity score for each person. The similarity score for a particular person may be a value that represents the similarity between the person's pose and the reference pose.

[0040] There are various ways to quantify the similarity between two poses, and one of these methods can be applied to the classifier 2060 to calculate a similarity score. In short, the similarity between a person's pose and a reference pose can be expressed by the similarity between the spatial arrangement of keypoints in the person's keypoint group and the spatial arrangement of keypoints in the reference pose.

[0041] In some embodiments, the classifier 2060 includes a machine learning-based feature extractor, such as a neural network, configured to receive a keypoint group as input and output features of the pose represented by the keypoint group (e.g., features of the spatial arrangement of keypoints in the keypoint group). In this case, the classifier 2060 inputs the keypoint group of the person to the feature extractor to obtain features of the pose of the person. The classifier 2060 also inputs the keypoint group of the reference pose to the feature extractor to obtain features of the reference pose. The classifier 2060 then calculates a value representing the similarity between the features of the pose of the person and the features of the reference pose as a similarity score.

[0042] As described above, in some embodiments, the classifier 2060 may classify people into pose groups using a time series of the person's poses (movements). In this case, a time series of reference poses called "reference movements" is prepared in advance. The reference movements can be represented by a time series of keypoint groups, each of which represents a reference pose at a certain point in time. For each person, the classifier 2060 calculates a similarity score that represents the degree of similarity between the person's movement and the reference movements.

[0043] There are various ways to quantify the similarity between two actions, and one of these methods can be applied to the classifier 2060. In short, the similarity between a person's action and a reference action can be expressed by the similarity between the time series of the spatial arrangement of keypoints of the person and the time series of the spatial arrangement of keypoints of the reference action.

[0044] In some embodiments, the classifier 2060 includes a machine learning-based feature extractor, such as a neural network, configured to receive as input a time series of keypoint groups and output features of actions represented by the keypoint groups. In this case, the classifier 2060 inputs multiple keypoint groups of a person into the feature extractor to obtain features of the person's actions. The classifier 2060 also inputs multiple keypoint groups of a reference action into the feature extractor to obtain features of the reference action. The classifier 2060 then calculates a similarity score, which is a value representing the similarity between the features of the person's actions and the features of the reference action.

[0045] Based on the person's similarity score, the person is classified into a pose group. For example, the classifier 2060 may generate a predetermined number of pose groups that are initialized to empty. Each pose group is associated with a range of similarity scores called a "score range." The score ranges are defined so that they do not overlap with each other.

[0046] Assume that the total range of the similarity score S is 0<=S=<100, and two posture groups GP1 and GP2 are defined. In this case, posture groups GP1 and GP2 can be defined as follows: posture group GP1 has a score range of 0<=S<50, and posture group GP2 has a score range of 50<=S<=100.

[0047] The classification unit 2060 determines, for each person, a score range including the similarity score of that person, and assigns that person to a posture group corresponding to the determined score range. Suppose there are the above two posture groups GP1 and GP2. Further, suppose there are person P1 with a similarity score of 20, person P2 with a similarity score of 70, person P3 with a similarity score of 60, person P4 with a similarity score of 45, and person P5 with a similarity score of 10. In this case, persons P1, P4, and P5 are assigned to the posture group GP1 because their similarity scores are within the score range of 0 <= S < 50, while persons P2 and P3 are assigned to the posture group GP2 because their similarity scores are within the score range of 50 < S <= 100.

[0048] When the person captured in the target image 10 is a trainee for performances such as dance or figure skating, it can be said that the posture group GP1 is a group of trainees whose performance quality is lower than that of the trainees in the posture group GP2. Therefore, the trainers of these trainees can grasp that they should pay more attention to the persons in the posture group GP1 than to the persons in the posture group GP2 in order to give detailed feedback to the persons in the posture group GP1.

[0049] <<Example 2>> In some embodiments, based on the similarity of each other's postures, persons can be divided into posture groups. This means assigning persons with similar postures to the same posture group and assigning persons with dissimilar postures to different posture groups.

[0050] To this end, the classifier 2060 can perform clustering, such as k-means clustering, on the keypoint groups to divide the keypoint groups into multiple clusters. Each cluster represents a group of people who have similar poses. This allows each cluster to be treated as a pose group. Note that the keypoint groups can be represented by multidimensional data (e.g., an array of body part positions), and there are various methods for performing clustering on a set of multidimensional data. Therefore, any of these methods can be applied to the classifier 2060 to perform clustering on the set of keypoint groups. Note that the number of clusters (i.e., the number of pose groups) may be predefined or may be dynamically determined as a result of clustering.

[0051] As a result of such classification, the quality of posture in each posture group may differ. Furthermore, suppose that the person captured in the target image 10 is a trainee, and the classification unit 2060 generates three posture groups as a result of the above-described clustering. In this case, these posture groups may include a first posture group representing a high level of performance, a second posture group representing a medium level of performance, and a third posture group representing a low level of performance. In this way, the posture analysis device 2000 can make it easier for trainers to identify the performance levels of trainees and to recognize trainees who require attention.

[0052] From another perspective, as a result of classification, a posture group may include people who make similar posture errors. In other words, a posture group may represent a group of trainees to whom a trainer can provide common advice. Therefore, the posture analysis device 2000 can improve the work efficiency of the trainer.

[0053] As described above, in some embodiments, the classifier 2060 can classify a person into pose groups using a time series of the person's poses (movements). In this case, the classifier 2060 can perform clustering on a series of movements of the person. A specific person's movement can be represented by a time series of keypoint groups, each of which is multidimensional data (e.g., an array of positions of body parts). There are various methods for clustering a set of multiple time series of multidimensional data, and one of these methods can be applied to the classifier 2060 to perform clustering on a series of movements of the person.

[0054] <<Consideration of posture types>> When a plurality of different types of poses are captured in the target image 10, the classification unit 2060 may classify the person's pose based on the type of pose. For example, suppose a person is taking a performance lesson in which different types of poses are simultaneously taken. This means that there are groups that take different poses. In this case, the quality of the poses should be compared for each type of pose. Therefore, it is preferable to generate a set of pose groups for each type of pose.

[0055] In this case, the estimation unit 2040 generates keypoint groups for the person and determines a type label for the person, thereby estimating the person's posture. The classification unit 2060 then classifies the person into groups called type groups (i.e., posture types) based on the person's type label. A type group is generated for each posture type. A specific posture type group includes people with type labels that indicate the posture types corresponding to the type group. For each type group, the classification unit 2060 classifies the people into posture groups as described above.

[0056] 5 shows how people are classified based on their pose type. First, a group 30 of people captured in a target image 10 is classified into type groups 40 based on the type of pose of each person. Each type group 40 is then classified into multiple pose groups 50 based on the poses of the people in that type group.

[0057] When classifying people into posture groups based on the reference posture (i.e., in Example 1), the ideal posture differs for each posture type, so a reference posture is prepared for each posture type. The classification unit 2060 operates as follows for each type group. First, the classification unit 2060 calculates a similarity score for the person, which represents the similarity between the posture of the person and the reference posture corresponding to the type group. Then, the classification unit 2060 assigns each person in the type group to one of the posture groups based on the similarity score.

[0058] Assume that the target image 10 includes ten people P1 to P10, with people P1 to P4 assuming a pose of type T1 and people P5 to P10 assuming a pose of type T2. In this case, the classification unit 2060 assigns people P1 to P4 to type group GT1 corresponding to type T1. On the other hand, the classification unit 2060 assigns people P5 to P10 to type group GT2 corresponding to type T2.

[0059] In this example, two posture groups are prepared for each type group: posture groups GP1 and GP2 for type group GT1, and posture groups GP3 and GP4 for type group GT2. The classification unit 2060 assigns persons P1 to P4 to posture group GP1 or GP2 based on the similarity of the postures to reference posture RP1, which represents the ideal posture for type T1. On the other hand, the classification unit 2060 assigns persons P5 to P10 to posture group GP3 or GP4 based on the similarity of the postures to reference posture RP2, which represents the ideal posture for type T2.

[0060] When classifying people into posture groups based on the similarity between postures (i.e., in the case of Example 2), the classification unit 2060 may divide each type group into multiple posture groups by clustering for each type group. Assume that the target image 10 includes the above-mentioned people P1 to P10. In this case, the classification unit 2060 performs clustering on the type group GT1 to divide the people P1 to P4 into multiple posture groups. Similarly, the classification unit 2060 performs clustering on the type group GT2 to divide the people P5 to P10 into multiple posture groups.

[0061] When classifying people based on their movements, type labels that represent the types of people's movements are determined, and type groups that include people performing the types of movements corresponding to the type groups are generated. When classifying people into posture groups based on reference movements (i.e., in the case of Example 1), each type of movement has its own individual ideal movement, so a reference movement is prepared for each type of movement.

[0062] <Output of group information 20: S108> The output unit 2080 outputs the group information 20 (S108). The group information 20 includes one or more pieces of information related to one or more pose groups. In some embodiments, the output unit 2080 modifies the target image 10 so that a viewer of the modified target image 10 can grasp one or more pose groups, and includes the modified target image 10 (hereinafter referred to as the "output image") in the group information 20. For example, the output image includes a common mark (such as a bounding box of the same color) on or around people who belong to the same pose group.

[0063] A mark may be added to a single posture group or multiple posture groups. In the former case, the mark is used to highlight posture groups of a person who should be noted by a viewer, such as a trainer. For example, a posture group with the lowest performance level (e.g., the smallest similarity score) may be highlighted.

[0064] When adding marks to multiple posture groups, different types of marks may be used for each posture group, for example, the color, shape, or line of the mark may be defined for each posture group.

[0065] FIG. 6 shows an example of an output image. In the example shown in FIG. 6, two pose groups GP1 and GP2 are generated. The output image 60 includes marks 70-1 to 70-3 that represent people included in pose group GP1 and marks 80-1 to 80-3 that represent people included in pose group GP2. Mark 70 is a bounding box with solid lines, and mark 80 is a bounding box with dotted lines. Because the types of lines are different, a viewer of the output image 60 can easily and naturally recognize that the people captured by the camera are divided into two groups and which people belong to which group.

[0066] The output image 60 may also include information indicating one or more characteristics of each posture group, which allows a viewer of the output image 60 to easily understand the characteristics of each posture group.

[0067] For example, the pose group feature may include the start, end, or both of the score range for the pose group. In another example, the pose group feature may include the rank of the pose group. In this case, the pose groups may be ranked by score range. Suppose three pose groups are generated based on the similarity scores of each person. In this case, these pose groups may be ranked as high quality, medium quality, and low quality, respectively. Therefore, the output image 60 may include information indicating which marks represent which ranks.

[0068] Additionally or alternatively, the output image 60 may include information indicative of one or more characteristics (e.g., a similarity score) of each person. If not all pose groups have marks, the output image 60 may only show the characteristics of the people that have marks.

[0069] Additionally or alternatively, the group information 20 may include statistics regarding the attitude groups. One example of statistics regarding the attitude groups is the percentage of people in each attitude group. There are two attitude groups, GP1 and GP2. GP1 is an attitude group with high performance quality and includes 6 people. GP2 is an attitude group with low performance quality and includes 14 people. In this case, the percentages of the attitude groups GP1 and GP2 are 30% and 70%, respectively. This information allows a viewer of the group information 20 to easily know what the percentage of a particular attitude group is, for example, what the percentage of people with low performance quality is.

[0070] When generating posture groups by clustering (i.e., Classification Example 2), the percentage of a particular posture group can represent the percentage of people who assume similar postures. This information is useful, for example, when the people captured in the target image 10 are performing a performance that requires everyone to assume the same posture, such as line dancing or artistic swimming. In this case, the percentage of posture groups in which people assume the correct posture can be used as an indicator of the quality of the overall performance. This allows viewers of the group information to easily recognize the quality of the overall performance.

[0071] There are various ways to output the group information 20. In some embodiments, the group information 20 may be stored in a storage device, displayed on a display device, or transmitted to another computer, such as a PC or smartphone, of the user of the posture analysis device 2000.

[0072] In some embodiments, the posture analysis device 2000 acquires target images 10 constituting a target video and outputs group information 20 in real time. In this case, viewers can easily understand posture groups in real time. For example, assume that one or more cameras are installed in a lesson room where multiple trainees receive performance lessons, generate target images 10 constituting a target video, and transmit them to the posture analysis device 2000. The posture analysis device 2000 also receives the target images 10, classifies people in the target images 10 into posture groups, and outputs a series of output images called "output images" to a display device in real time. In this case, the trainers' trainers can easily understand posture groups in real time by viewing the output images displayed on the display device. This allows the trainer to, for example, realize the performance level of each trainee. This allows the trainer to easily provide appropriate feedback to the trainees. In particular, as described above, the trainer can recognize groups of trainees whose performance quality is lower than that of other trainees, and can therefore pay attention to those trainees and provide detailed feedback.

[0073] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and RAMs). The program may also be provided to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can provide the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0074] Although the present disclosure has been described with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that are understandable to those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the invention. [Explanation of symbols]

[0075] 10 Target image 20 Group Information 30 gathering 40 types group 50 Posture Group 60 output images 70 marks 80 marks 1000 computers 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Attitude analysis device 2020 Acquisition Department 2040 Estimation Department 2060 Classification Department 2080 output unit

Claims

1. an acquisition means for acquiring a target image in which two or more people are captured; estimation means for estimating the posture of each person; a classification means for classifying the person into two or more pose groups based on the pose of the person; and an output means for outputting group information including information relating to at least one of the plurality of posture groups, A posture analysis device, wherein the group information includes an output image generated by modifying the target image so as to display a common mark for multiple people who belong to the same posture group.

2. The classification of the person is: calculating a similarity score representing a similarity between a reference posture and a posture of each of the plurality of people; assigning each person to the pose group corresponding to the person's similarity score; The posture analysis device according to claim 1 , wherein the plurality of posture groups are assigned to different ranges of the similarity score.

3. The pose analysis device of claim 1 , wherein the classification of the people includes performing pose-based clustering on the people to classify the people into two or more clusters, thereby obtaining a plurality of clusters as the pose groups.

4. The classification of the person is: classifying the plurality of people into two or more type groups based on the types of poses of the people; the plurality of type groups are associated with different types of postures, The posture analysis device according to claim 1 , further comprising: for each of a plurality of type groups, classifying a plurality of people in the type group into the posture groups.

5. an acquisition step of acquiring a target image in which two or more people are captured; an estimation step of estimating a pose of each person; a classification step of classifying the person into two or more pose groups based on the pose of the person; and an output step of outputting group information including information about at least one of the plurality of posture groups, A posture analysis method executed by a computer, wherein the group information includes an output image generated by modifying the target image to display a common mark for multiple people who belong to the same posture group.

6. The classification of the person is: calculating a similarity score representing a similarity between a reference posture and a posture of each of the plurality of people; assigning each person to the pose group corresponding to the person's similarity score; The posture analysis method according to claim 5 , wherein the plurality of posture groups are assigned to different ranges of the similarity score.

7. 6. The pose analysis method of claim 5, wherein the classifying of the people includes performing pose-based clustering on the people to classify the people into two or more clusters, thereby obtaining a plurality of clusters as the pose groups.

8. an acquisition step of acquiring a target image in which two or more people are captured; an estimation step of estimating a pose of each person; a classification step of classifying the person into two or more pose groups based on the pose of the person; an output step of outputting group information including information about at least one of the plurality of posture groups; The group information includes an output image generated by modifying the target image so as to display a common mark for multiple people who belong to the same posture group.

9. The classification of the person is: calculating a similarity score representing a similarity between a reference posture and a posture of each of the plurality of people; assigning each person to the pose group corresponding to the person's similarity score; The computer-readable medium according to claim 8 , wherein the plurality of pose groups are assigned to different ranges of the similarity score.

Images