System, method executed by the system, program

Abstract

The analysis of the subject's movements is carried out by utilizing a time series of indices that indicate the degree of mutual influence between the movements of one feature point (part) and another feature point (part). SOLUTION: A synchronization network calculation unit 800 calculates, based on video data 141 consisting of a plurality of frame images, a time series of inter-feature point (body part) synchronization network data 1005 indicating the interlocking between the feature points 123 for a plurality of feature points 123 possessed by a subject 102 shown in the video data 141. A leading role calculation unit 900 may calculate a time series of leading role data 143 for each feature point 123 based on the time series of inter-feature point (body part) synchronization network data 1005. A collective consciousness index calculation unit 1800 may calculate (a time series of) collective consciousness index data 1905 for a plurality of feature points 123 included in the subject 102 based on the time series of inter-feature point (body part) synchronization network data 1005.

Description

[Technical Field]

[0001] The present disclosure relates to a technique for analyzing the behavior (for example, movement) of a subject (for example, a person) based on video data (or a time series of image data). [Background technology]

[0002] In some fields (e.g., sports and healthcare), the use of sensor data or video data to analyze the behavior (e.g., movements) of individuals (people, animals, and other things) and the behavior of groups of individuals (e.g., relationships between individuals) is being considered.

[0003] Examples of prior art documents that perform analysis based on information obtained from sensor data or video data include Patent Documents 1, 2, 3, and 4 shown below. Patent Document 1 (JP 2018-81406 A) discloses a technology for analyzing a population network by calculating transfer entropy, which indicates the degree of influence that one individual has on another, based on time-series information about the individuals included in the population. Patent Document 2 (JP Patent Publication No. 2023-64238) discloses a technology for analyzing the synchronicity between element points (feature points) owned by a person by generating time series data of the amount of movement for each element point (feature point) owned by the person based on video footage of the person performing exercise, and then calculating the movement entropy, which indicates the degree of influence that one element point (feature point) has on another element point (feature point), based on the time series data of the amount of movement for each element point (feature point). Patent Document 3 (JP 2023-108290 A) discloses a technology that calculates transfer entropy, which indicates the degree of influence an individual has on another individual, based on time-series information about the individuals included in a population, and then calculates the probability of occurrence of a clique, which indicates the activation of one or more individuals in a network between individuals, and calculates a collective consciousness index consisting of a pair of an index related to the activity level of the entire clique and an index related to the diversity between cliques based on the occurrence probability. Patent Document 4 (JP Patent Publication No. 2024-54747) discloses a technique for analyzing a population network by calculating transfer entropy, which indicates the degree of influence that a certain part of one individual has on another part of another individual, based on time-series information regarding each part of the individuals in the population. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-81406 [Patent Document 2] Japanese Patent Application Publication No. 2023-64238 [Patent Document 3] Japanese Patent Application Laid-Open No. 2023-108290 [Patent Document 4] Japanese Patent Application Laid-Open No. 2024-54747 Summary of the Invention [Problem to be solved by the invention]

[0005] When video data is obtained by capturing the quick movements of a subject (e.g., a person) using a video camera with a standard shooting speed, and the video data is used to analyze the movements, the number of frame images contained in the video data obtained for a series of movements of the subject is small. For example, the movement of a baseball player swinging a bat takes about one second, but if the movement is video-captured at the standard shooting speed of 30 frames per second (30 fps), the video data obtained will only contain about 30 frames. In order to calculate a reliable index (e.g., transfer entropy) indicating the degree of mutual influence between two pieces of time-series information related to two feature points (body parts) of a subject, it is desirable for one piece of time-series information to contain at least about 30 pieces of time-series information. Therefore, as described above, from video data obtained by filming a baseball player's bat swing at a standard filming speed (30 fps), it is possible to obtain at most one index (e.g., transfer entropy) indicating the degree of mutual influence between the movement of one feature point (body part) and the movement of another feature point (body part) of the baseball player. Thus, even if video data is obtained by filming the quick movements of a subject (e.g., a person) at a standard filming speed (30 fps), it can be difficult to perform detailed analysis of the movements using the video data.

[0006] In recent years, so-called high-speed cameras, cameras for shooting video at speeds higher than the standard (30 fps), have become readily available. For example, cameras for shooting video at 120 fps, which is four times the standard speed, or 240 fps, which is eight times the standard speed, are relatively easy to obtain. If a video camera with such a high shooting speed is used to capture the rapid movements of a subject (e.g., a person) and obtain video data, it is expected that a time series of an index (e.g., transfer entropy) indicating the degree of mutual influence between the movements of one feature point (part) and another feature point (part) of the subject can be obtained. In other words, it is expected that detailed analysis of movements will be possible based on image data obtained by capturing the rapid movements of a subject.

[0007] From the above, one of the objectives of the present disclosure may be to obtain a time series of indices that indicate the degree of mutual influence between the movement of a certain feature point (part) and the movement of another feature point (part) with respect to feature points (parts) possessed by a subject shown in video data, and to analyze the movement of the subject by utilizing the time series of indices. [Means for solving the problem]

[0008] In order to achieve at least one of the above objects, the present disclosure may have the following features, for example. One aspect of the present disclosure is a system that includes a synchronization network calculation unit that calculates, based on video data consisting of a plurality of frame images, a time series of synchronization network data that indicates interlocking between feature points of a subject shown in the video data, and a leading role degree calculation unit that calculates, based on the time series of the synchronization network data, a time series of leading role degree data for each feature point. [Effects of the Invention]

[0009] As described above, the present disclosure calculates a time series of synchronization network data, each of which indicates connectivity between feature points in a subject, the connectivity reflecting the degree of mutual influence between the feature points. The present disclosure then utilizes the time series of synchronized network data to calculate a time series of dominance data for each feature point. The dominance of a feature point indicates a certain importance of the movement of that feature point in the movement of the subject. Obtaining such a time series of dominance data allows for more detailed analysis of the subject's movement.

[0010] As described above, the present disclosure makes it possible to obtain a time series of indices that indicate the degree of mutual influence between the movement of one feature point (part) and the movement of another feature point (part) with respect to feature points (parts) possessed by a subject shown in video data, and to analyze the movement of the subject by utilizing the time series of indices.

[0011] Methods and programs that achieve the same processing as the above system can also achieve the same effects as the above system. In the form of a program, costs can often be reduced. Programs also make it easier to make design changes to the processing. Other features that the present disclosure may have and the effects corresponding to those features will be disclosed in this specification, claims, or drawings. [Brief explanation of the drawings]

[0012] [Figure 1] 1 illustrates a basic functional configuration of an embodiment of the present disclosure. [Figure 2] 1 shows an overall configuration including a system 101 according to an embodiment of the present disclosure. [Figure 3] 1 shows a computer architecture for implementing the system 101. [Figure 4] 1 shows a functional configuration of a system 101 according to an embodiment of the present disclosure. [Figure 5] The parameters applied to the system 101 are shown. [Figure 6] 10 shows the processing of the feature point coordinate recognition unit and the feature point movement amount calculation unit. [Figure 7] 10 shows the processing of the synchronization determination calculation unit. [Figure 8] 10 shows the processing of the synchronization network calculation unit. [Figure 9] 10 shows the processing of the leading role degree calculation unit. [Figure 10] 10 shows a data set handled by the synchronization determination calculation unit and the synchronization network calculation unit. [Figure 11] The data group handled by the main calculation part, etc. is shown. [Figure 12] The data group handled by the main calculation unit is shown. [Figure 13] The relationship between the transition matrix and the degree of prominence (PageRank) is shown. [Figure 14] 10 shows an example of a screen related to displaying the degree of leading role, etc. [Figure 15] 10 shows the processing of the leading role degree evaluation unit. [Figure 16] 10 shows a group of data handled by the leading role evaluation unit. [Figure 17] 10 shows an example of a screen related to the individuality of the subject's movements. [Figure 18] 10 shows the processing of the collective consciousness index calculation unit. [Figure 19] 1 shows a group of data handled by the collective consciousness index calculation unit, etc. [Figure 20] The significance of the i-clique collective consciousness index is presented. [Figure 21] An example of a screen displaying the collective consciousness index is shown. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiments of the present disclosure will be described in detail below with reference to the drawings. Note that the embodiments described below do not limit the disclosure according to the claims, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solutions of the present disclosure. The following description and drawings are examples for explaining the present disclosure, and appropriate omissions and simplifications have been made for clarity of explanation. The present disclosure can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present disclosure is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. Each of the systems, devices, or functional units disclosed herein may be integrated into a single piece of hardware, or may be divided into multiple parts that work together to perform their functions. Several systems, devices, or functional units may be integrated into one hardware configuration. Each of the systems, devices, or functional units may be realized by causing a computer to execute software (programs) (as in FIG. 3). Some of the functions of the system, device, or functional unit may be realized by hardware (e.g., hardwired logic or a field programmable gate array (FPGA)), and the remaining functions may be realized by executing software (programs). All of the functions of each of the systems, devices, or functional units may be realized by hardware. Some or all of the steps shown in the flowcharts, etc. described in this disclosure may be realized by hardware. One or more systems, devices, or functional units of the present disclosure may be realized using one or more hardware resources. For this purpose, each of the systems, devices, or functional units of the present disclosure may be virtually realized. For example, a virtual computer or virtual container technique may be used. The term "program" may be included in the general concept of software, in which software and hardware resources cooperate to construct a specific system or method of operation according to the intended purpose. In other words, the term "program" is not limited to a specific type or form of program. The program may also be initially recorded in a compressed format. The same reference numbers are used in multiple drawings. In the drawings showing flowcharts, rectangular boxes indicate processing steps, and hexagonal boxes indicate conditional branching steps. In the drawings showing flowcharts, "step" is abbreviated as "S." The displays or outputs shown in the drawings are merely examples. The display or output may be in any form as long as the objectives of the present disclosure can be achieved.

[0014] 1. Basic functional configuration (Figure 1) FIG. 1 shows a basic functional configuration 100 (and the information handled) of a system 101 according to an embodiment of the present disclosure. Note that not all functional configurations shown in FIG. 1 are required. Furthermore, the existence of functional configurations other than those shown in FIG. 1 is not precluded. In FIG. 1 (and FIG. 4), solid rectangles with the word "unit" attached to their names indicate functional units. Furthermore, in FIG. 1 (and FIG. 4), items shown in dotted line frames indicate data groups handled by the functional units.

[0015] 1 of 1. Video Recording As shown in FIG. 1, a system 101 according to an embodiment of the present disclosure acquires video data 141 captured by a video camera 103 (denoted as a "video camera" in FIG. 1 (and FIG. 2)). The video data 141 may be a time series of frame images (a time series of image data). Note that in FIG. 4 (to be described later), form video data 401 is used as an example of the video data 141.

[0016] The video camera 103 captures the subject 102 at a predetermined capture speed (for example, 120 fps, which is four times the standard speed, or 240 fps, which is eight times the standard speed) to create video data 141. If the subject 102 is a person, the person may be holding equipment 122. For example, if the person who becomes the subject 102 is a baseball player, the equipment 122 may be a bat. In this case, the video camera 103 may capture the series of movements of the baseball player swinging the bat at 120 fps, which is four times the standard speed, or 240 fps, which is eight times the standard speed, to create video data 141 (form video data 401). If the subject 102 is a person, that person has a skeleton 121. Characteristic points (characteristic parts) in the skeleton 121 may be treated as feature points 123 (parts). Characteristic points (characteristic parts) in the equipment 122 may also be treated as feature points 123 (parts). Specific examples of feature points 123 (parts) are shown in FIG. 13, which will be described later. The system 101 identifies the degree to which the movement of a certain feature point 123 (part) affects the movement of another feature point 123 (part) (or the interrelationship between feature points 123 (parts)) based on the video data 141 (form video data 401), and analyzes the movement of the subject 102 based on the identified information.

[0017] In this way, when the subject 102 is a person, the video camera 103 captures both the person and the equipment 122 (e.g., a bat) to create video data 141, and the system 101 can analyze the movements of the person and the equipment 122 (e.g., a bat) together.

[0018] 1.2. Calculation of synchronization network data between feature points (regions) The subject 102 has a plurality of feature points 123 (regions). Normally, the movement of each feature point 123 (region) is not completely unrelated to the movements of the other feature points 123 (regions). The movement of each feature point 123 (region) may affect the movement of the other feature points 123 (regions), or may mutually affect each other, or may be correlated or linked with each other. Therefore, in the embodiment of the present disclosure, a network is interpreted as being formed between each of the feature points 123 (regions) of the subject 102, and an index indicating the state of the network is calculated. The synchronization network calculation unit 800 plays a part in this calculation role. The system 101 includes a synchronization network calculation unit 800. The synchronization network calculation unit 800 calculates a time series of synchronization network data 1005(F) between feature points (regions) of the subject 102 based on video data 141 (form video data 401) consisting of a plurality of frame images. Here, the synchronization network data 1005(F) between feature points (regions) indicates whether or not there is a certain degree of interlocking between the feature points 123 (regions) of the subject 102 shown in the video data 141 (form video data 401). 1 schematically shows an example of whether or not there is interlocking between feature points 123 (parts) indicated by inter-feature point (part) synchronization network data 1005(F). Specifically, FIG. 1 shows a schematic graph in which black circles indicate feature points 123 (parts) and arrows between the black circles indicate that the degree of influence from one feature point 123 (part) to the other feature point 123 (part) satisfies a predetermined condition (that there is a certain degree of interlocking). FIG. 1 also shows a plurality of such schematic graphs in a chronological order. In other words, the presence or absence of interlocking between feature points 123 (parts) can change between different synchronization network calculation time periods (Z_nw) (time periods that are units for calculating the synchronization network; see FIG. 11).

[0019] If the shooting speed of the video camera 103 is sufficiently high (for example, 120 fps, which is four times the standard, or 240 fps, which is eight times the standard), the number of frame images included in the video data 141 will be sufficiently large even if the subject 102 moves quickly (the movement is completed in about one second). Therefore, the synchronization network calculation unit 800 can calculate the time series of the inter-feature point (part) synchronization network data 1005(F). Furthermore, the user of the system 101 can perform detailed analysis of the behavior of the subject 102 by using the display or output of the time-series changes in the inter-feature point (part) synchronization network data 1005(F).

[0020] 1.3. Utilizing synchronization network data between feature points (parts) The system 101 may have one or more functional units that calculate indices useful for analyzing the behavior of the subject 102 by utilizing the time series of the inter-feature point (region) synchronization network data 1005(F) calculated by the synchronization network calculation unit 800. FIG. 1 shows a leading role degree calculation unit 900 (and a leading role degree evaluation unit 1500) and a group consciousness index calculation unit 1800 as examples of functional units that calculate indices useful for analyzing the behavior of the subject 102 by utilizing the time series of the inter-feature point (region) synchronization network data 1005(F). Note that while FIG. 1 shows the system 101 as including both the leading role degree calculation unit 900 (and a leading role degree evaluation unit 1500) and the group consciousness index calculation unit 1800, the system 101 may include only one of the leading role degree calculation unit 900 (and a leading role degree evaluation unit 1500) and the group consciousness index calculation unit 1800.

[0021] 1.3.1. Calculating the degree of prominence (PageRank) of each feature point (part) The dominance degree calculation unit 900 calculates a time series of dominance degree data 143 for each feature point 123 (region) based on the time series of inter-feature point (region) synchronization network data 1005(F). The dominance degree of a feature point 123 (region) indicates a certain importance of the action of that feature point 123 (region) in the action of the subject 102. For example, the dominance degree data 143 for each feature point 123 (region) may be one or both of output PageRank data 1203 (OUT PageRank, OUT PR) and input PageRank data 1213 (IN PageRank, IN PR) as shown in FIGS. 4, 11, 12, 13, and 14, which will be described later. The output page rank data 1203 (OUT PageRank, OUT PR) for each feature point (part) is an index indicating the degree to which the feature point 123 (part) in question influences all of the other feature points 123 (parts) other than the feature point 123 in question. The input page rank data 1213 (IN PageRank, IN PR) for each feature point (part) is an index indicating the degree to which the feature point 123 (part) in question is influenced by all of the other feature points 123 (parts) other than the feature point 123 in question. 1 shows a reference to the level of the degree of dominance (output page rank, input page rank) for each feature point (part), based on the aforementioned example of whether or not there is a certain degree of interlocking between feature points 123 (parts) indicated by the inter-feature point (part) synchronization network data 1005(F) shown schematically. That is, in the example of whether or not there is a certain degree of interlocking between feature points 123 (parts) indicated by the inter-feature point (part) synchronization network data 1005(F) shown schematically, the output page rank (OUT PageRank, OUT PR) values ​​of feature points 123 (parts) that are the roots of a relatively large number of directed graphs tend to be large. Also, the input page rank (IN PageRank, IN PR) values ​​of feature points 123 (parts) that are the ends of a relatively large number of directed graphs tend to be large. As will be mentioned in the explanation of Figure 13 below, the values ​​of the degree of dominance (output page rank, input page rank) for each feature point (part) can be highly likely to be used to estimate whether each feature point 123 (part) is the "starting point of an action" or the "ending point of an action," to estimate the "order in which an action is transmitted" between feature points 123 (parts), and to estimate feature points 123 (parts) that have an "important role in an action."

[0022] If the shooting speed of the video camera 103 is sufficiently high and the time width (T) of the synchronization network calculation time period (Z_nw) (shown in FIG. 11) can be made sufficiently small, and the number of synchronization network calculation time periods (Z_nw) included in the video data 141 (form video data 401) is sufficiently large, the number of inter-feature point (region) synchronization network data 1005 (F) calculated based on the video data 141 (form video data 401) will also be sufficiently large. Therefore, the leading role degree calculation unit 900 can calculate the time series of leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each feature point (region. Furthermore, the user of the system 101 can perform a detailed analysis of the behavior of the subject 102 by utilizing the display or output of the time series changes and cumulative values ​​of the dominance data 143 for each feature point (body part) (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)).

[0023] 1-3-2. Evaluation of the prominence (PageRank) of each feature point (part) The leading role degree evaluation unit 1500 calculates various information for evaluation using the time series of leading role degree data 143 (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)) for each feature point (part) calculated by the leading role degree calculation unit 900. For example, the various information calculated by the leading role degree evaluation unit 1500 may clarify the deviation of the behavior of each subject 102 from the standard behavior of a group of subjects 102. It can also be said that this deviation indicates the individuality of the behavior of each subject 102.

[0024] A user of the system 101 can perform a detailed analysis of the behavioral characteristics of a selected subject 102 by viewing a display or output that reveals deviations of the behavior of the individual subject 102 from standard behavior.

[0025] 1-3-3. Calculation of group consciousness index between characteristic points (parts) For example, if the subject 102 is a subject with will (such as a person), the feature points 123 (regions) of the subject 102 are assumed to exhibit mutually associated actions. Here, if each of the feature points 123 (regions) is considered to be an individual (person), and the entire set of the feature points 123 (regions) of the subject 102 is considered to be a group consisting of multiple individuals (persons), then each of the feature points 123 (regions) can be interpreted as if it were an individual (person) behaving with the awareness that it belongs to a group. Based on this interpretation, the group consciousness index calculation unit 1800 indexes the group consciousness of the multiple feature points 123 (regions) of the subject 102. Based on the time series of inter-feature point (part) synchronization network data 1005(F), the collective consciousness index calculation unit 1800 calculates group consciousness index data 1905 for multiple feature points 123 (parts) of the subject 102, or the time series of the group consciousness index data 1905. For example, as shown in Figures 18, 19, and 20 described below, the collective consciousness index calculation unit 1800 may calculate, for each clique number i, collective consciousness index data 1905-i for i cliques, which are cliques involving i feature points 123 (parts) in the network between the feature points 123 (parts), (index k_i indicating the activity level of the entire set of i cliques, index 1 - H_i / H_i(max) indicating the degree of randomness or cooperation between the i cliques).

[0026] If the shooting speed of the video camera 103 is sufficiently high and the duration (T) of the synchronization network calculation time period (Z_nw) (shown in FIG. 19) can be sufficiently small, and the number of synchronization network calculation time periods (Z_nw) included in the video data 141 (form video data 401) is sufficiently large, the number of inter-feature point (part) synchronization network data 1005(F) calculated based on the video data 141 (form video data 401) will also be sufficiently large. In other words, it is possible to ensure a sufficient number of inter-feature point (part) synchronization network data 1005(F) to calculate a certain degree of reliability as the activation probability 1901 (occurrence probability) of each of the i cliques. In this way, it is possible to calculate the collective consciousness index data 1905-i of the i clique for each clique number i (the index (k_i) indicating the activity level of the entire set of i cliques and the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between the i cliques).

[0027] If the video data 141 (form video data 401) includes multiple group consciousness index calculation time periods (Z_gc), which are time periods that serve as units for calculating the group consciousness index (shown in FIG. 19), it becomes possible to calculate the time series of group consciousness index data 1905-i for i cliques for each clique number i (index (k_i) indicating the activity level of the entire set of i cliques, index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques).

[0028] Furthermore, by referring to the display or output of the collective consciousness index data 1905-i of i clique (the index (k_i) indicating the activity level of the entire set of i cliques, and the index (1-H_i / H_i(max)) indicating the randomness or the degree of cooperation between i cliques), the user of the system 101 can confirm the overall activity level or the randomness (degree of cooperation) in the behavior of the subject 102 and confirm the changes over time, thereby enabling a detailed analysis of the behavior of the subject 102.

[0029] The system 101 according to the embodiment of the present disclosure has the above-described functional configuration, and therefore can have the effects described in the above-described [Effects of the Invention].

[0030] 2. Overall configuration including system 101 (Fig. 2) Fig. 2 shows an overall configuration 200 including a system 101 according to an embodiment of the present disclosure. Note that not all of the functional configurations shown in Fig. 2 are essential. Furthermore, the presence of functional configurations other than those shown in Fig. 2 is not prohibited. The location where the subject 102 is photographed by a video camera 103 (labeled "video camera" in Figure 2) and video data 141 (form video data 401) is generated and the location where the system 101 that analyzes the movements of the subject 102 based on the video data 141 (form video data 401) is located may be close to each other or may be separated from each other.

[0031] In a local environment where the video camera 103 and the system 101 are in close proximity, the video camera 103 may transmit the video data 141 (form video data 401) directly to the communication device 308 or external input / output port 309 of the system 101 (shown in Figure 3) via a wired or wireless communication path. Alternatively, the video camera 103 may record the video data 141 (form video data 401) on a recording medium connected to the camera 103. The recording medium may then be connected to the system 101 as an external recording medium 305 (shown in FIG. 3), allowing the system 101 to acquire the video data 141 (form video data 401).

[0032] In a cloud environment in which the video camera 103 and the system 101 are isolated, the video camera 103 may transmit the video data 141 (form video data 401) to the communication device 308 of the system 101 (shown in FIG. 3) via a wired or wireless network 299. Alternatively, the video camera 103 may record the video data 141 (form video data 401) on a recording medium connected to the camera 103. The recording medium may then be transported to a location where the system 101 is located, and the recording medium may be connected to the system 101 as an external recording medium 305 (shown in FIG. 3), thereby allowing the system 101 to acquire the video data 141 (form video data 401).

[0033] As described above, the present disclosure is applicable whether the location where the subject 102 and the video camera 103 are located are close to or far from the location where the system 101 is located.

[0034] 3. Computer Architecture for Implementing Embodiments of the Present Disclosure (FIG. 3) 3 shows a computer architecture 300 for implementing the system 101 of the embodiment of the present disclosure. The computer architecture 300 shown in FIG. 3 may be referred to as an information processing device or an information processing system. To realize the system 101, some or all of the processing unit 301, storage device 302, non-volatile storage medium (storage device) 303, external storage medium drive 304, input device 306, display or output device 307, communication device 308, external input / output port 309, and reading device 310 may be interconnected via an interconnection unit 311. (Note that some or all of the interconnection unit 311 may be a network. In that case, the system 101 is realized by a plurality of devices connected via the network.) The arithmetic processing device 301 may be, for example, a processor. Examples of this processor include a CPU, an MPU, or a GPU. Alternatively, the processor referred to here may be any other semiconductor device that executes predetermined processing. Furthermore, the arithmetic processing device 301 may be one or more (micro)processors. For example, the arithmetic processing device 301 may be a multi-core processor having multiple processing cores (CPU cores). The storage device 302 may be, for example, a memory. The non-volatile recording medium (recording device) 303 may be, for example, a non-volatile memory (e.g., flash memory) or a non-volatile disk device. The external recording medium drive 304 may be, for example, a disk drive. The input device 306 may be, for example, a mouse, keyboard, imaging device, sensor, touch panel, or pointing device. The display or output device 307 may be, for example, a display, printer, or speaker. The communication device 308 may be, for example, a communication device for wired communication or a communication device for wireless communication. The communication device 308 may be a network interface device (NIC) that controls communication with other systems, devices, terminals, or servers according to a predetermined protocol. The interconnection unit 311 may be, for example, a bus or a crossbar switch. (As described above, part or all of the interconnection unit 311 may be a network.)

[0035] The non-volatile recording medium (recording device) 303 may record various programs included in the program group 331 (for example, programs for realizing the functional configuration related to the present disclosure; for example, various programs for implementing each of the functional units realized in the system 101), various data groups included in the data group 332, or information included in the various information 333. The program group 331 may include various programs for realizing each of the functional units indicated as "units" in the functional configuration diagrams of Figures 1 and 4. Some of the above programs may be integrated into one program. Also, any of the above programs may be divided into multiple programs. The data group 332 may include information (data, etc.) handled by the above-mentioned functional units. For example, the data group 332 may include information constituting each of the data groups indicated by dotted-line frames in the functional configuration diagrams of Figures 1 and 4, and the parameter groups indicated by double-lined rectangles in the functional configuration diagram of Figure 4. (Note that some or all of the information contained in the data groups and parameter groups may be stored in the storage device 302 (memory).) Alternatively, some or all of the various programs included in the program group 331, the various data groups (or parameter groups) included in the data group 332, or the information included in the various information 333 may be acquired from outside the configuration shown in FIG. 3.

[0036] The external recording medium drive 304 can be connected to an external recording medium 305. The external recording medium 305 may be, for example, a portable recording disk (such as a DVD), an IC card, an SD card, a nonvolatile memory (such as a flash memory), or a portable hard disk. Various programs included in the program group 331, various data (or parameter groups) included in the data group 332, or information similar to the information included in the various information 333 may be transferred and stored from the external recording medium 305 to the nonvolatile recording medium (recording device) 303 or the storage device 302. The external recording medium 305 may be used to record programs and data handled in the system 101. The external recording medium drive 304 and the external recording medium 305 may be connected to the system 101 shown in FIG. 3 via a network. The various programs included in the program group 331, the various data included in the data group 332, or the information included in the various information 333 may be brought via the communication device 308, the external input / output port 309, the input device 306, or the reading device 310, and recorded or stored in the non-volatile recording medium (recording device) 303 or the storage device 302.

[0037] In order for the architecture of FIG. 3 to function as the system 101, each functional unit within the system 101, or a portion of each functional unit (to execute one or a series of processes (steps)), various programs included in the program group 331 may be loaded into the storage device 302 (for example, from the non-volatile recording medium (recording device) 303). The loaded program is indicated by 321 in FIG. 3. The arithmetic processing device 301 may then execute the program 321 (using, as necessary, various data and the like included in the data group 332 stored in the non-volatile recording medium (recording device) 303, or information included in the various information 333). Execution of the program 321 realizes the function of the system 101, each functional unit within the system 101, or a portion of each functional unit (to execute one or a series of processes (steps)). At this time, various buffers 323 temporarily formed in the storage device 302 may also be used as appropriate.

[0038] 4. Functional configuration of system 101 (Figs. 4 and 5) Fig. 4 shows a functional configuration 400 (and the information handled) of the system 101 according to an embodiment of the present disclosure. Note that not all of the functional configurations shown in Fig. 4 are essential. Furthermore, the presence of functional configurations other than those shown in Fig. 4 is not prohibited. The content of the processing performed by the system 101 shown in Figure 4 will be described in detail in Section "5. Processing Performed by an Embodiment of the Present Disclosure" below. In Section "4. Functional Configuration of System 101", an outline of the content of the processing performed by the system 101 and an outline of the information (data) handled by the system 101 will be described. Note that parts that have already been described with reference to Figure 1 may be omitted below. In addition, in Section "4. Functional Configuration of System 101", an outline of a group of parameters applied to the system 101 will also be described. In Figure 4, solid-lined rectangles with the word "unit" in their names indicate functional units. Furthermore, dotted-lined frames indicate information (data) to be handled. Furthermore, double-lined rectangles indicate parameters (groups) applied to the system 101.

[0039] As shown in FIG. 4 , the system 101 may have, as functional units, a skeletal feature point coordinate recognition unit 610A, an equipment feature point coordinate recognition unit 610B, a skeletal feature point movement amount calculation unit 620A, an equipment feature point movement amount calculation unit 620B, a synchronization determination calculation unit 700, a synchronization network calculation unit 800, a leading role degree calculation unit 900, a leading role degree evaluation unit 1500, a collective awareness index calculation unit 1800, a parameter setting unit 501, a video display output control unit 1401, a movement amount display output control unit 1404, a synchronization network display output control unit 1402, a leading role degree display output control unit 1403, and a collective awareness index display output control unit 2101. In the computer architecture 300 shown in Fig. 3 described above, when a program corresponding to each functional unit is executed to realize each functional unit in software, such software-realized functional unit does not need to be constantly realized in the system 101. For example, when a function provided by a functional unit is needed, the functional unit may be realized in software in the system 101. Also, in the system 101, any of the functional units (or a part of the function of any of the functional units) may be implemented more in hardware.

[0040] As shown in FIG. 4, the system 101 stores, as data indicating information handled by any of the functional units, form video data 401, video management data 403 (data indicating management information for each of the form video data 401), skeleton model data 404A, equipment model data 404B, skeleton feature point coordinate data 405A, equipment feature point coordinate data 405B, skeleton feature point movement amount data 1001A, equipment feature point movement amount data 1001B, random variable movement amount data 1002, frequency distribution data 1003, and feature point synchronization data 406 (feature point (part) ) transfer entropy data 1004(E)), inter-feature point (part) synchronization network data 1005(F), transition matrix data 1201(A), output page rank data 1203, output page rank cumulative value data 1601, output page rank normalized data 1602, output page rank average value data 1603, input page rank data 1213, input page rank cumulative value data 1611, input page rank normalized data 1612, input page rank average value data 1613, and collective consciousness index data 1905. Of the above data, the form video data 401 may be a time series of frame images. Furthermore, of the above data, each of the following may form a time series: skeleton feature point coordinate data 405A, equipment feature point coordinate data 405B, skeleton feature point movement amount data 1001A, equipment feature point movement amount data 1001B, random variable movement amount data 1002, feature point synchronization data 406 (inter-feature point (body part) movement entropy data 1004(E)), inter-feature point (body part) synchronization network data 1005(F), transition matrix data 1201(A), output page rank data 1203, input page rank data 1213, and collective consciousness index data 1905. 4, the system 101 may have vt, gt, f_n, f_p, T, S, ε, M, N, C, and D as parameter(s) used when any of the functional units performs processing. A schematic description of each of the parameters vt, gt, f_n, f_p, T, S, ε, M, N, C, and D is shown in the table of FIG. 5. How the parameters are specifically used when the functional units perform processing will be described in detail later in the section "5. Processing Performed by the Embodiments of the Present Disclosure." Each of the above data and parameters may be recorded as part of data group 332 in non-volatile recording medium (storage device) 303 in computer architecture 300 of Fig. 3 described above. Alternatively, each of the above data and parameters may be stored as part of various buffers 323 in storage device (memory) 302 in computer architecture 300. Alternatively, each of the above data and parameters may be held in a recording medium, storage medium, device, system, server, etc. that is accessible to or communicable with computer architecture 300.

[0041] 4.1. Overview of Functional Parts Below, an overview of the processing of each of the functional units shown in Fig. 4 will be shown. However, what has already been explained together with Fig. 1 may be omitted below.

[0042] 4-1-1. Overview of the feature point coordinate recognition unit and feature point movement amount calculation unit When the subject 102 is a person, the video camera 103 may capture a video of the person holding the equipment 122 (for example, a bat) and use the captured video as form video data 401. Here, it may be appropriate to separately handle the skeletal model created from information obtained by capturing a video of the person and the equipment model created from information obtained by capturing a video of the equipment 122. Therefore, for the process of acquiring or calculating time-series information for each of the feature points 123 (body parts) of the subject 102 based on the form video data 401, the system 101 may have separate functional units that perform processing related to the skeleton 121 and processing related to the equipment 122. In the example of FIG. 4, the system 101 has a skeleton feature point coordinate recognition unit 610A and a skeleton feature point movement amount calculation unit 620A as functional units that perform processing related to the skeleton 121. The system 101 also has an equipment feature point coordinate recognition unit 610B and an equipment feature point movement amount calculation unit 620B as functional units that perform processing related to the equipment 122. (Note that the system 101 may have a single feature point coordinate recognition unit and feature point movement amount calculation unit without distinguishing between the skeleton 121 and the equipment 122. In this case, the skeleton model and the equipment model may not be distinguished from each other and may be considered to be a single model.)

[0043] The skeleton feature point coordinate recognition unit 610A creates skeleton model data 404A based on each of the frame images included in the form movie data 401. Furthermore, the skeleton feature point coordinate recognition unit 610A extracts position information (for example, coordinate information on the frame images) of each of the feature points 123 (parts) in the skeleton 121 of the subject 102 based on the skeleton model data 404A, and generates skeleton feature point coordinate data 405A. Because the form movie data 401 includes a time series of the frame images, the time series of the skeleton feature point coordinate data 405A is calculated. Based on the skeletal feature point coordinate data 405A, the skeletal feature point movement amount calculation unit 620A calculates the amount of movement (for example, the magnitude of the velocity vector of the position and coordinates, or the velocity vector itself) of each of the feature points 123 (parts) in the skeleton 121 of the subject 102, and generates the resulting skeletal feature point movement amount data 1001A. Since the time series of the skeletal feature point coordinate data 405A is handled, the time series of the skeletal feature point movement amount data 1001A is calculated. As will be described later, the skeletal feature point movement amount calculation unit 620A may perform processing using vt, gt, f_n, and f_p from the parameters.

[0044] The equipment feature point coordinate recognition unit 610B creates equipment model data 404B based on each of the frame images included in the form movie data 401. The equipment feature point coordinate recognition unit 610B also extracts position information (for example, coordinate information on the frame images) of each of the feature points 123 (body parts) of the equipment 122 of the subject 102 based on the equipment model data 404B, and generates equipment feature point coordinate data 405B. Because the form movie data 401 includes a time series of frame images, the time series of the equipment feature point coordinate data 405B is calculated. The equipment feature point movement amount calculation unit 620B calculates the amount of movement (for example, the magnitude of the position-coordinate velocity vector, or the velocity vector itself) of each feature point 123 (site) on the equipment 122 of the subject 102 based on the equipment feature point coordinate data 405B, and generates equipment feature point movement amount data 1001B. Since the time series of the equipment feature point coordinate data 405B is handled, the time series of the equipment feature point movement amount data 1001B is calculated. As will be described later, the equipment feature point movement amount calculation unit 620B may perform processing using vt, gt, f_n, and f_p from the parameters.

[0045] 4-1-2. Overview of the synchronization determination calculation unit The synchronization determination calculation unit 700 calculates feature point synchronization data 406, which is an index indicating the degree of influence between feature points 123 (body parts), based on the time series of skeleton feature point movement amount data 1001A and the time series of equipment feature point movement amount data 1001B. As shown in FIGS. 10 and 11 described below, the feature point synchronization data 406 may be, for example, inter-feature point (body part) movement entropy data 1004(E). Note that the feature point synchronization data 406 may be an index of a type other than movement entropy, as long as it is an index indicating the degree of influence between feature points 123 (body parts). For example, the feature point synchronization data 406 may be mutual information, information obtained by correlation analysis, information obtained by regression analysis, or information obtained by multivariate analysis. An example in which movement entropy is used as the feature point synchronization data 406 will be described below.

[0046] As shown in FIG. 11 , which will be described later, the synchronization determination calculation unit 700 calculates one piece of feature point synchronization data 406 (inter-feature point (body part) movement entropy data 1004(E)) (for example, one set of movement entropy for each permutation between feature points 123 (body parts)) based on a group of skeletal feature point movement amount data 1001A and a group of equipment feature point movement amount data 1001B derived from multiple frame images included in the synchronization network calculation time period (Z_nw), which is the time period that serves as the unit for calculating the synchronization network, out of the group of frame images included in the form video data 401. 11, the time width (calculation width) of the synchronous network calculation time period (Z_nw) may be determined by a parameter T. Furthermore, the time difference between adjacent synchronous network calculation time periods (Z_nw) on the time series may be determined by a parameter S. 11, by making the time difference indicated by S smaller than the time span indicated by T, it is possible to increase the likelihood that even an event that occurred within an extremely short period of time among the actions of the subject 102 captured in the form video data 401 will be captured in one of the synchronization network calculation time periods (Z_nw). In other words, it is possible to increase the likelihood that the video analysis results will be appropriate.

[0047] If the shooting speed of the video camera 103 is sufficiently high (for example, 120 fps, which is four times the standard, or 240 fps, which is eight times the standard), the time width (T) of the synchronization network calculation time period (Z_nw) (shown in FIG. 11) can be made sufficiently small, thereby making it possible to sufficiently increase the number of synchronization network calculation time periods (Z_nw) included in the form video data 401. In this case, even if the form video data 401 is about one second long and captures the quick movement of the subject 102, the synchronization determination calculation unit 700 can calculate the time series of the feature point synchronization data 406 (inter-feature point (part) movement entropy data 1004(E)).

[0048] When the synchronization determination calculation unit 700 calculates the inter-feature point (part) transfer entropy data 1004(E), in the process leading up to the calculation of the inter-feature point (part) transfer entropy data 1004(E), it calculates a time series of random variable transfer amount data 1002 and frequency distribution data 1003. Details of the calculation process of the inter-feature point (part) transfer entropy data 1004(E), including the handling of the time series of random variable transfer amount data 1002 and frequency distribution data 1003, will be described later with reference to FIGS. 7, 10, and 11.

[0049] 4-1-3. Overview of the Synchronization Network Calculation Unit The synchronization network calculation unit 800 calculates inter-feature point (part) synchronization network data 1005(F) based on the feature point synchronization data 406 (inter-feature point (part) transfer entropy data 1004(E)). As shown in Figs. 10 and 11 described below, for one piece of feature point synchronization data 406 (inter-feature point (part) transfer entropy data 1004(E)) (for example, one set of transfer entropy for each permutation between feature points 123 (parts)), the synchronization network calculation unit 800 calculates one piece of inter-feature point (part) synchronization network data 1005(F) (one set of information indicating whether or not there is a certain degree of interlocking for each permutation between feature points 123 (parts)). In other words, as shown in Figures 10, 11, and 12 described below, synchronization network data 1005 (F) between one feature point (part) may be information that indicates, for each of the feature points 123 (parts), whether the degree of influence from one feature point 123 (part) to the other feature point 123 (part) satisfies a predetermined condition (whether it can be said that there is a certain degree of interlocking), based on information about a group of frame images included in one of the synchronization network calculation time periods (Z_nw), which are time periods that serve as units for synchronization network calculation.

[0050] 8 and 10 , the synchronization network calculation unit 800 determines information (F(I→J)) on the presence or absence of a certain interlocking from one feature point 123 (site) (I) to the other feature point 123 (site) (J) and information (F(J→I)) on the presence or absence of a certain interlocking from the other feature point 123 (site) (J) to the one feature point 123 (site) (I), based on the feature point synchronization data 406 (an index (e.g., a transfer entropy value (E(I→J))) that indicates the degree of influence from one feature point 123 (site) (I) to the other feature point 123 (site) (J) and an index (e.g., a transfer entropy value (E(J→I))) that indicates the degree of influence from the other feature point 123 (site) (J) to the one feature point 123 (site) (I), which are included in the inter-feature point (site) transfer entropy data 1004 (E). A parameter ε may be used at this time. Specifically, when the difference between an index (for example, the value of transfer entropy (E(I→J))) indicating the degree of influence from one feature point 123 (part) (I) to the other feature point 123 (part) (J) and an index (for example, the value of transfer entropy (E(J→I))) indicating the degree of influence from the other feature point 123 (part) (J) to the one feature point 123 (part) (I) is equal to or greater than the parameter ε, the synchronization network calculation unit 800 may set one of the information (F(I→J)) indicating the presence or absence of a certain interlocking from the one feature point 123 (part) (I) to the other feature point 123 (part) (J) and the information (F(J→I)) indicating the presence or absence of a certain interlocking from the other feature point 123 (part) (J) to the one feature point 123 (part) (I) to indicate that a certain interlocking exists. The larger the value of the parameter ε is set, the less likely it is that information (F(I→J)) about the presence or absence of a certain interlocking relationship from one feature point 123 (part) (I) to the other feature point 123 (part) (J) or information (F(J→I)) about the presence or absence of a certain interlocking relationship from the other feature point 123 (part) (J) to the one feature point 123 (part) (I) indicates that there is a certain interlocking relationship. Conversely, the larger the value of the parameter ε is set, the more prominent the information becomes when information (F(I→J)) about the presence or absence of a certain interlocking relationship from one feature point 123 (part) (I) to the other feature point 123 (part) (J) or information (F(J→I)) about the presence or absence of a certain interlocking relationship from the other feature point 123 (part) (J) to the one feature point 123 (part) (I) indicates that there is a certain interlocking relationship.

[0051] 4.1.4. Overview of the Synchronous Network Display Output Control Unit The synchronous network display output control unit 1402 controls to display or output the time series changes of the synchronous network data 1005 between feature points (parts) in a manner similar to a synchronous network time series playback display window 1412 included in FIG. 14 described below. A user of the system 101 can perform detailed analysis of the behavior of the subject 102 by using the display or output of the time-series changes in the inter-feature point (part) synchronization network data 1005(F).

[0052] 4-1-5. Overview of the main role calculation section The outline of the processing performed by the leading role degree calculation unit 900 has already been explained in the section "1.3.1. Calculation of leading role degree (PageRank) for each feature point (part)" along with FIG. In calculating the output PageRank data 1203 (OUT PageRank, OUT PR) or the input PageRank data 1213 (IN PageRank, IN PR), which are the dominance data 143 for each feature point (part), a time series of inter-feature point (part) synchronous network data 1005 (F) calculated based on information on a group of frame images included in one of the dominance degree (PageRank) calculation time periods (Z_pr), which are time periods serving as a unit for calculating the dominance degree (PageRank), may be used, as shown in FIG. 11 , which will be described later. Here, the dominance degree (PageRank) calculation time period (Z_pr) may include M synchronous network calculation time periods (Z_nw). The number of network calculation time periods (Z_nw) included in the dominance degree (PageRank) calculation time period (Z_pr) may be controlled by a parameter M that can be set in the system 101. Furthermore, a parameter C that can be set in the system 101 may control the time difference (shift width of the dominance degree calculation time period) between adjacent dominance degree (PageRank) calculation time periods (Z_pr) on the time series.

[0053] 11, the synchronization determination calculation unit 700 calculates inter-feature point (part) transfer entropy data 1004(E) for each synchronization network calculation time period (Z_nw) based on information about a group of frame images included in one of the leading role (PageRank) calculation time periods (Z_pr), thereby calculating a total of M pieces (M sets) of inter-feature point (part) transfer entropy data 1004(E). Next, the synchronization network calculation unit 800 calculates inter-feature point (part) synchronization network data 1005(F) based on each of the inter-feature point (part) transfer entropy data 1004(E), thereby calculating M pieces of inter-feature point (part) synchronization network data 1005(F) corresponding to the leading role (PageRank) calculation time period (Z_pr). Then, the leading role degree calculation unit 900 calculates one or both of output PageRank data 1203 (OUT PageRank, OUT PR) and input PageRank data 1213 (IN PageRank, IN PR), which are leading role degree data 143 for each feature point (part) corresponding to one of the leading role degree (PageRank) calculation time periods (Z_pr), based on the synchronization network data 1005 (F) between M feature points (parts) included in the leading role degree (PageRank) calculation time period (Z_pr).

[0054] If the shooting speed of the video camera 103 is sufficiently high and the time width (T) of the synchronization network calculation time period (Z_nw) (shown in FIG. 11) can be made sufficiently small, and the number of synchronization network calculation time periods (Z_nw) included in the form video data 401 is sufficiently large, then the number of inter-feature point (body part) synchronization network data 1005 (F) calculated based on the form video data 401 will also be sufficiently large. In such a situation, the user of the system 101 will be able to try out various settings for the parameters M and C.

[0055] In the process in which the dominance degree calculation unit 900 calculates the dominance degree data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR) or input PageRank data 1213 (IN PageRank, IN PR)) based on the time series of inter-feature point (part) synchronization network data 1005(F), the dominance degree calculation unit 900 calculates the time series of transition matrix data 1201(A). Details of the calculation process for the dominance degree data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR) or input PageRank data 1213 (IN PageRank, IN PR)), including the handling of the transition matrix data 1201(A), will be described later with reference to FIGS. 9, 11, 12, and 13.

[0056] 4.1.6.Outline of the Leading Role Evaluation Section The outline of the processing performed by the leading role degree evaluation unit 1500 has already been explained in the section "1.3.2. Evaluation of leading role degree (PageRank) for each feature point (portion)" along with FIG. 16 and 17, which will be described later, the leading role degree evaluation unit 1500 may calculate a cumulative value for each feature point (part) over a certain period of time based on the time series of leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each feature point (part), and calculate leading role degree cumulative value data (output PageRank cumulative value data 1601, input PageRank cumulative value data 1611). Next, the leading role degree evaluation unit 1500 may normalize the leading role degree cumulative value data (output PageRank cumulative value data 1601, input PageRank cumulative value data 1611) for each subject 102, and calculate leading role degree normalized data (output PageRank normalized data 1602, input PageRank normalized data 1612). Furthermore, the leading role degree evaluation unit 1500 may calculate the average value for each feature point for the leading role degree normalized data (output page rank normalized data 1602, input page rank normalized data 1612) for each subject 102 to calculate leading role degree average value data (output page rank average value data 1603, input page rank average value data 1613). This leading role degree average value data (output page rank average value data 1603, input page rank average value data 1613) may indicate the standard behavior of the subjects 102 in a group.

[0057] 4-1-7. Overview of the main character display output control unit The leading role degree display output control unit 1403 may be capable of performing control so as to display or output time-series changes in the leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each feature point (part). Furthermore, the leading role degree display output control unit 1403 may be capable of performing control so as to display or output the cumulative value of the leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each feature point (part), in a form like a leading role degree graph display window 1413 for displaying cumulative values ​​included in FIG. 14 described below. In other words, the user of the system 101 can perform a detailed analysis of the behavior of the subject 102 by using the display or output of the time series changes and cumulative values ​​of the dominance data 143 for each feature point (body part) (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)).

[0058] The leading role degree display output control unit 1403 may be capable of controlling the leading role degree average value data (output page rank average value data 1603, input page rank average value data 1613) to be displayed or output in a manner that allows comparison with the leading role degree normalized data of the selected subject 102 (output page rank normalized data 1602, input page rank normalized data 1612), in a manner similar to the leading role degree graph display window 1713 for comparative display with the average value shown in Figure 17 described below. The user of the system 101 can perform a detailed analysis of the behavioral characteristics of the selected subject 102 by referring to the above-mentioned contrasted displays or outputs.

[0059] 4-1-8. Overview of the Collective Consciousness Index Calculation Unit The outline of the processing performed by the collective consciousness index calculation unit 1800 has already been explained in the section "1.3.3. Calculation of collective consciousness index between feature points (parts)" along with FIG. In calculating the collective consciousness index data 1905-i for i clique (an index (k_i) indicating the activity level of the entire set of i cliques, and an index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques), a time series of inter-feature point (location) synchronization network data 1005(F) calculated based on information on a group of frame images included in one of the collective consciousness index calculation time slots (Z_gc), which are time slots that serve as units for calculating the collective consciousness index, may be used, as shown in FIG. 19 . Here, the collective consciousness index calculation time slot (Z_gc) may include N synchronization network calculation time slots (Z_nw). The number of network calculation time slots (Z_nw) included in the collective consciousness index calculation time slot (Z_gc) may be controlled by a parameter N that can be set in the system 101. Furthermore, a parameter D that can be set in the system 101 may be used to control the time lag (shift width of the collective consciousness index calculation time period) between adjacent collective consciousness index calculation time periods (Z_gc) on the time series.

[0060] 19, the synchronization determination calculation unit 700 calculates inter-feature point (part) transfer entropy data 1004(E) for each synchronization network calculation time period (Z_nw) based on information about a group of frame images included in one of the collective consciousness index calculation time periods (Z_gc), thereby calculating a total of N pieces (N sets) of inter-feature point (part) transfer entropy data 1004(E). Next, the synchronization network calculation unit 800 calculates inter-feature point (part) synchronization network data 1005(F) based on each of the inter-feature point (part) transfer entropy data 1004(E), thereby calculating N pieces of inter-feature point (part) synchronization network data 1005(F) corresponding to the collective consciousness index calculation time period (Z_gc). Then, the collective consciousness index calculation unit 1800 calculates the collective consciousness index data 1905-i (index (k_i) indicating the activity level of the entire set of i cliques, index (1-H_i / H_i(max)) indicating the randomness or cooperation level between the i cliques) of the i cliques corresponding to one of the collective consciousness index calculation time periods (Z_gc) based on the synchronization network data 1005(F) between N feature points (locations) included in the collective consciousness index calculation time period (Z_gc).

[0061] If the shooting speed of the video camera 103 is sufficiently high and the time width (T) of the synchronization network calculation time period (Z_nw) (shown in FIG. 19) can be made sufficiently small, and the number of synchronization network calculation time periods (Z_nw) included in the form video data 401 is sufficiently large, the number of inter-feature point (body part) synchronization network data 1005 (F) calculated based on the form video data 401 will also be sufficiently large. In such a situation, the user of the system 101 will be able to try out various settings for the parameters N and D.

[0062] 4-1-9. Overview of the Collective Awareness Index Display Output Control Unit The collective consciousness index display output control unit 2101 can control to display or output as a plot or a time series display or output of the collective consciousness index data 1905-i of i cliques for each number of cliques i (index (k_i) indicating the activity level of the entire set of i cliques, index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques), in a manner similar to time series display 2111-2 of the collective consciousness index for two cliques and time series display 2111-3 of the collective consciousness index for three cliques included in FIG. 21 described below. By referring to such a display or output, the user of the system 101 can check the overall activity and randomness (coordination) of the movements of the subject 102, as well as check the changes in these over time, thereby enabling a detailed analysis of the movements of the subject 102.

[0063] 4.1.10. Overview of the video display output control unit The video display output control unit 1401 controls the form video data 401 to be displayed or output in a format similar to a video playback display window 1411 included in FIG. 14, which will be described later. This display or output may be accompanied by a display or output that highlights the skeleton 121. The video display output control unit 1401 may cooperate with the synchronization network display output control unit 1402 to control the display of the video playback of the form video data 401 and the display of the playback showing the time-series changes in the inter-feature point (part) synchronization network data 1005(F) to be synchronized on the screen (for example, on the example screen shown in FIG. 14). Users of the system 101 can view the video playback of the form video data 401 while verifying the time series changes in the feature point (body part) synchronization network data 1005 (F), allowing them to perform a detailed analysis of the movements of the subject 102.

[0064] 4-1-11.Outline of the movement amount display output control section The movement amount display output control unit 1404 can control the display or output of graphs, tables, etc. showing the movement amount for each feature point 123 (body part) (for example, the magnitude of the velocity vector of the position / coordinate (on the frame image), or the velocity vector itself) based on the time series of the skeleton feature point movement amount data 1001A and the time series of the equipment feature point movement amount data 1001B. The movement amount display output control unit 1404 can cooperate with the synchronization network display output control unit 1402 and the video display output control unit 1401 to control the display of video playback of the form video data 401 and the display of playback showing the time series changes in the inter-feature point (body part) synchronization network data 1005(F) and the display showing the time series changes in the graphs, tables, etc. showing the movement amount for each feature point 123 (body part) so as to synchronize them on the screen. Users of the system 101 can view the video playback of the form video data 401 and the time series changes in the feature point (body part) synchronization network data 1005 (F), while also verifying the time series changes in the amount of movement for each feature point 123 (body part), allowing them to perform a detailed analysis of the movements of the subject 102.

[0065] 4.1.12.Overview of the parameter setting section The parameter setting unit 501 sets one or more of vt, gt, f_n, f_p, T, S, ε, M, N, C, and D, which are parameters that can be set in the system 101, based on input from a user of the system 101. The parameter setting unit 501 may set one or more of the parameters vt, gt, f_n, f_p, T, S, ε, M, N, C, and D in response to parameter setting input by the user of the system 101 via, for example, a slide bar 511 for synchronization network threshold (ε) and a various parameter setting field 519 included in Figures 14 and 21 described below, a slide bar 512 for synchronization network set coefficient for dominance degree (M) and a slide bar 514 for dominance degree calculation time zone shift width (C) included in Figure 14, or a slide bar 513 for synchronization network set coefficient for collective consciousness index (N) and a slide bar 515 for collective consciousness index calculation time zone shift width (D) included in Figure 21. In addition, the parameter setting unit 501 may set one or more of the parameters vt, gt, f_n, f_p, T, S, ε, M, N, C, and D in response to parameter setting input by a user of the system 101 via a user interface dedicated to parameter setting (e.g., a screen on which parameter values ​​can be input). An overview of each of the parameters vt, gt, f_n, f_p, T, S, ε, M, N, C, and D that can be set in the system 101 will be explained in "4.2. Overview of Parameters." In addition, the specific usage of each parameter will be explained together with the details of the process in which the parameter is used.

[0066] 2. Parameter Overview Below is an overview of the parameters that are referenced when the functional units shown in FIG. 4 perform processing. Fig. 5 shows a parameter group 500 that is applied to the system 101. The table shown in Fig. 5 shows, for each parameter type, the parameter name, the meaning of the parameter, a recommended value as the parameter setting, and the application of the parameter. Below, an overview of each of the parameters that can be set in the system 101 will be explained in order.

[0067] The parameter v_t is the velocity threshold of the feature point 123 (part). In video data 141 (form video data 401) obtained by video capture using a video capture camera 103 with a shooting speed higher than the standard (30 fps), small blurs and noises tend to occur in the coordinates and speeds (amounts of movement) of feature points 123 (parts) of the subject 102. On the other hand, in analyzing the movements of the subject 102, it is often more useful to remove such blurs and noises. Therefore, the skeleton feature point movement amount calculation unit 620A and the equipment feature point movement amount calculation unit 620B of the system 101 may be configured to treat the velocity (movement amount) of a feature point 123 (part) having an absolute value equal to or less than the threshold indicated by the parameter v_t as zero (zeroing it). A recommended value for the parameter v_t may be, for example, 0.2 [pixel / frame] when the amount of movement between adjacent frame images in a time series (for example, the amount obtained by multiplying the magnitude of the velocity vector by the time interval between the frame images) is expressed as the number of pixels on the frame image. Note that although the user of the system 101 can set the parameter v_t, it is desirable not to set the parameter v_t to any value other than the recommended value.

[0068] The parameter g_t is the velocity threshold of the center of gravity. The parameters v_t and g_t have the same meaning and significance, except that the parameter v_t is a threshold value for the speed (amount of movement) related to the coordinates of the feature point 123 (part) of the subject 102, while the parameter g_t is a threshold value for the speed (amount of movement) related to the coordinates of the center of gravity of the subject 102. Furthermore, the coordinates and speed (amount of movement) of the center of gravity of the subject 102 may be calculated, for example, from the video data 141 (form video data 401) and the estimated volume and estimated specific gravity of the area around each of the feature points 123 (parts) of the subject 102. A recommended value for the parameter g_t may be, for example, 0.02 [pixel / frame] when the amount of movement between adjacent frame images in the time series (for example, the amount obtained by multiplying the magnitude of the velocity vector by the time interval between the frame images) is expressed as the number of pixels on the frame image. Note that although the user of the system 101 can set the parameter g_t, it is desirable not to set the parameter g_t to any value other than the recommended value.

[0069] The parameter f_n is a velocity filter. As already pointed out in the explanation of the parameter v_t, in analyzing the movement of the subject 102, it is often more useful to remove blur and noise from the coordinates and speed (amount of movement) of the feature points 123 (parts). Therefore, the skeleton feature point movement amount calculation unit 620A and the equipment feature point movement amount calculation unit 620B of the system 101 may be configured to apply a smoothing method (filter) for removing the above-mentioned blur and noise to the speed (movement amount) of the feature point 123 (part) having an absolute value exceeding the threshold indicated by the above-mentioned parameter v_t. The parameter f_n indicates the type of smoothing method (filter) used by the skeleton feature point movement amount calculation unit 620A and the equipment feature point movement amount calculation unit 620B. If the type of smoothing method (filter) can be set, an appropriate smoothing method (filter) can be selected depending on the aspect of the movement captured in the video data 141 (form video data 401). The recommended value of the parameter f_n may be, for example, "Savgol," which means a Savitzky-Golay filter. The user of the system 101 may set the parameter f_n to indicate another type of smoothing method (filter). The user of the system 101 may also set the parameter f_n to "no smoothing (no filter)," which means that no smoothing method (filter) is applied to the speed (movement amount) of the feature point 123 (part).

[0070] The parameter f_p is a velocity filter parameter that indicates the detailed settings when the smoothing method (filter) set by the parameter f_n is performed. The content to be set as parameter f_p differs depending on the type of smoothing method (filter) set by parameter f_n. For example, if the type of smoothing method (filter) is one in which movement amount data at N_time adjacent times on the time series is used to express a function indicating the movement amount with respect to time by an approximation formula that is an N_degree degree polynomial, then parameter f_p will be N_time and N_degree. When the parameter f_p is made up of N_time and N_degree, the recommended values ​​of N_time and N_degree vary depending on the situation, but for example, the value of N_time may be 11 and the value of N_degree may be 5. In this case, an approximate equation that is a fifth-order polynomial is obtained based on 11 time-series data related to the amount of movement. As described above, the parameter f_p can be set in more detail depending on the type of smoothing method (filter), thereby enabling fine adjustment of the smoothing method (filter) according to the state of the form video data 401.

[0071] The parameter T is the time width (synchronous network calculation width) of the synchronous network calculation time period (Z_nw). As shown in Figures 11 and 19, the parameter T is used to set the duration of the synchronous network calculation time period (Z_nw), which is a time period in the video data 141 (form video data 401) for calculating one (one set) of inter-feature point (part) transfer entropy data 1004 (E) and one (one set) of inter-feature point (part) synchronous network data 1005 (F). For example, if the video data 141 (form video data 401) is 240 fps and the parameter T is set to 0.25 seconds, 60 frame images correspond to one synchronization network calculation time period (Z_nw). In this case, one (one set) of inter-feature point (part) transfer entropy data 1004(E) and one (one set) of inter-feature point (part) synchronization network data 1005(F) are calculated based on information obtained from the 60 frame images. A recommended value for parameter T may be, for example, 0.25 seconds. Although parameter T can be set by the user of system 101, it is desirable not to reduce parameter T significantly below the recommended value from the viewpoint of obtaining appropriate inter-feature point (region) transfer entropy data 1004(E) and inter-feature point (region) synchronization network data 1005(F). For example, it is desirable that the number of frame images included in the time period determined by parameter T be 30 or more. As described above, the time width (synchronization network calculation width) of the synchronization network calculation time period (Z_nw) can be determined by setting the parameter T, and therefore, adjustments can be made to obtain appropriate inter-feature point (part) movement entropy data 1004(E) and inter-feature point (part) synchronization network data 1005(F) depending on the state of the form video data 401. Incidentally, when a camera 103 for shooting moving images with a higher shooting speed than the standard shooting speed (30 fps) is used, the moving image data 141 (form moving image data 401) may be conventionally expressed as a time when stretched along the time axis to be equivalent to the standard shooting speed (30 fps). In accordance with such convention, for example, in moving image data 141 (form moving image data 401) obtained at a shooting speed of 240 fps, which is eight times the standard shooting speed (30 fps), in order to set 0.25 seconds as the value of parameter T, the notation "2 seconds (= 2000 milliseconds)", which is eight times 0.25 seconds, may be used in setting parameter T.

[0072] The parameter S is the time difference (synchronization network shift width) between adjacent synchronization network calculation time periods (Z_nw) on the time series. 11 and 19, for example, when the parameter S is set to 0.125 seconds, eight (eight sets) pieces of inter-feature point (region) transfer entropy data 1004(E) and eight (eight sets) pieces of inter-feature point (region) synchronization network data 1005(F) can be calculated per second. Also, as is clear from Fig. 11 and 19, when the value of parameter S is set to be smaller than the value of parameter T, information on a frame image at a certain time can be reflected in multiple (multiple sets) pieces of inter-feature point (region) transfer entropy data 1004(E) and multiple (multiple sets) pieces of inter-feature point (region) synchronization network data 1005(F). The recommended value for parameter S depends on the case. For example, parameter S may be set to 0.125 seconds. As described above, by setting the parameter S, it is possible to determine the time lag (synchronization network shift width) between adjacent synchronization network calculation time periods (Z_nw) on the timeline. Therefore, depending on the state of the form video data 401, it is possible to adjust, for example, the number of inter-feature point (part) movement entropy data 1004(E) and inter-feature point (part) synchronization network data 1005(F) calculated per second. If the convention described in the explanation of parameter T is followed, for example, in video data 141 (form video data 401) obtained at a shooting speed of 240 fps, which is eight times the standard shooting speed (30 fps), in order to set 0.125 seconds as the value of parameter S, the notation "1 second (= 1000 milliseconds)", which is eight times 0.125 seconds, may be used in setting parameter S.

[0073] The parameter ε is the synchronization network threshold. 8 and 10, the parameter ε is used as a threshold when the synchronization network calculation unit 800 determines whether there is a certain degree of interrelationship between feature points 123 (regions). If it is assumed that the inter-feature point (region) synchronization network data 1005(F) is represented graphically, the parameter ε may be used as a threshold when determining whether to set a directed graph indicating that there is a certain degree of interrelationship between feature points (regions). Details of how the parameter ε is used will be described later with reference to FIGS. 8 and 10. When the value of the parameter ε is large, it becomes difficult to set the directed graph described above. Conversely, when the value of the parameter ε is large, the set directed graph becomes more prominent. By setting the value of the parameter ε, the user of the system 101 can obtain a time series of inter-feature point (body part) synchronization network data 1005(F) that matches the behavior of the subject 102 included in the video data 141 (form video data 401) and the purpose of the analysis of the behavior.

[0074] The parameter M is the synchronization network aggregation coefficient for the leading role. As shown in FIG. 11 , based on the number (number of sets) M determined by the parameter M, the leading role degree calculation unit 900 uses M pieces (M sets) of inter-feature point (region) synchronization network data 1005(F) to calculate leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each one (one set) of feature points. As shown in FIG. 11 , the leading role degree calculation unit 900 uses M pieces (M sets) of inter-feature point (region) synchronization network data 1005(F) to directly calculate one piece of transition matrix data 1201(A). The processing of the leading role degree calculation unit 900, including the handling of the transition matrix data 1201(A), will be described later with reference to FIGS. 9 , 11 , 12 , and 13 . As the value of parameter M increases, the number of frame images in video data 141 (form video data 401) reflected in the dominance data 143 for each (one set) of feature point (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) increases (the length of time increases). Therefore, as parameter M increases, there is a higher possibility that, among the movements of subject 102, linkages of movements that take a long time to be transmitted between feature points 123 (regions) will be sufficiently reflected in the dominance data 143 for each (one set) of feature point (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)). By setting the value of parameter M, the user of system 101 can obtain a time series of leading role data 143 (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)) for each feature point that is suited to the manner of movement of subject 102 contained in video data 141 (form video data 401) and the purpose of analyzing the movement. The parameter N is the synchronization network aggregation coefficient for the collective consciousness index. As shown in FIG. 19 , based on the number (number of sets) N determined by the parameter N, the collective consciousness index calculation unit 1800 uses N pieces (N sets) of feature point (location) synchronization network data 1005(F) to calculate group consciousness index data 1905-i for one (one set) i clique (index (k_i) indicating the activity level of the entire set of i cliques, and index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques). As shown in FIG. 19 , the collective consciousness index calculation unit 1800 uses N pieces (N sets) of feature point (location) synchronization network data 1005(F) to directly calculate occurrence probability data 1901-i(p_i,j) for i clique. The processing of the collective consciousness index calculation unit 1800, including the handling of the occurrence probability data 1901-i(p_i,j) for i clique, will be described later with reference to FIGS. 18 , 19 , and 20 . As the value of the parameter N increases, the number of frame images in the video data 141 (form video data 401) reflected in the collective consciousness index data 1905-i of one (one set) of i cliques (the index (k_i) indicating the activity level of the entire set of i cliques, and the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques increases (the length of time increases). Therefore, as the parameter N increases, the linkage of the movements of the subject 102 that take a long time to be transmitted between the feature points 123 (regions) increases in the collective consciousness index data 1905-i of one (one set) of i cliques. This is likely to be sufficiently reflected in the index (k_i) indicating the activity level of the entire set of i cliques and the index (1-H_i / H_i(max)) indicating the randomness or degree of cooperation between i cliques. By setting the value of parameter N, the user of system 101 can obtain the time series of collective consciousness index data 1905-i of i cliques (index (k_i) indicating the activity level of the entire set of i cliques and the index (1-H_i / H_i(max)) indicating the randomness or degree of cooperation between i cliques) that suits the behavior of subject 102 included in video data 141 (form video data 401) and the purpose of behavior analysis.

[0075] Parameter C indicates (in the form of a scaling factor for parameter S) the time lag (shift width of the dominance degree calculation time zone) between adjacent time zones (Z_pr) for calculating the dominance degree (PageRank) on the time series. The value of parameter C is a natural number. The value of parameter C can usually be set to any value between 1 and M. As shown in FIG. 11, the value of C*S, which is the product of the value of parameter C and the value of parameter S indicating the aforementioned synchronization network shift width, indicates the time lag (shift width of the dominance degree calculation time zone) itself between adjacent time zones (Z_pr) for calculating the dominance degree (PageRank) on the time series. As shown in Figure 11, when the value of parameter C is set to the same value as the value of parameter M, each of the synchronization network calculation time periods (Z_nw) is included in only one of the dominance degree (PageRank) calculation time periods (Z_pr). On the other hand, when the value of parameter C is set to a value smaller than the value of parameter M, any of the synchronization network calculation time periods (Z_nw) can be included in multiple dominance degree (PageRank) calculation time periods (Z_pr). In this way, the setting of parameter C makes it possible to flexibly determine the dominance degree (PageRank) calculation time periods (Z_pr). Parameter D indicates (as a scaling factor for parameter S) the time lag (the collective consciousness index calculation time zone shift width) between adjacent collective consciousness index calculation time zones (Z_gc) on the time series. The value of parameter D is a natural number. The value of parameter D can usually be set to any value between 1 and N. As shown in FIG. 19, the value of D*S, which is the product of the value of parameter D and the value of parameter S indicating the aforementioned synchronization network shift width, indicates the time lag (the collective consciousness index calculation time zone shift width) itself between adjacent collective consciousness index calculation time zones (Z_gc) on the time series. As shown in Figure 19, when the value of parameter D is set to the same value as the value of parameter N, each synchronization network calculation time slot (Z_nw) is included in only one collective consciousness index calculation time slot (Z_gc). On the other hand, when the value of parameter D is set to a value smaller than the value of parameter N, any synchronization network calculation time slot (Z_nw) can be included in multiple collective consciousness index calculation time slots (Z_gc). In this way, the setting of parameter D allows for flexible determination of the collective consciousness index calculation time slots (Z_gc).

[0076] 5. Processing performed by the embodiment of the present disclosure The following describes the processing performed by an embodiment of the present disclosure (system 101). Note that it is not necessary to realize all of the functional configurations and perform all of the processing described below. Furthermore, it is not prohibited to realize functional configurations and perform processing other than the functional configurations and processing described below. Furthermore, the steps of the processes described below may be combined to form a method executed by a system (information processing device or information processing system).

[0077] 5-1. Processing of the feature point coordinate recognition unit and feature point movement amount calculation unit (Fig. 6, Fig. 4) FIG. 6 shows a flowchart of the processing executed by the feature point coordinate recognition units, namely, the skeleton feature point coordinate recognition unit 610A and the equipment feature point coordinate recognition unit 610B, and the feature point movement amount calculation units, namely, the skeleton feature point movement amount calculation unit 620A and the equipment feature point movement amount calculation unit 620B. The processing will be described below in the order shown in FIG. 6, with appropriate reference to the functional configuration shown in FIG. 4. Of the processing steps in the flowchart of FIG. 6, each processing step executed by the feature point coordinate recognition unit may be considered to form a "feature point coordinate recognition step." Furthermore, of the processing steps in the flowchart of FIG. 6, each processing step executed by the feature point movement amount calculation unit may be considered to form a "feature point movement amount calculation step." The functions described below are realized, so that time-series information (e.g., information regarding the amount of movement) for each feature point 123 (part) can be obtained based on the form video data 401, which can be easily used to analyze the movements of the subject 102.

[0078] In step 601A of Fig. 6, the skeletal feature point coordinate recognition unit 610A extracts a skeletal model based on the form video data 401. The skeletal feature point coordinate recognition unit 610A extracts the skeletal model, for example, by inputting information about a group of frame images included in the form video data 401 into a functional unit equivalent to a deep neural network that has undergone machine learning. The skeletal model is accompanied by a plurality of feature points 123 (parts) as exemplified in Fig. 13. The skeletal feature point coordinate recognition unit 610A stores information about the extracted skeletal model as skeletal model data 404A within the system 101 or in a medium or the like external to the system 101 that can be used by the system 101. In step 602A of FIG. 6, the skeletal feature point coordinate recognition unit 610A extracts the coordinates (position information) of each of the feature points 123 (parts) included in the skeleton 121 based on the skeletal model data 404A (and, in some cases, the form movie data 401). The extracted coordinates (position information) may indicate coordinates in a frame image. The skeletal feature point coordinate recognition unit 610A stores the coordinates (position information) of each of the extracted feature points 123 (parts) as skeletal feature point coordinate data 405A in the system 101 or in a medium external to the system 101 that can be used by the system 101. Since the form movie data 401 is a time series of frame images, the time series of the skeletal feature point coordinate data 405A is stored. (Note that the skeletal feature point coordinate recognition unit 610A may also calculate and store the coordinates (position information) of the center of gravity, as necessary.) When one piece of form moving image data 401 taken by one camera 103 for shooting moving images is handled for one subject 102, the coordinates (position information) of each of the feature points 123 (regions) may be coordinates (position information) in a two-dimensional coordinate system. On the other hand, when multiple pieces of form moving image data 401 taken by multiple cameras 103 for shooting moving images are handled for one subject 102, the coordinates (position information) of each of the feature points 123 (regions) may be coordinates (position information) in a three-dimensional coordinate system.

[0079] In step 603A of FIG. 6, skeletal feature point movement amount calculation unit 620A calculates information regarding the speed (movement amount) of each feature point 123 (part) based on the time series of skeletal feature point coordinate data 405A. For example, for each feature point 123 (part), skeletal feature point movement amount calculation unit 620A may calculate the magnitude of a velocity vector indicating the change in coordinate per unit time as the movement amount data. Alternatively, the velocity vector itself may be used as the movement amount data. skeletal feature point movement amount calculation unit 620A stores the calculated movement amount data for each feature point 123 (part) as skeletal feature point movement amount data 1001A in system 101 or in a medium external to system 101 that can be used by system 101. Because form video data 401 is a time series of frame images, the time series of skeletal feature point movement amount data 1001A is stored. (Note that skeletal feature point movement amount calculation unit 620A may also calculate and store movement amount data of the center of gravity as necessary.) 6, the skeletal feature point movement amount calculation unit 620A may perform zeroing and smoothing (filtering) on ​​the time series of the skeletal feature point movement amount data 1001A. (Note that the skeletal feature point movement amount calculation unit 620A may also perform zeroing and smoothing (filtering) on ​​the time series of the movement amount data of the center of gravity, as necessary.) For example, the skeletal feature point movement amount calculation unit 620A may set to zero (zero out) the value of the velocity (feature amount) of a feature point 123 (part) having an absolute value equal to or less than the threshold indicated by the parameter v_t in the time series of the skeletal feature point movement amount data 1001A. Furthermore, when the time series of movement amount data of the center of gravity for the subject 102 has also been calculated, the skeletal feature point movement amount calculation unit 620A may set to zero (zero out) the value of the velocity (feature amount) of the center of gravity having an absolute value equal to or less than the threshold indicated by the parameter g_t. The skeletal feature point movement amount calculation unit 620A may perform smoothing processing (filtering processing) on ​​the time series of the skeletal feature point movement amount data 1001A, for example, based on the type of smoothing method (filter) indicated by the parameter f_n, using the filter parameter indicated by the parameter f_p.

[0080] The above steps 601A, 602A, 603A, and 604A are processes performed in relation to the skeleton 121 of the subject 102, but if the subject 102 is a person and the person is holding an implement 122 (for example, a bat), similar processes may be performed in relation to the implement 122. The processes performed in relation to the implement 122 are steps 601B, 602B, 603B, and 604B. Step 601B is the same process as step 601A. However, the main processing unit of step 601B is the equipment feature point coordinate recognition unit 610B. The model data created and stored is the equipment model data 404B. Step 602B is the same process as step 601B. However, the processing entity of step 602B is the equipment feature point coordinate recognition unit 610B. Furthermore, the feature point coordinate data (time series) extracted and stored is the equipment feature point coordinate data 405B (time series). Step 603B is the same process as step 603A. However, the processing entity of step 603B is the equipment feature point movement amount calculation unit 620B. Furthermore, the feature point movement amount data (time series) calculated and stored is equipment feature point movement amount data 1001B (time series). Step 604B is the same process as step 604A. However, the processing entity of step 604B is the equipment feature point movement amount calculation unit 620B. Furthermore, the feature point movement amount data (time series) handled is the equipment feature point movement amount data 1001B (time series).

[0081] Note that if the processing becomes more efficient or the validity of the processing results improves when the skeleton model and the equipment model are treated in a unified manner, the processing shown in Fig. 6 may be performed in an integrated manner for both the skeleton 121 and the equipment 122. In that case, the system 101 may be provided with a single feature point coordinate recognition unit (which does not distinguish between the skeleton 121 and the equipment 122) and a single feature point movement amount calculation unit.

[0082] 5-2. Processing of the synchronization determination calculation unit (transfer entropy calculation) (Figures 7, 10, 11, and 4) Fig. 7 shows a flowchart of the processing executed by the synchronization determination calculation unit 700. Below, the processing will be explained in the order shown in Fig. 7, with appropriate reference to the functional configuration shown in Fig. 4, the handled data group 1000 shown in Fig. 10, and the handled data group 1100 shown in Fig. 11. Note that each of the processing steps performed by the synchronization determination calculation unit 700 in the flowchart of Fig. 7 may be considered to form a "synchronization determination calculation step" (or a "transfer entropy calculation step" if transfer entropy is calculated). By realizing the functions described below, a time series of an index indicating the degree of influence between feature points 123 (regions) can be calculated based on a time series of information for each feature point 123 (region) (e.g., a time series of movement amount data). For example, if the type of index is movement entropy, a time series of movement entropy data 1004(E) between feature points (regions) can be calculated. When such a time series of an index (e.g., a time series of movement entropy data 1004(E) between feature points (regions)) is obtained, it becomes possible to analyze the relationship between feature points 123 (regions) in analyzing the movements of the subject 102, thereby enabling detailed movement analysis. Furthermore, when handling combinations (permutations) of feature points 123 (body parts) to calculate an index (for example, inter-feature point (body part) movement entropy data 1004(E)) indicating the degree of influence between feature points 123 (body parts), the synchronization determination calculation unit 700 handles the feature points 123 (body parts) of the skeleton 121 without making any particular distinction between the feature points 123 (body parts) of the equipment 122. In other words, it can be said that the synchronization determination calculation unit 700 handles the time series of the feature points 123 (body parts) and movement amount data related to the skeleton 121 together with the time series of the feature points 123 (body parts) and movement amount data related to the equipment 122. In this way, even if the subject 102 includes the skeleton 121 and the equipment 122, the skeleton 121 and the equipment 122 can be handled together in analyzing the movements of the subject 102. In the following, a case will be described in which the inter-feature-point (region) transfer entropy data 1004(E) is calculated as an index indicating the degree of influence between the feature points 123 (regions). As described above, other types of indexes may also be calculated.

[0083] 7, the synchronization determination calculation unit 700 acquires a parameter T indicating the time width (synchronization network calculation width) of the synchronization network calculation time period (Z_nw) and a parameter S indicating the time difference (synchronization network shift width) between adjacent synchronization network calculation time periods (Z_nw) on the timeline. By acquiring the parameters T and S, the synchronization determination calculation unit 700 grasps the start and end of each synchronization network calculation time period (Z_nw) in the manner shown in FIG. 11 or 19. The synchronization network calculation time period (Z_nw) is also the time period that serves as the unit for calculating the inter-feature point (site) transfer entropy data 1004(E).

[0084] In step 702 of FIG. 7, the synchronization determination calculation unit 700 classifies the time series of movement amount data for each of the feature points 123 (body parts) included in the skeleton 121 and the equipment 122, based on the time series of skeleton feature point movement amount data 1001A and the time series of equipment feature point movement amount data 1001B. Whether the movement amount data is the magnitude of a velocity vector or the velocity vector itself (two-dimensional or three-dimensional), the movement amount data can take on any of many values, and therefore the movement amount data itself is difficult to treat as a random variable (a variable in which each variable value (each combination of variable values) has a probability) that serves as the basis for calculating the movement entropy data 1004(E) between feature points (parts). Therefore, in step 702, several bins are set, and for each bin, a range of values ​​for the movement amount data corresponding to that bin is set (a range of values ​​if the movement amount data is the magnitude of a velocity vector; if the movement amount data is the velocity vector itself, a range of values ​​for each coordinate axis of the velocity vector). These bins become the values ​​that the discretized random variable can take. The number of classes (the number of possible class values) may be determined arbitrarily. For example, according to Sturgess's law, if the number of movement amount data included in the time series of movement amount data of one feature point 123 (part) is N_v (here, for example, N_v may be the number of time series data (movement amount data) of one feature point 123 (part) in one synchronization network calculation time period (Z_nw)), and log(N_v) is the logarithm of N_v with base 2, the number of classes (the number of possible class values) may be determined as log(N_v) + 1 (or a natural number close to this value). The range of values ​​of the movement amount data corresponding to the class value may be set differently (or may be the same) for each synchronization network calculation time period (Z_nw) and each feature point 123 (portion). 7, the synchronization determination calculation unit 700 converts the time series of movement amount data for each feature point 123 (part) into a time series of bin values ​​for each feature point 123 (part), in accordance with the number of bins determined in step 702 and the range of movement amount data values ​​corresponding to each bin value. These bin values ​​may be considered to be discretized (normalized) random variables. The synchronization determination calculation unit 700 stores the time series of discretized (normalized) random variables for each feature point 123 (part) obtained by the conversion as a time series of random variable movement amount data 1002 in a medium or the like external to the system 101 that can be used by the system 101. In the following, a discretized (normalized) random variable for I, which is a feature point 123 (site), may be denoted as i_t(i_1, i_2...i_n-m...i_n-1,i_n,i_n+1...), and a discretized (normalized) random variable for J, which is a feature point 123 (site), may be denoted as j_t(j_1, j_2...j_n-m...j_n-1,j_n,j_n+1...). Also, as shown in FIG. 10 , a set of m+1 random variables from i_n-m to i_n may be denoted as i_n(m), and a set of m+1 random variables from j,nm to j_n may be denoted as j_n(m). The value of m may be determined appropriately in the system 101.

[0085] 7, the synchronization determination calculation unit 700 obtains frequency information used to calculate the joint probability and conditional probability used to calculate the inter-feature point (part) transfer entropy data 1004(E). As shown in Fig. 10, to calculate the transfer entropy E(J→I) from feature point (part) J to feature point (part) I, the values ​​of the synchronization probability P(i_n+1,i_n(m),j_n(m)), the conditional probability P(i_n+1|i_n(m),j_n(m)), and the conditional probability P(i_n+1|i_n(m)) are required. 10, calculation of transfer entropy E(I→J) from feature point (part) I to feature point (part) J requires the values ​​of synchronization probability P(j_n+1, j_n(m), i_n(m)), conditional probability P(j_n+1|j_n(m), i_n(m)), and conditional probability P(j_n+1|j_n(m)). Therefore, in step 704, the synchronization determination calculation unit 700 counts the frequencies that can be used to calculate the above-mentioned synchronization probability and conditional probability using, from the time series of the discretized (normalized) random variables (class values) for each feature point 123 (part) obtained in step 703, the time series within a range included in one synchronization network calculation time zone (Z_nw). The synchronization determination calculation unit 700 stores the collected frequency information as frequency distribution data 1003 (histogram data) in a medium or the like outside the system 101 that can be used by the system 101 .

[0086] In step 705 of FIG. 7, the synchronization determination calculation unit 700 calculates the transfer entropy E(J→I) from feature point (part) J to feature point (part) I and the transfer entropy E(I→J) from feature point (part) I to feature point (part) J for each synchronization network calculation time period (Z_nw) and for each combination of feature points 123 (parts) (the combination is expressed as feature point I and feature point J). The synchronization determination calculation unit 700 uses frequency distribution data 1003 for the processing of step 705. The synchronization determination calculation unit 700 calculates the transfer entropy E(J→I) from feature point (part) J to feature point (part) I and the transfer entropy E(I→J) from feature point (part) I to feature point (part) J according to the following equations. E(J→I):=ΣP(i_n+1,i_n(m),j_n(m))*log(P(i_n+1|i_n(m),j_n(m)) / P(i_n+1|i_n(m))) E(I→J):=ΣP(j_n+1,j_n(m),i_n(m))*log(P(j_n+1|j_n(m),i_n(m)) / P(j_n+1|j_n(m)))

[0087] 5-3. Processing of the Synchronization Network Calculation Unit (Figures 8, 10, 11, 12, and 4) Figure 8 shows a flowchart of the processing executed by the synchronization network calculation unit 800. Below, the processing will be described in the order shown in Figure 8, with appropriate reference to the functional configuration shown in Figure 4, the handled data group 1000 shown in Figure 10, the handled data group 1100 shown in Figure 11, and the handled data group 1200 shown in Figure 12. Note that each of the processing steps performed by the synchronization network calculation unit 800 in the flowchart of Figure 8 may be considered to form a "synchronization network calculation step." Because the functions described below are realized, it is possible to calculate a time series of inter-feature point (part) synchronization network data 1005(F) indicating whether or not there is a certain degree of interlocking between feature points 123 (parts) based on a time series of indices indicating the degree of influence between feature points 123 (parts) (for example, a time series of inter-feature point (part) transfer entropy data 1004(E)). When the time series of inter-feature point (part) synchronization network data 1005(F) is obtained, it becomes easier to analyze the relationships between feature points 123 (parts) in analyzing the movements of the subject 102, and therefore movement can be analyzed in detail. Specifically, the user of the system 101 can examine the time series of the inter-feature point (part) synchronization network data 1005(F) itself, and can also examine new indexes that can be calculated based on the time series of the inter-feature point (part) synchronization network data 1005(F) (for example, the time series of the prominence data 143 and the (time series of) collective consciousness index data 1905), and therefore, information that is in line with the analysis objectives of the user of the system 101 can be provided from the system 101.

[0088] In step 801 of Figure 8, the synchronization network calculation unit 800 acquires a parameter ε that indicates a synchronization network threshold. The parameter ε is used in steps 805 and 807. As described above, the value of the parameter ε determines the degree to which information indicating the presence or absence of a certain degree of inter-connectivity between feature points 123 (parts) in the inter-feature point (part) synchronization network data 1005(F) is likely to be positive (indicating the presence of a certain degree of inter-connectivity). Therefore, by adjusting the parameter ε, the user of the system 101 can obtain inter-feature point (part) synchronization network data 1005(F) that is suited to the purpose of analysis.

[0089] In step 802 of FIG. 8, the synchronization network calculation unit 800 selects one of the synchronization network calculation time periods (Z_nw). 11 and 19, the position (start and end) of each synchronization network calculation time period (Z_nw) on the time axis of the form video data 401 is determined based on a parameter T indicating the time width (synchronization network calculation width) of the synchronization network calculation time period (Z_nw) and a parameter S indicating the time difference (synchronization network shift width) between adjacent synchronization network calculation time periods (Z_nw) on the timeline. The synchronization network calculation unit 800 may acquire the parameters T and S stored in a recording medium or storage medium inside or outside the system 101, or the parameters T and S acquired by the synchronization determination calculation unit 700 in step 701 may be passed from the synchronization determination calculation unit 700 to the synchronization network calculation unit 800.

[0090] 8, the synchronization network calculation unit 800 selects one of the combinations of feature points 123 (parts) of the subject 102 (which may be either the skeleton 121 or the tool 122). For convenience, the combination of feature points 123 (parts) selected in the most recently executed step 803 will be referred to as the combination of feature point (part) I and feature point (part) J below. In step 804 of FIG. 8 , the synchronization network calculation unit 800 acquires an index indicating the degree of influence from feature point (part) J to feature point (part) I and an index indicating the degree of influence from feature point (part) I to feature point (part) J for the combination of feature point (part) I and feature point (part) J selected in the most recently executed step 803 during the synchronization network calculation time period (Z_nw) selected in the most recently executed step 802. For example, this index may be the inter-feature point (part) transfer entropy. In that case, the index indicating the degree of influence from feature point (part) J to feature point (part) I is the transfer entropy E(J→I) from feature point (part) J to feature point (part) I, and the index indicating the degree of influence from feature point (part) I to feature point (part) J is the transfer entropy E(I→J) from feature point (part) I to feature point (part) J. In step 804, the synchronization network calculation unit 800 acquires the above index (e.g., transfer entropy data) from the time series of feature point synchronization data 406 (time series of inter-feature point (region) transfer entropy data 1004(E)). In the following, the case where the index acquired in step 804 is transfer entropy E(J→I) and transfer entropy E(I→J) is shown. (Other types of indexes may also be used.)

[0091] 8, the synchronization network calculation unit 800 determines whether the difference (E(J→I) - E(I→J)) obtained by subtracting the transfer entropy E(I→J) from feature point (site) I to feature point (site) J from the transfer entropy E(J→I) obtained in the most recently executed step 804 is equal to or greater than the value of parameter ε. If the determination result in step 805 is affirmative, control transitions to step 806. If the determination result in step 805 is negative, control transitions to step 807. Upon determining that the value of the difference (E(J→I) - E(I→J)) is equal to or greater than the value of the parameter ε, in step 806 of FIG. 8 , the synchronization network calculation unit 800 sets F(J→I), which indicates whether or not there is an influence from feature point (part) J to feature point (part) I (the presence of a certain degree of interlocking), in the inter-feature point (part) synchronization network data 1005 (F) for the synchronization network calculation time period (Z_nw) selected in the most recently executed step 802, to indicate information indicating that "a certain degree of interlocking exists." For example, the synchronization network calculation unit 800 sets the value of F(J→I) to 1. Then, the synchronization network calculation unit 800 sets F(I→J), which indicates whether or not there is an influence from feature point (part) I to feature point (part) J (the presence of a certain degree of interlocking), to indicate information indicating that "a certain degree of interlocking does not exist." For example, the synchronization network calculation unit 800 sets the value of F(I→J) to 0. After step 806, control transitions to step 810.

[0092] In step 807 of Fig. 8, the synchronization network calculation unit 800 determines whether the difference (E(I→J) - E(J→I)) obtained by subtracting the transfer entropy E(J→I) from feature point (part) J to feature point (part) I from the transfer entropy E(I→J) from feature point (part) I, which was acquired in the most recently executed step 804, is equal to or greater than the value of parameter ε. If the determination result in step 807 is affirmative, control transitions to step 808. If the determination result in step 807 is negative, control transitions to step 809. Upon determining that the value of the difference (E(I→J) - E(J→I)) is equal to or greater than the value of the parameter ε, in step 808 of FIG. 8 , the synchronization network calculation unit 800 sets F(I→J), which indicates whether or not there is an influence (existence of a certain interlocking) from feature point (part) I to feature point (part) J, in the inter-feature point (part) synchronization network data 1005 (F) for the synchronization network calculation time period (Z_nw) selected in the most recently executed step 802, to indicate information indicating that "a certain interlocking exists." For example, the synchronization network calculation unit 800 sets the value of F(I→J) to 1. Then, the synchronization network calculation unit 800 sets F(J→I), which indicates whether or not there is an influence (existence of a certain interlocking) from feature point (part) J to feature point (part) I, to indicate information indicating that "a certain interlocking does not exist." For example, the synchronization network calculation unit 800 sets the value of F(J→I) to 0. After step 808, control transitions to step 810.

[0093] Upon determining that the absolute value of the difference between E(J→I) and E(I→J) is less than the parameter ε, in step 809 of FIG. 8, the synchronization network calculation unit 800 sets, in the inter-feature-point (part) synchronization network data 1005 (F) for the synchronization network calculation time period (Z_nw) selected in the most recently executed step 802, information indicating "the absence of a certain degree of interlocking" for both F(J→I), which indicates whether or not feature point (part) J has an influence on feature point (part) I (the presence of a certain degree of interlocking), and F(I→J), which indicates whether or not feature point (part) I has an influence on feature point (part) J (the presence of a certain degree of interlocking). For example, the synchronization network calculation unit 800 sets the values ​​of both F(J→I) and F(I→J) to 0. After step 809, control transitions to step 810.

[0094] 8, the synchronization network calculation unit 800 determines whether all of the combinations of feature points 123 (regions) have been selected in step 803 for the synchronization network calculation time period (Z_nw) selected in the most recently executed step 802. If the determination result in step 810 is positive, control transitions to step 811. If the determination result in step 810 is negative, control returns to step 803, and one of the unselected combinations of feature points 123 (regions) is selected. In step 811 of Fig. 8, the synchronization network calculation unit 800 determines whether all of the synchronization network calculation time periods (Z_nw) for which inter-feature point (part) synchronization network data 1005(F) are to be created have been selected in step 802. If the determination result in step 811 is positive, the processing shown in Fig. 8 ends. If the determination result in step 811 is negative, control returns to step 802, and one of the synchronization network calculation time periods (Z_nw) that has not been selected is selected.

[0095] Inter-feature point (region) synchronization network data 1005(F) for one synchronization network calculation time period (Z_nw), generated by the process shown in FIG. 8, may be interpreted as forming matrix data. Specifically, where I and J are natural numbers, F(J→I), which indicates whether or not feature point (region) J has an influence on feature point (region) I (the presence of a certain degree of interlocking), may be considered to be the element in the I-th row and J-th column of matrix F. Similarly, F(I→J), which indicates whether or not feature point (region) I has an influence on feature point (region) J (the presence of a certain degree of interlocking), may be considered to be the element in the J-th row and I-th column of matrix F. Hereinafter, when inter-feature point (region) synchronization network data 1005(F) is treated as matrix data, it may be referred to as inter-feature point (region) synchronization network matrix data or matrix F. The form of matrix F is shown in the lower right corner of FIG. 10. Furthermore, the synchronization network data 1005(F) between feature points (parts) for one synchronization network calculation time period (Z_nw), generated by the processing shown in Fig. 8, may be interpreted as forming a graph. Specifically, each feature point (part) may be interpreted as a node of the graph, and when F(J→I), which indicates whether or not feature point (part) J has an influence on feature point (part) I (the presence of a certain degree of interlocking), indicates that "a certain degree of interlocking exists," it may be interpreted as a directed edge being set from the node of feature point (part) J to the node of feature point (part) I. The aspect of the synchronization network data 1005(F) between feature points (parts) when interpreted as a graph is shown at the top of Fig. 12.

[0096] 5-4. Processing of the leading role calculation unit (Figures 9, 11, 12, 13, and 4) Fig. 9 shows a flowchart of the processing executed by the leading role degree calculation unit 900. Below, the processing will be explained in the order shown in Fig. 9, with appropriate reference to the functional configuration shown in Fig. 4, the handled data group 1100 shown in Fig. 11, the handled data group 1200 shown in Fig. 12, and the relationship between the transition matrix and leading role degree (PageRank) shown in Fig. 13. Note that each of the processing steps performed by the leading role degree calculation unit 900 in the flowchart of Fig. 9 may be understood to form a "leading role degree calculation step" (or a "PageRank calculation step"). By realizing the functions described below, it is possible to calculate the time series of the dominance data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)), which is a type of index derived by utilizing the time series of the inter-feature point (part) synchronization network data 1005 (F). While the inter-feature point (part) synchronization network data 1005 (F) is information that primarily provides one-to-one relationships between different feature points 123 (parts), the dominance data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) can also be said to be information that provides relationships between a certain feature point 123 (part) and the other feature points 123 (parts) as a whole (a kind of one-to-many relationship). That is, the group of indices calculated by the leading role degree calculation unit 900 enables the user of the system 101 to analyze the behavior of the subject 102 from a new perspective.

[0097] In step 901 of Fig. 9, the leading role degree calculation unit 900 acquires a parameter M indicating a leading role degree synchronous network set coefficient. As shown in Fig. 11, according to the value of the parameter M, the leading role degree calculation unit 900 calculates leading role degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for each one (one set) of feature points (parts) in response to M pieces (M sets) of inter-feature point (part) synchronous network data 1005 (F). As the value of parameter M increases, the time width of the dominance degree (PageRank) calculation time period (Z_pr), which is the time period that serves as the unit for calculating the dominance degree (PageRank), also increases. Therefore, as the value of parameter M increases, even for information about actions of subject 104 that take a long time to propagate influence between feature points 123 (regions), the entirety of that information is more likely to be reflected in the dominance degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for one (one set) of feature points (regions). On the other hand, as the value of parameter M decreases, only for information about actions of subject 104 that take a short time to propagate influence between feature points 123 (regions), the entirety of that information is more likely to be reflected in the dominance degree data 143 (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) for one (one set) of feature points (regions). Therefore, by setting the value of the parameter M according to the purpose of analyzing the behavior of the subject 102, the user of the system 101 can obtain the dominance data 143 (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)) for each desired feature point (part). Also, in step 901, the leading role degree calculation unit 900 acquires a parameter C indicating the time difference (leading role degree calculation time period shift width) between adjacent leading role degree (PageRank) calculation time periods (Z_pr) on the time series.

[0098] In step 902 of FIG. 9, the leading role degree calculation unit 900 selects one of the leading role degree (PageRank) calculation time periods (Z_pr). As shown in FIG. 11, each of the dominance degree (PageRank) calculation time periods (Z_pr) is determined by parameters T, S, M, and C. Specifically, one dominance degree (PageRank) calculation time period (Z_pr) includes M synchronous network calculation time periods (Z_nw) (each having a time width of T). The time difference between adjacent synchronous network calculation time periods (Z_nw) on the time line is S. Therefore, the time width of one dominance degree (PageRank) calculation time period (Z_pr) is T+(M-1)*S. The dominance degree (PageRank) calculation time periods (Z_pr) on the time line are shifted in time by C synchronous network calculation time periods (Z_nw) (arranged at a time interval indicated by parameter S). Therefore, the time difference between adjacent dominance degree (PageRank) calculation time periods (Z_pr) on the time line is C*S. The leading role degree calculation unit 900 may acquire the parameters T and S by itself, or may receive them from the synchronization determination calculation unit 700 or the synchronization network calculation unit 800 . In step 902, one of the time periods (Z_pr) for calculating the degree of prominence (PageRank) determined as described above is selected.

[0099] In step 903 of FIG. 9, the leading role degree calculation unit 900 obtains one piece of transition matrix data 1201(A) for the leading role degree (PageRank) calculation time period (Z_pr) selected in the most recently executed step 902, based on the M inter-feature point (part) synchronization network data 1005(F) belonging to the leading role degree (PageRank) calculation time period (Z_pr). Specifically, the leading role degree calculation unit 900 may treat the M inter-feature point (part) synchronization network data 1005(F) as the above-mentioned matrix data (matrix F), perform a logical sum operation on each of the M elements of matrix F (permutations of two feature points 123 (parts)), and may use the result of the logical sum operation as one transition matrix A, and may use the information of the transition matrix A as transition matrix data 1201(A). In other words, the leading role degree calculation unit 900 may perform a logical sum operation on the M inter-feature point (part) synchronization network data 1005(F) for information indicating whether or not there is a certain interrelationship from one feature point 123 (part) to another feature point 123 (part) (in other words, if any one of the M pieces has information indicating that "there is a certain interrelationship," the result of the logical sum operation also indicates that "there is a certain interrelationship"), and may reflect the result of the logical sum operation in the transition matrix data 1201(A). The upper part of Fig. 12 shows an example where the feature points (parts) are I, J, K, and L, and M=3. In the upper part of Fig. 12, the inter-feature point (part) synchronization network data 1005(F) is displayed as a graph. In the upper part of Fig. 12, a logical OR operation is performed on M (three in the example of Fig. 12) inter-feature point (part) synchronization network data 1005(F) to obtain transition matrix data 1201(A). After step 903, steps 904, 905, and 906 are used as processes for using the transition matrix data 1201(A) without transposing it. On the other hand, after step 903, steps 907, 908, 909, and 910 are used as processes for using the transition matrix data 1201(A) after transposing it. These steps will be explained in order below.

[0100] In step 904 of Fig. 9, the leading role degree calculation unit 900 adjusts the transition matrix A indicated by the transition matrix data 1201(A) by multiplying each element included in the same column by the same magnification so that the sum of the values ​​of the elements included in the column becomes 1. The leading role degree calculation unit 900 obtains an adjusted transition matrix A' as the adjusted matrix. Fig. 12 shows an example of an adjusted transition matrix 1202(A') for the example of the transition matrix data 1201(A). In step 905 of FIG. 9 , the leading role degree calculation unit 900 calculates an eigenvector x_out corresponding to an eigenvalue whose value is 1 for the adjusted transition matrix A'. In other words, the leading role degree calculation unit 900 calculates an eigenvector x_out that satisfies x_out = A'·x_out (where "·" indicates an inner product operation between a matrix and a vector). Note that a known method for obtaining an eigenvector may be used, and for example, a method belonging to a direct method or an iterative method may be used. Note that even if the eigenvector x_out cannot be calculated strictly because the adjusted transition matrix A' does not include 1 as an eigenvalue, the leading role degree calculation unit 900 may calculate an approximate eigenvector x_out by an iterative method or the like. 9, the leading role degree calculation unit 900 adjusts the eigenvector x_out calculated in step 905 by multiplying each element by the same magnification so that the sum of the squares of the elements becomes 1 (since eigenvectors generally have arbitrariness regarding the vector length). The leading role degree calculation unit 900 obtains, as the adjusted vector, an output PageRank vector p_out for the leading role degree (PageRank) calculation time period (Z_pr) selected in the most recently executed step 902. The leading role degree calculation unit 900 stores information about the output PageRank vector p_out as output PageRank data 1203 (OUT PageRank, OUT PR) in the system 101 or in a medium or the like external to the system 101 that can be used by the system 101. Each element of the output page rank vector p_out indicates the degree to which the feature point 123 (part) corresponding to that element influences the other feature points 123 (parts) as a whole.

[0101] In step 907 of Fig. 9, the leading role degree calculation unit 900 transposes the rows and columns of the transition matrix A indicated by the transition matrix data 1201(A) to obtain a transposed transition matrix A_t. Fig. 12 shows an example of a transposed transition matrix A_t (transposed transition matrix data 1211(A_t)) for the example of the transition matrix A indicated by the transition matrix data 1201(A). 9, the processes performed by the leading role calculation unit 900 are the same as those performed in steps 904, 905, and 906. However, the matrices handled in steps 908, 909, and 910 are the transposed transition matrix A_t (transposed transition matrix data 1211(A_t)) and the adjusted transposed transition matrix 1212(A_t'). Furthermore, the vectors handled in steps 908, 909, and 910 are the eigenvector x_in and the input page rank vector p_in (input page rank data 1213(IN PageRank, IN PR)). Each element of the input page rank vector p_in indicates the degree to which the feature point 123 (part) corresponding to that element is influenced by all of the other feature points 123 (parts).

[0102] After the processing of step 906 and step 910 is completed, in step 911 of FIG. 9, the dominance degree calculation unit 900 determines whether all of the dominance degree (PageRank) calculation time periods (Z_pr) for calculating the dominance degree data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)) have been selected in step 902. If the determination result in step 911 is positive, the processing of FIG. 9 ends. If the determination result in step 911 is negative, control is returned to 902, and one of the unselected dominance degree (PageRank) calculation time periods (Z_pr) is newly selected.

[0103] FIG. 13 shows a graph corresponding to the transition matrix A indicated by the transition matrix data 1201(A) and a relationship 1300 between the degree of prominence (output page rank (OUT PageRank, OUT PR) and input page rank (IN PageRank, IN PR)) for each feature point 123 (part). In Fig. 13, each of the feature points 123 (regions) is represented by a circular node. Also, in Fig. 13, directed edges on the graph are shown in response to the fact that the transition matrix A contains information indicating that there is a certain degree of linkage from one feature point 123 (region) to another feature point 123 (region). Many directed edges are output from feature points 123 (parts) marked with a "★" in Figure 13, so the output page rank (OUT PageRank, OUT PR) values ​​of such feature points 123 (parts) tend to be large. On the other hand, many directed edges are input to feature points 123 (parts) marked with a "◆" in Figure 13, so the input page rank (IN PageRank, IN PR) values ​​of such feature points 123 (parts) tend to be large. Among the feature points 123 (regions), feature points 123 (regions) with relatively large values ​​of output page rank (OUT PageRank, OUT PR) can be estimated to be highly likely to be the "starting point of the action," and feature points 123 (regions) with relatively large values ​​of input page rank (IN PageRank, IN PR) can be estimated to be highly likely to be the "ending point of the action." It can also be inferred that the order of magnitude of the output page rank (OUT PageRank, OUT PR) values ​​or the order of magnitude of the input page rank (IN PageRank, IN PR) values ​​for a group of feature points 123 (regions) located close to each other may indicate the "order in which the action is conveyed." Furthermore, feature points 123 (regions) with relatively high values ​​of output page rank (OUT PageRank, OUT PR) or input page rank (IN PageRank, IN PR) can be inferred to be highly likely to play an "important role in the action." For example, if the subject 102 is a baseball player holding a bat and form video data 401 obtained as a result of videotaping the baseball player swinging the bat is to be analyzed, as shown in Figure 13 (or Figure 14 described below), if the input page rank (IN PageRank, IN PR) value of the feature point 123 (part) indicating the tip of the bat is relatively large, it can be estimated that the tip of the bat is likely to be the "end point of the movement." Furthermore, if the order of magnitude of the output page rank (OUT PageRank, OUT PR) values ​​in form video data 401 obtained by videotaping a baseball player throwing a ball is right shoulder, right elbow, and right wrist (of the feature points 123 (parts) representing each), it can be inferred that the "order in which the movement is conveyed" in the action of throwing a ball is likely to be right shoulder, right elbow, and right wrist. Furthermore, if the group of feature points 123 (parts) in the lower body have high PageRank values, it can be inferred that the lower body as a whole is likely to play an "important role in the movement" in the action of throwing a ball. The term "PageRank" is used in the embodiments of the present disclosure because it appears similar to the "PageRank" known as an indicator of the importance assigned to each website on the Internet. For example, in FIG. 13, if each feature point 123 (site) is likened to a website and each directed edge is likened to a hyperlink between websites, they appear similar. However, the evaluation of the importance of a general website and the evaluation of the interconnectivity between feature points 123 (sites) of a subject 102 in the embodiments of the present disclosure are significantly different technical fields.

[0104] 5.5. Display output of synchronization network or leading role (PageRank), etc. (Figure 14, Figure 4, Figure 5) Fig. 14 shows an example of a display or output under the control of the system 101. Fig. 14 shows, for example, a screen example 1400 that is displayed on a display included in the display or output device 307 that the system 101 has. The screen example 1400 includes a video playback display window 1411, a synchronized network time-series playback display window 1412, and a leading role degree graph display window for cumulative value display 1413. Therefore, in controlling the display of the screen example 1400 on the display, the video display output control unit 1401 (which performs control related to the video playback display window 1411), the synchronized network display output control unit 1402 (which performs control related to the synchronized network time-series playback display window 1412), and the leading role degree display output control unit 1403 (which performs control related to the leading role degree graph display window 1413 for cumulative value display) may operate in cooperation with one another. Although not shown in the screen example 1400, the screen example 1400 may further include a window (feature point movement amount display window) that displays, in conjunction with video playback, time-series changes in movement amount data (the magnitude of the velocity vector, or the velocity vector itself) for each feature point 123 (site). In this case, the movement amount display output control unit 1404 may also operate in conjunction with the video display output control unit 1401, etc. The screen example 1400 also includes a user interface for setting various parameters by the user of the system 101. When an input for parameter setting is made using such a user interface, the parameter setting unit 501 operates. The example screen 1400 will now be described.

[0105] The screen example 1400 may include a video playback display window 1411. The video playback display window 1411 displays the frame images included in the form video data 401 in chronological order. When the form movie data 401 is played back and displayed, the display may be such that the skeleton 121 and equipment 122 of the subject 102 are easily visible, and the display may be such that the feature points 123 (body parts) included in the skeleton 121 and equipment 122 are easily visible. To achieve such a display, not only the movie data 141 (form movie data 401) but also skeleton model data 404A, equipment model data 404B, skeleton feature point coordinate data 405A, and equipment feature point coordinate data 405B may be used. A user interface for video playback, such as a slide bar 1421 for video playback time and a play icon, may be provided near (directly below in FIG. 14) the video playback display window 1411. Such a user interface for video playback allows the user of the system 101 to view frame images at desired timings in the form video data 401 and the time-series changes in the frame images.

[0106] The screen example 1400 may include a synchronous network time series playback display window 1412. The synchronous network time series playback display window 1412 displays the synchronous network data 1005(F) between feature points (sites) in chronological order. 14, the synchronization network data 1005(F) between feature points (regions) is displayed in a graph format. In the graph, each feature point 123 (region) is shown as a circle node, and the existence of a certain degree of interrelationship between the feature points 123 (regions) is shown by a directed edge between the nodes. The display of the synchronous network data 1005(F) between feature points (parts) in the synchronous network time series playback display window 1412 may be linked to the display of the frame images included in the form video data 401 in the video playback display window 1411. For example, a graph of the synchronous network data 1005(F) between feature points (parts) for any of the synchronous network calculation time periods (Z_nw) that include the frame images displayed in the video playback display window 1411 may be displayed in the synchronous network time series playback display window 1412. By displaying the form in this manner, the user of the system 101 can grasp the situation at a specific timing in the form video data 401 in detail.

[0107] Although not shown in FIG. 14, if a feature point movement amount display window also exists, the feature point movement amount display window may display movement amount data (the magnitude of the velocity vector, or the velocity vector itself) for each feature point 123 (body part) for the time corresponding to the frame image displayed in the video playback display window 1411, based on the skeleton feature point movement amount data 1001A and the equipment feature point movement amount data 1001B. By displaying the linked information in this way, the user of the system 101 can grasp the situation at a specific timing in the form video data 401 in more detail.

[0108] The screen example 1400 may include a leading role degree graph display window 1413 for displaying cumulative values. The leading role degree graph display window 1413 for displaying cumulative values ​​displays information related to the leading role degree data 143 (output page rank data 1203 (OUT PageRank, OUT PR), input page rank data 1213 (IN PageRank, IN PR)) for each feature point 123 (part). In the example of Fig. 14, the leading role degree graph display window 1413 for displaying cumulative values ​​displays information related to the output page rank data 1203 (OUT PageRank, OUT PR) and information related to the input page rank data 1213 (IN PageRank, IN PR) for each feature point 123 (part) near the display of the feature point (part) name 1423 (directly above in Fig. 14). Specifically, the information relating to the output PageRank data 1203 (OUT PageRank, OUT PR) for each feature point 123 (part) may be displayed as the cumulative value (sum) of the output PageRank data 1203 (OUT PageRank, OUT PR) for each feature point 123 (part) in the set of time periods (Z_pr) for calculating the degree of prominence (PageRank). Similarly, the information relating to the input PageRank data 1213 (IN PageRank, IN PR) for each feature point 123 (part) may be displayed as the cumulative value (sum) of the input PageRank data 1213 (IN PageRank, IN PR) for each feature point 123 (part) in the set of time periods (Z_pr) for calculating the degree of prominence (PageRank). The certain period referred to here may be the entire period in the form video data 401, or it may be a period determined by a certain number of leading role (PageRank) calculation time periods (Z_pr) before and after the leading role (PageRank) calculation time period (Z_pr) that includes the frame image being played. By displaying the cumulative value of the output page rank data 1203 (OUT PageRank, OUT PR) for each feature point 123 (part), and the cumulative value of the input page rank data 1213 (IN PageRank, IN PR) for each feature point 123 (part), the user of the system 101 can grasp the feature points 123 (parts) that have a large impact on the behavior of the subject 102 over a certain period of time.

[0109] The example screen 1400 may include a leading role degree graph display window for displaying time-series changes, instead of the leading role degree graph display window 1413 for displaying cumulative values, or in addition to the leading role degree graph display window 1413 for displaying cumulative values. The leading role degree graph display window for displaying time-series changes may display the value of the output page rank data 1203 (OUT PageRank, OUT PR) for each feature point 123 (part) and the value of the input page rank data 1213 (IN PageRank, IN PR) for each feature point 123 (part), for any of the leading role degree (PageRank) calculation time periods (Z_pr) that include the frame image being played back. By displaying the time series changes in the output page rank data 1203 (OUT PageRank, OUT PR) for each feature point 123 (body part) and the time series changes in the input page rank data 1213 (IN PageRank, IN PR) for each feature point 123 (body part) in conjunction with video playback, users of the system 101 can grasp the feature points 123 (body parts) that have a large impact on the behavior of the subject 102 at specific times in the video data 141 (form video data 401).

[0110] The example screen 1400 may have a user interface for setting various parameters. In Fig. 14, a slide bar 511 for the synchronization network threshold (ε), a slide bar 512 for the dominance synchronization network set coefficient (M), a slide bar 514 for the dominance calculation time period shift width (C), and various parameter setting fields 519 are shown. The slide bar 511 for the synchronization network threshold (ε) is used by the user of the system 101 to set the parameter ε. As shown in Fig. 14, a user interface for inputting the value of the parameter ε by text may be provided near the slide bar 511 for the synchronization network threshold (ε) (to the right in Fig. 14). The slider 512 for the leading role synchronization network set coefficient (M) is used by the user of the system 101 to set the parameter M. As shown in Fig. 14, a user interface for inputting the value of the parameter M by text may be provided near the slider 512 for the leading role synchronization network set coefficient (M) (to the right in Fig. 14). The slider 514 for the leading role degree calculation time period shift width (C) is used by the user of the system 101 to set the parameter C. As shown in Fig. 14, a user interface for inputting the value of the parameter C as text may be provided near the slider 514 for the leading role degree calculation time period shift width (C) (to the right in Fig. 14). The various parameter setting field 519 may have a user interface for inputting the parameter T as text, and a user interface for inputting the parameter S as text. Slide bars for setting the parameter T and the parameter S may also be provided. The various parameter setting field 519 may have an "Other parameter setting" icon for calling a dedicated pop-up window used when setting all the parameters that can be set in the system 101. When the "Other parameter setting" icon is clicked, a dedicated pop-up window may appear on the screen. The dedicated pop-up window may provide a user interface that allows the parameters that can be set in the system 101 (the parameters listed in FIG. 5) to be set. The screen example 1400 provides a user interface for setting various parameters, and therefore a user of the system 101 who views the content displayed in the various windows provided on the screen example 1400 can change the settings of the various parameters. Then, after the system 101 executes processing based on the changed parameter values, the execution results may be reflected again in the various windows provided on the screen example 1400. Such an operation improves the convenience for users of the system 101. Furthermore, if the setting of any of the parameters is changed, the functional units shown in Figure 4 that are affected by the changed parameters and that directly or indirectly use the processing results of those functional units may redo their processing.

[0111] The example screen 1400 may include a "CSV data output" icon 1424. When the "CSV data output" icon 1424 is clicked, the results extracted, calculated, and computed by each functional unit of the system 101 are output, for example, as a CSV file. (Note that a file format other than a CSV file may also be used.) When outputting the results, it is possible to specify that all results be output, or that only specific types of results be output. For example, a dedicated window that appears on the screen when the "CSV data output" icon 1424 is clicked may include a user interface that allows the user to set the types of results to be output. In this way, the user of the system 101 can not only view the processing results of the system 101, but also obtain them as file data.

[0112] 5-6. Processing of the leading role evaluation section (Fig. 15, Fig. 16, Fig. 17, Fig. 4) Fig. 15 shows a flowchart of the processing executed by the leading role degree evaluation unit 1500. Below, the processing will be described in the order shown in Fig. 15, with appropriate reference to the functional configuration shown in Fig. 4, the handled data group 1600 shown in Fig. 16, and the example screen 1700 relating to the individuality of the subject's movement shown in Fig. 17. It should be noted that each of the processing steps executed by the leading role degree evaluation unit 1500 in the flowchart of Fig. 15 may be understood to form a "leading role degree evaluation step." The functions described below are realized, so that it is possible to calculate information indicating the individuality of the behavior of each subject 102 relative to the standard behavior of a group of subjects 102 based on the time series of the dominance data 143 for each feature point (part) (output PageRank data 1203 (OUT PageRank, OUT PR), input PageRank data 1213 (IN PageRank, IN PR)). Therefore, when analyzing the behavior of the subject 102, the user of the system 101 can perform a more detailed analysis while comparing it with the standard behavior of the group of subjects 102.

[0113] Of the processes shown in Fig. 15, steps 1501 to 1505 indicate a series of processes for calculating output PageRank average value data 1603 and input PageRank average value data 1613, which indicate standard behavior in a group of subjects. Due to space limitations, Fig. 16 shows a group of data involved in calculating output PageRank average value data 1603, but does not show a group of data involved in calculating input PageRank average value data 1613. However, the group of data involved in calculating input PageRank average value data 1613 is similar if the word "output" in Fig. 16 is replaced with the word "input." In addition, the following explanation of steps 1501 to 1506 only refers to the process for calculating the output page rank average data 1603 (and the process for displaying or outputting the output page rank average data 1603). However, (except for the fact that the data handled is different) the process for the input page rank average data 1613 is also the same as the process shown in FIG.

[0114] In step 1501 of FIG. 15, the leading role degree evaluation unit 1500 selects one subject 102 from among the subjects 102 included in the set of subjects 102 whose standard actions are to be calculated. 15, the leading role degree evaluation unit 1500 acquires output page rank data 1203 (OUT PageRank, OUT PR) corresponding to each leading role degree (PageRank) calculation time period (Z_pr) included in a certain period for the subject 102 selected in the most recently executed step 1501. The leading role degree evaluation unit 1500 calculates an accumulated value (sum) of the output page rank (OUT PageRank, OUT PR) for each feature point 123 (part) for all of the acquired output page rank data 1203 (OUT PageRank, OUT PR). The leading role degree evaluation unit 1500 stores the calculated accumulated value (sum) of the output page rank (OUT PageRank, OUT PR) for each feature point 123 (part) as output page rank accumulated value data 1601 in the system 101 or in a medium or the like external to the system 101 that can be used by the system 101. The "certain period" mentioned above may be the entire period of the form video data 401, or may be a period determined by a predetermined number of time periods (Z_pr) for calculating the degree of leading role (PageRank). In step 1503 of FIG. 15, the leading role degree evaluation unit 1500 calculates the sum of the cumulative values ​​(sums) of the output page rank (OUT PageRank, OUT PR) for each feature point 123 (part) calculated in step 1502 for the subject 102 selected in the most recently executed step 1501 by adding up the cumulative values ​​(sums) for each feature point 123 (part) for all feature points 123 (parts). Then, the leading role degree evaluation unit 1500 multiplies each of the cumulative values ​​(sums) of the output page rank (OUT PageRank, OUT PR) for each feature point 123 (part) by the reciprocal of the calculated sum. The leading role degree evaluation unit 1500 sets the result of the multiplication as the output page rank normalized data for each feature point 123 (part). In other words, when the output page rank normalized data for all feature points 123 (parts) is added up, the total value becomes 1. The leading role degree evaluation unit 1500 stores the output page rank normalized data for each feature point 123 (part) as output page rank normalized data 1602 in the system 101 or in a medium or the like outside the system 101 that can be used by the system 101 . 15, the leading role degree evaluation unit 1500 determines whether all of the subjects 102 included in the set of subjects 102 for which the standard action is to be calculated have been selected in step 1501. If the determination result in step 1504 is affirmative, control transitions to step 1505. If the determination result in step 1505 is negative, control returns to step 1501, and one of the unselected subjects 102 is newly selected.

[0115] 15, the leading role degree evaluation unit 1500 acquires output PageRank normalized data 1602 for all of the subjects 102 included in the set of subjects 102 for which standard actions are to be calculated. The leading role degree evaluation unit 1500 calculates the average value of the output PageRank normalized data of each of the subjects 102 for each feature point 123 (body part), and sets this as output PageRank average value data. The leading role degree evaluation unit 1500 stores the output PageRank average value data for each feature point 123 (body part) as output PageRank average value data 1603 in the system 101 or in a medium or the like external to the system 101 that can be used by the system 101. The above is a series of processes for calculating the output PageRank average value data 1603, which indicates the standard behavior of a group of subjects. There is a deviation for each feature point 123 (part) between the output PageRank average value data 1603 and the output PageRank normalized data 1602 of each subject 102. This deviation may be considered to indicate the individuality of each subject 102.

[0116] In step 1506 of FIG. 15, the leading role degree display output control unit 1403 controls to display or output in a manner that allows comparison between the value in the output page rank average data 1603 and the value in the output page rank normalized data 1602 for the subject 102 whose behavior individuality is to be confirmed, for each feature point 123 (part). 17 shows an example screen 1700 displayed on a display included in the display or output device 307 as an example of the display or output in step 1506. Note that although the example screen 1700 shows the output page rank (OUT PageRank, OUT PR), the input page rank (IN PageRank, IN PR) may also be displayed or output in a similar manner.

[0117] The example screen 1700 includes a leading role degree graph display window 1713 for comparison with the average value. The leading role degree graph display window 1713 for comparison with the average value may be included in the example screen 1400 of FIG. The leading role degree graph display window 1713 for comparison with the average value displays, adjacent to (directly above in FIG. 17) each of the feature point (part) names 1423, a bar graph (hatched bar graph in FIG. 17) of the value of output page rank average value data 1603 for the feature point 123 (part) indicated by the feature point (part) name 1423, and a bar graph (open bar graph in FIG. 17) of the value of output page rank normalized data 1602 of the subject 102 for which the individuality of the action is to be confirmed. Note that the output page rank average value data 1603 and the output page rank normalized data 1602 may be displayed or output in a form other than a bar graph. As described above, for each feature point 123 (body part), the value of the output page rank average value data 1603 indicating the standard movement and the value of the output page rank normalized data 1602 of the subject 102 are displayed or output in a manner that allows comparison, so that the user of the system 101 can easily understand the individual characteristics of the movement of the subject 102.

[0118] 5-7. Processing of the Collective Awareness Index Calculation Unit (Figures 18, 19, 20, and 4) Figure 18 shows a flowchart of the processing executed by the collective consciousness index calculation unit 1800. The following will be explained in the order of the processing shown in Figure 18, with appropriate reference to the functional configuration shown in Figure 4, the handled data group 1900 shown in Figure 19, and the meaning of the i-clique collective consciousness index 2000 shown in Figure 20. It should be noted that each of the processing steps performed by the collective consciousness index calculation unit 1800 in the flowchart of Figure 18 may be understood to form a "collective consciousness index calculation step." The functions described below are realized, and it is possible to calculate group consciousness index data 1905 for multiple feature points 123 (regions) of the subject 102, or the time series of the group consciousness index data 1905, based on the time series of inter-feature point (region) synchronization network data 1005(F). The group consciousness index data 1905 may consist of group consciousness index data 1905-i for i cliques (cliques involving i feature points 123 (regions)), where i is the number of cliques. The group consciousness index data 1905-i for i cliques may be a pair of an index (k_i) indicating the activity level of the entire set of i cliques and an index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the i cliques. By referring to the index (k_i) indicating the activity level of the entire set of i cliques, a user of the system 101 can grasp the overall level of activity of the feature points 123 (regions) of the subject 102 and the overall level of activity of the relationships between the feature points 123 (regions). Furthermore, by referring to the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between the i cliques, a user of the system 101 can grasp the level of randomness (irrelevance) or cooperation level of the behavior (activity) between different feature points 123 (regions) of the subject 102, and the level of randomness (irrelevance) or cooperation level of the behavior (activity) between combinations of different feature points 123 (regions). In other words, a user of the system 101 can analyze the behavior of the subject 102 in detail. Furthermore, if the time series of group consciousness index data 1905 can be calculated, the user of system 101 can also grasp time series changes in the overall level of activity of the feature points 123 (regions) of subject 102, the overall level of activity of the relationships between the feature points 123 (regions), the degree of randomness (degree of unrelatedness) or degree of cooperation between the behaviors (activity) of different feature points 123 (regions) of subject 102, and the degree of randomness (degree of unrelatedness) or degree of cooperation between the behaviors (activity) of combinations of different feature points 123 (regions). In other words, the user of system 101 can analyze the behavior of subject 102 in more detail.

[0119] In step 1801 of FIG. 18, the collective consciousness index calculation unit 1800 acquires a parameter N indicating a synchronization network set coefficient for a collective consciousness index. As shown in FIG. 19, in accordance with the value of the parameter N, the collective consciousness index calculation unit 1800 calculates one (one set) of group consciousness index data 1905 corresponding to N (N sets) of synchronization network data 1005(F) between feature points (locations). The group consciousness index data 1905 consists of group consciousness index data 1905-i for i cliques, where i is the number of cliques. The group consciousness index data 1905-i for i cliques is a pair of an index (k_i) indicating the activity level of the entire group of i cliques and an index (1-H_i / H_i(max)) indicating the randomness or degree of cooperation between i cliques. As the value of parameter N increases, the duration of the collective consciousness index time period (Z_gc), which is the time period that serves as the unit for calculating the collective consciousness index, also increases. Therefore, as the value of parameter N increases, it becomes more likely that information about movements of subject 102 that take a long time to propagate influence between feature points 123 (regions) will be reflected in its entirety in one piece (one set) of collective consciousness index data 1905. On the other hand, as the value of parameter N decreases, it becomes more likely that information about movements of subject 102 that take a short time to propagate influence between feature points 123 (regions) will be reflected in its entirety in one piece (one set) of collective consciousness index data 1905. Therefore, a user of system 101 can obtain desired collective consciousness index data 1905 by setting the value of parameter N according to the purpose of analyzing the movements of subject 102. Also, in step 1801, the collective consciousness index calculation unit 1800 acquires a parameter D that indicates the time difference (shift width of collective consciousness index calculation time period) between adjacent collective consciousness index calculation time periods (Z_gc) on the time series.

[0120] In step 1802 of FIG. 18, the collective consciousness index calculation unit 1800 selects one of the collective consciousness index calculation time periods (Z_gc). As shown in FIG. 19, each collective consciousness index calculation time slot (Z_gc) is determined by parameters T, S, N, and D. Specifically, one collective consciousness index calculation time slot (Z_gc) includes N synchronization network calculation time slots (Z_nw) (each having a time span of T). The time difference between adjacent synchronization network calculation time slots (Z_nw) on the time line is S. Therefore, the time span of one collective consciousness index calculation time slot (Z_gc) is T+(N-1)*S. The time difference between adjacent synchronization network calculation time slots (Z_gc) on the time line is D synchronization network calculation time slots (Z_nw) (arranged at a time interval indicated by parameter S). Therefore, the time difference between adjacent synchronization network calculation time slots (Z_gc) on the time line is D*S. The collective consciousness index calculation unit 1800 may acquire the parameters T and S by itself, or may receive them from the synchronization determination calculation unit 700 or the synchronization network calculation unit 800. In step 1802, one of the collective consciousness index calculation time periods (Z_gc) determined as described above is selected.

[0121] 18, the collective consciousness index calculation unit 1800 selects one clique number i from the clique numbers i for which the collective consciousness index is to be calculated. For example, the clique numbers i for which the collective consciousness index is to be calculated may be 1, 2, and 3, or the clique numbers i for which the collective consciousness index is to be calculated may be 2 and 3.

[0122] In step 1804 of FIG. 18 , the collective consciousness index calculation unit 1800 calculates the probability (occurrence probability) p_i,j (occurrence probability data 1901-i of i clique) that each of the i cliques (hereinafter, the j-th clique of the i clique) will be activated, for the number i of cliques selected in the most recently executed step 1803, based on the synchronization network data 1005(F) between N feature points (locations) included in the collective consciousness index calculation time period (Z_gc) selected in the most recently executed step 1802. Here, a clique with a clique number of 1 may correspond to a feature point 123 (part). A clique may be considered to be associated with a node. A clique with a clique number of 2 may correspond to a (directed) connection, influence, or relationship from one feature point 123 (part) to another feature point 123 (part). In other words, a clique with a clique number of 2 may correspond to a permutation containing two feature points 123 (parts). A clique with a clique number of 2 may correspond to a (directed) connection, influence, or relationship between three distinct feature points 123 (parts). In other words, a clique with a clique number of 3 may correspond to a permutation containing three feature points 123 (parts). In step 1804, the collective consciousness index calculation unit 1800 calculates the probability p_i,j that each of the i cliques will be activated (probability of occurrence) using information indicating whether or not a certain level of interrelationship exists between the feature points 123 (regions), which is included in the N inter-feature point (region) synchronization network data 1005 (F). When i=2, for example, the collective consciousness index calculation unit 1800 may count the number of times that a certain degree of interrelationship is determined to exist from a first feature point 123 (part) to a second feature point 123 (part) in N inter-feature point (part) synchronization network data 1005 (F) corresponding to two cliques (2,j), and divide the counted number by N to calculate the activation probability (appearance probability) p_2,j of each of the two cliques (2,j). When i=3, for example, in N inter-feature point (part) synchronization network data 1005(F), for a first feature point 123 (part), a second feature point 123 (part), and a third feature point 123 (part) corresponding to three cliques (3,j), the collective consciousness index calculation unit 1800 may count the number of times that it is determined that a certain level of connectivity exists from the first feature point 123 (part) to the second feature point 123 (part), that a certain level of connectivity exists from the second feature point 123 (part) to the third feature point 123 (part), and that a certain level of connectivity exists from the first feature point (part) to the third feature point 123 (part), and divide the counted number by N to calculate the activation probability (appearance probability) p_3,j of each of the three cliques (3,j). When i=1, the collective consciousness index calculation unit 1800 may determine the activation probability (appearance probability) p_1,j of each clique (1,j) as (A) the probability that there is a certain degree of interrelationship between the feature point 123 (part) corresponding to each (1,j) of one clique and at least one other feature point 123 (part), or (B) the probability that separate information about the feature point 123 (part) (information other than position information and movement amount information obtained from the video data 141 (form video data 401); for example, temperature information for each feature point 123 (part) identified by a thermal camera) satisfies a predetermined condition.

[0123] In step 1805 of FIG. 18, the group consciousness index calculation unit 1800 calculates an index (k_i) indicating the activity level of the entire set of i cliques, which is one of the group consciousness indices 1905-i of i cliques corresponding to the number of cliques i selected in the most recently executed step 1803 for the group consciousness index calculation time period (Z_gc) selected in the most recently executed step 1802. The index (k_i) indicating the activity level of the entire set of i cliques may be the sum of the activation probabilities (occurrence probabilities) p_i,j of each (i,j) of the i cliques (total occurrence probability data 1902-i of i cliques). The calculation formula is as follows: Note that in the following, Σ indicates the sum of the total number n_i of i cliques that can be formed by the inter-feature point (location) synchronization network data 1005(F). k_i := Σ p_i,j The collective consciousness index calculation unit 1800 stores the calculated index (k_i) indicating the activity level of the entire set of i cliques (i clique appearance probability sum data 1902-i) in a medium or the like outside the system 101 that can be used by the system 101.

[0124] 18, the collective consciousness index calculation unit 1800 calculates the information entropy (H_i) for the i clique (information entropy data 1903-i for i clique) corresponding to the number of cliques i selected in the most recently executed step 1803 for the collective consciousness index calculation time period (Z_gc) selected in the most recently executed step 1802. The calculation formula for the information entropy (H_i) for the i clique is as follows: Note that in the following, Σ represents the sum of the total number n_i of i cliques that can be formed by the inter-feature point (part) synchronization network data 1005(F). H_i := - Σ p_i,j * log(p_i,j)

[0125] 18, the group consciousness index calculation unit 1800 calculates the average occurrence probability (k_i / n_i) corresponding to the number of cliques i selected in the most recently executed step 1803 for the group consciousness index calculation time period (Z_gc) selected in the most recently executed step 1802. Specifically, the average occurrence probability (k_i / n_i) corresponding to the number of cliques i is calculated by dividing the index (k_i) indicating the activity level of the entire set of i cliques calculated in step 1805 by the total number n_i of i cliques that can be formed by the inter-feature point (site) synchronization network data 1005(F).

[0126] 18, the collective consciousness index calculation unit 1800 calculates information entropy based on the average occurrence probability (k_i / n_i) corresponding to the number of cliques i, calculated in the most recently executed step 1807. The information entropy data calculated in this way may be called equal-probability information entropy data 1904-i(H_i(max)) of the i clique. The calculation formula for equal-probability information entropy data 1904-i(H_i(max)) of the i clique is as follows: Note that in the following, Σ represents the sum of the total number n_i of i cliques that can be formed by the inter-feature point (part) synchronization network data 1005(F). H_i(max) := - Σ (k_i / n_i) * log(k_i / n_i) = k_i * log(n_i / k_i)

[0127] In step 1809 of FIG. 18, the collective consciousness index calculation unit 1800 calculates an index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between i cliques, which is one of the collective consciousness indexes 1905-i for the i cliques corresponding to the number of cliques i selected in the most recently executed step 1803 for the collective consciousness index calculation time period (Z_gc) selected in the most recently executed step 1802. The index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between i cliques is calculated by the following formula using the information entropy (H_i) for the i clique calculated in step 1806 and the equiprobable information entropy (H_i(max)) for the i clique calculated in step 1808. Note that in the following formula, the calculated index is represented as r_i. r_i := 1 - H_i / H_i(max) The collective consciousness index calculation unit 1800 stores the calculated index (r_i:=1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the i cliques in a medium or the like within the system 101 or outside the system 101 that can be used by the system 101.

[0128] 18, the collective consciousness index calculation unit 1800 determines whether all of the clique numbers i for which the collective consciousness index is to be calculated have been selected in step 1803. If the determination result in step 1810 is positive, control transitions to step 1811. If the determination result in step 1810 is negative, control returns to step 1803, and one of the unselected clique numbers i is newly selected. In step 1811 of Fig. 18, the collective consciousness index calculation unit 1800 determines whether all of the collective consciousness index calculation time periods (Z_gc) for calculating the collective consciousness index have been selected in step 1802. If the determination result in step 1811 is positive, the processing in Fig. 18 ends. If the determination result in step 1811 is negative, control returns to step 1802, and one of the unselected collective consciousness index calculation time periods (Z_gc) is newly selected.

[0129] In the above, the "total number n_i of i cliques that can be formed by the inter-feature point (part) synchronization network data 1005(F)" has a different value for each clique number i. Here, if the number of feature points 123 (parts) is N_node, the value of n_i for i=1, 2, 3 is as follows: n_1 := N_node n_2 := N_node * ( N_node - 1 ) n_3 := N_node * ( N_node - 1 ) * ( N_node - 2 )

[0130] Figure 20 illustrates the meaning 2000 of the index (k_i) indicating the activity level of the entire set of i cliques, and the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques, contained in the collective consciousness index data 1905-i of i cliques. In Figure 20, the horizontal axis represents the magnitude of the value of the index (k_i) indicating the activity level of the entire set of i cliques, and the vertical axis represents the magnitude of the value of the index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the i cliques. As shown in Figure 20, the closer the value of the index (k_i) indicating the activity level of the entire set of i cliques is to the total number n_i of i cliques, the more active all i cliques are overall. Conversely, the closer the value of the index (k_i) indicating the activity level of the entire set of i cliques is to 0, the more inactive most i cliques are. The closer the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques is to 1 (i.e., the closer the information entropy H_i of i cliques is to 0), the less diverse the relationships between i cliques are and the more fixed the relationships between i cliques are. The closer the index (1-H_i / H_i(max)) indicating the randomness or cooperation level between i cliques is to 0 (i.e., the closer the information entropy H_i of i cliques is to the value of the equiprobable information entropy H_i(max)), the more diverse the relationships between i cliques are and the more random the relationships between i cliques are.

[0131] When analyzing the behavior of subject 102, if the index (k_i) indicating the activity level of the entire set of i cliques and the index (1-H_i / H_i(max)) indicating the randomness or cooperation between i cliques, which are included in the i clique collective consciousness index data 1905-i, are utilized, for example, if the index (1-H_i / H_i(max)) indicating the randomness or cooperation between i cliques is around 0.5 and does not fluctuate much, then the behavior of subject 102 can be interpreted as being smooth.

[0132] 5.8. Collective consciousness index display output (Figure 21, Figure 4, Figure 5) Fig. 21 shows an example of a display or output under the control of the system 101. Fig. 21 shows, for example, a screen example 2100 that is displayed on a display included in the display or output device 307 that the system 101 has. The example screen 2100 includes a time series display 2111-2 of the collective consciousness index for two cliques and a time series display 2111-3 of the collective consciousness index for three cliques. Therefore, in controlling the display of the example screen 2100 on the display, the collective consciousness index display output control unit 2101 (which controls the time series display 2111-2 of the collective consciousness index for two cliques and the time series display 2111-3 of the collective consciousness index for three cliques) may operate. The screen example 2100 also includes a user interface for setting various parameters by the user of the system 101. When an input for parameter setting is made using such a user interface, the parameter setting unit 501 operates. Below, the example screen 2100 will be explained.

[0133] The time series display 2111-2 of the collective consciousness index for two cliques and the time series display 2111-3 of the collective consciousness index for three cliques are similar to those shown in Fig. 20. The time series display 2111-2 of the collective consciousness index for two cliques is intended to plot and time-series display the collective consciousness index data 1905-2 for two cliques, corresponding to the number of cliques being two. The time series display 2111-3 of the collective consciousness index for three cliques is intended to plot and time-series display the collective consciousness index data 1905-3 for three cliques, corresponding to the number of cliques being three. The time series display 2111-2 of the collective consciousness index for two cliques and the time series display 2111-3 of the collective consciousness index for three cliques may each display one plot for each collective consciousness index calculation time period (Z_gc) on a two-dimensional plane, with the horizontal axis representing the value of the index (k_i) indicating the activity level of the entire set of i cliques and the vertical axis representing the value of the index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the i cliques. Adjacent collective consciousness index calculation time periods (Z_gc) on the time series may be connected by straight lines. As shown in FIG. 21, when a time series display 2111-2 of the collective consciousness index for two cliques and a time series display 2111-3 of the collective consciousness index for three cliques are produced, the user of system 101 can confirm the overall activity and randomness (degree of cooperation) of the behavior of subject 102 and can confirm changes over time, thereby enabling a detailed analysis of the behavior of subject 102. 21, the collective consciousness index data 1905-1 for one clique may be displayed or output in the same manner as the time series display 2111-2 of the collective consciousness index for two cliques or the time series display 2111-3 of the collective consciousness index for three cliques. The same applies to cases with four or more cliques.

[0134] The example screen 2100 may have a slide bar 511 for the synchronization network threshold (ε) and various parameter setting fields 519. These functions may be similar to those in the example screen 1400 of FIG. The example screen 2100 may have a "CSV data output" icon 2124. This function may be similar to the function of the "CSV data output" icon 1424 in the example screen 1400 of Fig. 21. However, there may be a difference in the type of data between the data such as processing results that can be output by clicking the "CSV data output" icon 2124 and the data such as processing results that can be output by clicking the "CSV data output" icon 1424.

[0135] The example screen 2100 may have a slide bar 513 for the synchronization network aggregation coefficient (N) for the collective consciousness index. The synchronization network aggregation coefficient (N) for the collective consciousness index slide bar 513 is used by a user of the system 101 to set the parameter N. As shown in FIG. 21 , a user interface for entering a value for the parameter N by text may be provided near the synchronization network aggregation coefficient (N) for the collective consciousness index slide bar 513 (to the right in FIG. 21 ). The example screen 2100 may have a slide bar 515 for the collective consciousness index calculation time zone shift amount (D). The slide bar 515 for the collective consciousness index calculation time zone shift amount (D) is used by a user of the system 101 to set the parameter D. As shown in FIG. 21 , a user interface for entering the value of parameter D as text may be provided near the slide bar 515 for the collective consciousness index calculation time zone shift amount (D) (to the right in FIG. 21 ).

[0136] The screen example 2100 provides a user interface for setting various parameters, and therefore a user of the system 101 who views the content displayed in the various windows provided on the screen example 2100 can change the settings of the various parameters. Then, after the system 101 executes processing based on the changed parameter values, the execution results may be reflected again in the various windows provided on the screen example 2100. Such an operation improves the convenience for users of the system 101. Furthermore, if the setting of any of the parameters is changed, the functional units shown in Figure 4 that are affected by the changed parameters and that directly or indirectly use the processing results of those functional units may redo their processing.

[0137] 6. Other (variations) The present disclosure is not limited to the above-described embodiments and includes various modifications. Part of the configurations and processes of the embodiments may be replaced with the configurations and processes of other conceivable embodiments. The configurations and processes of other conceivable embodiments may be added to the configurations and processes of the embodiments. For example, the present disclosure may include the following modified embodiments.

[0138] (Variation A) Optionality of functional units utilizing time series of synchronous network data As already mentioned in the explanation of FIG. 1, not all of the leading role degree calculation unit 900, leading role degree evaluation unit 1500, and collective consciousness index calculation unit 1800 are essential to the system 101 as functional units that utilize the time series of the inter-feature point (part) synchronization network data 1005(F). Depending on the purpose of analyzing the behavior of the subject 102, the system 101 may be modified in a manner that does not include the collective consciousness index calculation unit 1800, that does not include the leading role calculation unit 900 and the leading role evaluation unit 1500, or that does not include the leading role evaluation unit 1500. Alternatively, depending on the purpose of analyzing the behavior of the subject 102, the system 101 may have functional units other than the leading role degree calculation unit 900, the leading role degree evaluation unit 1500, and the collective consciousness index calculation unit 1800 as functional units that utilize the time series of the inter-feature point (part) synchronization network data 1005(F). Alternatively, since the calculation of the time series of inter-feature point (part) synchronization network data 1005(F) is itself useful for analyzing the behavior of the subject 102, the system 101 does not need to have functional units that utilize the time series of inter-feature point (part) synchronization network data 1005, such as the leading role degree calculation unit 900, the leading role degree evaluation unit 1500, and the collective consciousness index calculation unit 1800. In that case, the system 101 displays or outputs the time series of inter-feature point (part) synchronization network data 1005(F), or provides the time series of inter-feature point (part) synchronization network data 1005 to another system. As described above, the functional configuration of the system 101 according to the embodiment of the present disclosure can be flexibly determined.

[0139] The technical matters shown in the above-described embodiments of the present disclosure and the modified examples of the embodiments can be combined as appropriate as long as no technical contradiction occurs.

Claims

1. 1. A system comprising: a synchronization network calculation unit that calculates, based on video data consisting of a plurality of frame images, a time series of synchronization network data that indicates interlocking between a plurality of feature points of a subject shown in the video data; a leading role degree calculation unit that calculates a time series of leading role degree data for each of the feature points based on the time series of the synchronous network data;

2. 2. The system of claim 1, The system is configured to set a synchronization network calculation width (T) indicating the time width of a synchronization network calculation time period (Z_nw), which is a time period that is a unit for calculating the synchronization network data, a synchronization network shift width (S) indicating the time difference between the synchronization network calculation time periods (Z_nw) that are adjacent to each other on a time series, and a synchronization network threshold (ε), which is a threshold for determining whether there is a certain degree of interrelationship between feature points in the calculation of the synchronization network data; The system further includes a synchronization determination calculation unit, the synchronization determination calculation unit calculates an index indicating a degree of influence between the feature points for each synchronization network calculation time period (Z_nw) determined by the synchronization network calculation width (T) and the synchronization network shift width (S) based on a time series of data relating to a predetermined type of information for each feature point, the time series of feature point synchronization data; The synchronization network calculation unit determines whether there is interlocking between the feature points based on an index indicating the degree of influence between the feature points, which is included in the time series of the feature point synchronization data, and on the synchronization network threshold (ε), and calculates the time series of the synchronization network data by using the result of the determination for each of the feature points as the synchronization network data.

3. 3. The system of claim 2, the data relating to the predetermined type of information for each feature point is feature point movement amount data, which is information indicating the movement amount of coordinates on the frame image for each feature point; the index indicating the degree of influence between the feature points is a transfer entropy between the feature points; The synchronization determination calculation unit converting information indicating the movement amount included in the feature point movement amount data into a discretized random variable, thereby calculating random variable movement amount data; Calculating frequency distribution data by aggregating the frequencies of the random variable movement amount data; The system calculates a time series of the feature point synchronization data by calculating a transfer entropy between feature points based on the frequency distribution data.

4. 3. The system of claim 2, the subject includes a person having a skeleton and a tool used by the person having a skeleton, the data relating to the predetermined type of information for each feature point includes data relating to the predetermined type of information for each feature point included in the person having the skeleton and data relating to the predetermined type of information for each feature point included in the tool, The synchronization determination calculation unit, when forming the time series of the feature point synchronization data, handles both the feature points included in the person having the skeleton and the feature points included in the tool.

5. 2. The system of claim 1, The system further includes a synchronous network display output control unit; The synchronous network display output control unit controls the synchronous network data to be displayed or output in time series.

6. 2. The system of claim 1, The system wherein the dominance data for each feature point is composed of one or both of output page rank data indicating the degree to which each of the feature points influences all of the other feature points, and input page rank data indicating the degree to which each of the feature points is influenced by all of the other feature points.

7. 7. The system of claim 6, The system is configured to set a synchronization network set coefficient (M) for the degree of dominance and a time zone shift width (C) for calculating the degree of dominance, The leading role degree calculation unit From the time series of the synchronization network data, a set of the synchronization network data including the number of the synchronization network data indicated by the synchronization network set coefficient for dominance (M), and each of the sets of the synchronization network data that are adjacent to each other on the time series is selected while ensuring a time shift indicated by the dominance degree calculation time zone shift width (C) between the sets, and calculates a transition matrix corresponding to each of the selected sets of synchronization network data; The system calculates a time series of one or both of the output PageRank data and the input PageRank data by calculating one or both of the output PageRank data and the input PageRank data corresponding to each of the transition matrices.

8. 8. The system of claim 7, The system includes a leading role degree evaluation unit and a leading role degree display output control unit, the leading role evaluation unit calculates output PageRank cumulative value data by calculating a cumulative value of the output PageRank data for each feature point in a predetermined period based on the time series of the output PageRank data, and calculates input PageRank cumulative value data by calculating a cumulative value of the input PageRank data for each feature point in a predetermined period based on the time series of the input PageRank data, The system, wherein the leading role display output control unit controls to display or output one or more of the time series of the output page rank data, the output page rank cumulative value data, the time series of the input page rank data, and the input page rank cumulative value data.

9. 9. The system of claim 8, There are a plurality of the subjects, the system handles the video data for each of the subjects, The leading role evaluation unit Calculating the output PageRank cumulative value data for each of the objects and / or calculating the input PageRank cumulative value for each of the objects; performing one or both of the following for each of the objects: calculating output PageRank normalized data by normalizing the output PageRank cumulative value data; and calculating input PageRank normalized data by normalizing the input PageRank cumulative value data for each of the objects; calculating output PageRank average data by calculating an average value for each feature point based on the output PageRank normalized data for each subject, or calculating input PageRank average data by calculating an average value for each feature point based on the input PageRank normalized data for each subject, The system is configured such that the leading role display output control unit performs one or both of the following: control to display or output the output PageRank average data in a manner that allows comparison between the output PageRank normalized data of the selected subject; and control to display or output the input PageRank average data in a manner that allows comparison between the input PageRank normalized data of the selected subject.

10. 2. The system of claim 1, The system further comprises a collective consciousness index calculation unit; the collective consciousness index calculation unit calculates collective consciousness index data or a time series of the collective consciousness index data for a plurality of the feature points included in the subject based on the time series of the synchronized network data, The group consciousness index data for the plurality of feature points included in the subject is composed of an index (k_i) indicating the activity level of the entire set of cliques (i, j) for each clique number (i), which is the number of feature points included in one clique (i, j), and an index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the cliques (i, j).

11. 11. The system of claim 10, The system is configured to set a synchronization network aggregation coefficient (N) for collective consciousness index and a collective consciousness index calculation time zone shift width (D), The collective consciousness index calculation unit From the time series of the synchronization network data, sets of the synchronization network data are selected, each set including the number of the synchronization network data indicated by the synchronization network set coefficient for collective consciousness index (N), and each set of the synchronization network data is selected so that there is a time shift between the sets indicated by the collective consciousness index calculation time zone shift width (D); For each of the selected sets of synchronization network data, calculate an occurrence probability (p_i,j) for each clique number (i), which is the probability that each of the cliques (i,j) will be activated; For each clique number (i), the sum of the occurrence probabilities (p_i,j) of the cliques (i,j) is calculated to calculate occurrence probability sum data (k_i), which is used as an index (k_i) indicating the activity level of the entire set of the cliques (i,j); For each clique number (i), calculate information entropy data (H_i) based on the occurrence probability (p_i,j) of each clique (i,j); For each clique number (i), calculate equal probability information entropy data (H_i(max)) based on the occurrence probability sum data (k_i) of the clique number (i) and the total number (n_i) of cliques (i, j); A system that calculates, for each number of cliques (i), an index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the cliques (i, j) based on the information entropy data (H_i) and the equal probability information entropy data (H_i(max)).

12. 11. The system of claim 10, The system further comprises a collective consciousness index display output control unit; The collective consciousness index display output control unit controls the display or output of a plot showing a combination of a value indicated by an index (k_i) indicating the activity level of the entire set for one or more clique numbers (i) and a value indicated by an index (1-H_i / H_i(max)) indicating the degree of randomness or cooperation between the cliques (i, j), for the clique numbers (i).

13. 13. The system of claim 12, The collective consciousness index display output control unit controls the display or output of the time series of the plots in accordance with the time series of the collective consciousness index data.

14. A method performed by a system, comprising: a synchronization network calculation step of calculating a time series of synchronization network data indicating interlocking between feature points of a plurality of feature points of a subject shown in the video data based on the video data consisting of a plurality of frame images; The method further comprises a step of calculating a time series of the prominence degree data for each of the feature points based on the time series of the synchronous network data.

15. A program, The program includes: a synchronization network calculation step of calculating a time series of synchronization network data indicating interlocking between feature points of a plurality of feature points of a subject shown in the video data based on the video data consisting of a plurality of frame images; a program for executing a leading role degree calculation step of calculating a time series of leading role degree data for each of the feature points based on the time series of the synchronous network data;

Images