Group behavior analysis device, group behavior analysis system, and group behavior analysis method

Abstract

To explanatorily present a network among individuals in a group.SOLUTION: A group action analysis device 100 comprises: a synchronous calculation part (synchronous calculation program 152) which estimates causality among individuals in each combination of a plurality of individuals from sensor data 161 acquired from the plurality of individuals, estimates, for each combination of the plurality of individuals, a distance between individuals and a direction of the other individual viewed from one individual from the sensor data 161, and also estimates accuracy of recognition between the individuals based upon the estimated distance between the individuals and the directions of the individuals; and a network display part (network display program 153) which displays a network among the individuals in group action based upon the causality among the individuals in each combination of the plurality of individuals and the accuracy of the recognition which are estimated by the synchronous calculation part.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a group behavior analysis device, a group behavior analysis system, and a group behavior analysis method, and is suitable for application to a group behavior analysis device, a group behavior analysis system, and a group behavior analysis method that analyze the network between individuals in group behavior.

Background Art

[0002] Conventionally, in the start of group sports or the like, there is a technique for detecting direct interactions such as ball passing exchanges and evaluating cooperation to analyze the network of the group. In recent years, a network analysis system has been proposed that detects and evaluates the correlation of body movements and direct face-to-face relationships with sensors.

[0003] For example, Patent Document 1 discloses a network analysis system that analyzes and visualizes the network of a group when individuals who are not facing each other in a group move with a relationship. The network analysis system of Patent Document 1 converts time-series data of acceleration into discrete probability variables to calculate movement entropy, and visualizes a network having a relationship with a certain threshold value or more using this movement entropy.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, causal estimation using a single variable, such as that employed in the network analysis system of Patent Document 1, is a convenient method, but under unrestricted circumstances, it may inadvertently infer causality between data of two parties. In such cases, it is impossible to distinguish whether the two parties truly have a relationship of information exchange or communication. Furthermore, the network analysis system of Patent Document 1 also has the problem that the estimated causality and correlation lack explanatory power, making human interpretation difficult.

[0006] This invention has been made in consideration of the above points, and aims to propose a group behavior analysis device, a group behavior analysis system, and a group behavior analysis method that can present the network between individuals in group behavior in an explainable manner. [Means for solving the problem]

[0007] To solve these problems, the present invention provides a group behavior analysis device that analyzes the network between individuals in a group behavior of multiple individuals, and estimates the causality between individuals in each combination of the multiple individuals from sensor data acquired from the multiple individuals or video data of the multiple individuals. Causality estimation process and For each combination of the multiple individuals, the distance between the individuals and the orientation of the individuals (the direction of one individual as seen from the other individual) are estimated from the sensor data or video data, and the accuracy of recognition between the individuals is estimated based on the estimated distance and orientation between the individuals. Recognition accuracy estimation process, and capable of performing The system comprises a synchronization calculation unit and a network display unit that displays the network between individuals in the group behavior based on the causality between individuals and the accuracy of recognition in each combination of the plurality of individuals estimated by the synchronization calculation unit. The synchronization calculation unit, in the causality estimation process, uses the interpersonal movement entropy calculated using time-series data of the amount of movement per unit time of each individual as an index value indicating causality between the individuals, and in the recognition accuracy estimation process, the synchronization calculation unit estimates a higher accuracy of recognition of the other individual by the first individual the closer the distance from the first individual to the other individual is, and estimates a higher accuracy of recognition of the other individual by the first individual the closer the direction of the other individual as seen from the first individual is to the center of the first individual's field of view. A group behavior analysis device characterized by the above is provided.

[0008] Furthermore, in order to solve the above problems, the present invention provides a group behavior analysis system for analyzing the network between individuals in group behavior of multiple individuals, comprising: a wearable device with sensors attached to each of the multiple individuals, or a camera for photographing the multiple individuals; and a group behavior analysis device connected to receive sensor data detected by the sensors of the wearable devices or video data captured by the camera, wherein the group behavior analysis device estimates the causality between individuals in each combination of the multiple individuals from the sensor data or the video data. Causality estimation process and For each combination of the multiple individuals, the distance between the individuals and the orientation of the individuals (the direction of one individual as seen from the other individual) are estimated from the sensor data or video data, and the accuracy of recognition between the individuals is estimated based on the estimated distance and orientation between the individuals. Recognition accuracy estimation process, and capable of performing The system includes a synchronization calculation unit and a network display unit that displays the network between individuals in the group behavior based on the causality between individuals and the accuracy of recognition in each combination of the plurality of individuals estimated by the synchronization calculation unit. Furthermore, in the causality estimation process, the synchronization calculation unit uses the interpersonal movement entropy calculated using time-series data of the amount of movement per unit time of each individual as an index value indicating causality between the individuals, and in the recognition accuracy estimation process, the synchronization calculation unit estimates a higher accuracy of recognition of the other individual by the first individual the closer the distance from the first individual to the other individual is, and estimates a higher accuracy of recognition of the other individual by the first individual the closer the direction of the other individual as seen from the first individual is from the center of the first individual's field of view. A group behavior analysis system characterized by the above is provided.

[0009] Furthermore, in order to solve the above problems, the present invention provides a method for analyzing group behavior using a group behavior analysis device that analyzes the network between individuals in group behavior of multiple individuals, comprising a causality estimation step in which the group behavior analysis device estimates the causality between individuals in each combination of the multiple individuals from sensor data acquired from the multiple individuals or video data of the multiple individuals, The aforementioned The group behavior analysis device estimates the distance between individuals and the orientation of each individual (the direction of one individual as seen from the other individual) for each combination of the plurality of individuals from the sensor data or the video data, and estimates the accuracy of recognition between individuals based on the estimated distance between individuals and the orientation of each individual in the recognition accuracy estimation step. The aforementionedThe group behavior analysis device includes a network display step that displays the network between individuals in the group behavior based on the causality estimation step and the recognition accuracy estimation step, which estimate the causality between individuals and recognition accuracy in each combination of the plurality of individuals. In the causality estimation step, the group behavior analysis device uses the interpersonal movement entropy calculated using time-series data of the amount of movement per unit time of each individual as an index value indicating causality between the individuals, and in the recognition accuracy estimation step, the group behavior analysis device estimates a higher accuracy of recognition of the other individual by the first individual the closer the distance from the first individual to the other individual, and estimates a higher accuracy of recognition of the other individual by the first individual the closer the direction of the other individual as seen from the first individual is to the center of the first individual's field of view. A method for analyzing group behavior is provided, characterized by the following features. [Effects of the Invention]

[0010] According to the present invention, it is possible to present the networks between individuals in group behavior in an explainable manner. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing an example configuration of the group behavior analysis system 1 according to the first embodiment of the present invention. [Figure 2] This figure shows an example of how the wearable device 300 is worn. [Figure 3] This is an illustrative diagram illustrating an example of how to set up and use a group behavior analysis system. [Figure 4] This figure shows an example of sensor data 161. [Figure 5] This figure shows an example of user management data 162. [Figure 6] This figure shows an example of game management data 163. [Figure 7] This figure shows an example of graph data 164. [Figure 8] This figure shows an example of recognition accuracy data 165. [Figure 9] This figure shows an example of synchronization data 166. [Figure 10] This figure shows an example of display setting data 167. [Figure 11] This flowchart shows an example of the processing procedure for synchronous calculation. [Figure 12] This is a diagram to explain the calculation method for recognition accuracy. [Figure 13]It is a flowchart showing an example of a processing procedure for moving entropy calculation processing. [Figure 14] It is a diagram showing an image for calculating moving entropy from two probability variables. [Figure 15] It is a flowchart showing an example of a processing procedure for network display processing. [Figure 16] It is a diagram showing an example (part 1) of the display of an analysis result display screen. [Figure 17] It is a diagram showing an example (part 2) of the display of an analysis result display screen. [Figure 18] It is a diagram showing an example (part 3) of the display of an analysis result display screen. [Figure 19] It is a block diagram showing a configuration example of a group behavior analysis system 2 according to a second embodiment of the present invention. [Figure 20] It is a diagram showing an example of person coordinate data 562. [Figure 21] It is a diagram showing an example of ball coordinate data 563. [Figure 22] It is a diagram for explaining a method of estimating the orientation of the body using ball coordinates. [Figure 23] It is a block diagram showing a configuration example of a group behavior analysis system 3 according to a third embodiment of the present invention. [Figure 24] It is a diagram showing an image of a skeletal part to be analyzed by image analysis. [Figure 25] It is a diagram showing an example of skeletal coordinate data 861. [Figure 26] It is a diagram showing an example of user management data 862. [Figure 27] It is a diagram showing an example of the display of an analysis result display screen in the third embodiment.

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] The following descriptions and drawings are illustrative examples for explaining the present invention, and have been omitted and simplified as appropriate for clarity of explanation. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. The present invention is not limited to the embodiments, and any application that aligns with the spirit of the invention falls within its technical scope. Those skilled in the art can make various additions and modifications within the scope of the present invention. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0014] In the following explanation, various types of information may be described using terms such as "table," "list," and "queue," but these types of information may also be represented using other data structures. To indicate independence from data structure, "XX table," "XX list," etc., may be referred to as "XX information." When describing the content of each type of information, terms such as "identification information," "identifier," "name," "ID," and "number" will be used, but these terms are interchangeable.

[0015] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code or a common number in the reference code will be used. When describing similar elements with distinction, the reference code of that element will be used, or the ID assigned to that element will be used instead of the reference code.

[0016] Furthermore, while the following description may include explanations of processes performed by executing a program, the processor may be the primary entity performing the processing, as a program is executed by at least one processor (e.g., a CPU) and performs defined processes using appropriate memory resources (e.g., memory) and / or interface devices (e.g., communication ports). Similarly, the primary entity performing the processing by executing a program may be a controller, device, system, computer, node, storage system, storage device, server, management computer, client, or host having a processor. The primary entity performing the processing by executing a program (e.g., a processor) may include hardware circuits that perform some or all of the processing. For example, the primary entity performing the processing by executing a program may include hardware circuits that perform encryption and decryption, or compression and decompression. The processor operates as a functional unit that realizes predetermined functions by operating according to the program. Devices and systems including a processor are devices and systems including these functional units.

[0017] A program may be installed from its program source into a device such as a computer. The program source may be, for example, a program distribution server or a non-temporary storage medium readable by a computer. If the program source is a program distribution server, the program distribution server includes a processor (e.g., a CPU) and non-temporary storage resources, which may further store the distribution program and the program to be distributed. The processor of the program distribution server may then execute the distribution program, thereby distributing the program to other computers. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0018] (1) First Embodiment (1-1) Composition Figure 1 is a block diagram showing an example configuration of a group behavior analysis system 1 according to the first embodiment of the present invention. The group behavior analysis system 1 is a system that analyzes the network between individuals in group behavior and presents the analysis results to the user, and as shown in Figure 1, comprises a group behavior analysis device 100, a wireless communication device 200, and a wearable device 300.

[0019] The group behavior analysis device 100 is a computer such as a personal computer (PC), and has an external communication unit 110, a calculation unit 120, a control unit 130, a memory 140, and a storage unit 150. Each unit is interconnected by an internal communication line (bus).

[0020] The external communication unit 110 has the function of enabling communication between the group behavior analysis device 100 and an external device (wireless communication device 200). Specifically, the external communication unit 110 is, for example, a NIC (Network Interface Card). The communication standard used by the external communication unit 110 is not particularly limited and includes, for example, LTE (Long Term Evolution), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the case of Figure 1, the external communication unit 110 is connected to the wireless communication device 200 and receives sensor data detected by the wearable device 300 in real time via the wireless communication device 200. If real-time response is not required, the sensor data detected by the wearable device 300 may be stored in a flash memory 330 or the like, and the stored sensor data may be input to the group behavior analysis device 100 at a predetermined timing and in a predetermined manner.

[0022] The calculation unit 120 has the function of performing various calculations according to the control of the control unit 130. The control unit 130 also has the function of controlling various processes such as communication, input / output, and calculations in the group behavior analysis device 100. The calculation unit 120 and the control unit 130 are processors such as a CPU (Central Processing Unit), and they realize various functions by executing programs read from, for example, the storage unit 150 or the memory 140 from an external source.

[0023] Memory 140 is a main memory device that temporarily stores programs and data used in the group behavior analysis device 100, and is, for example, RAM (Random Access Memory).

[0024] The memory unit 150 is an auxiliary storage device that non-temporarily stores programs and data used in the group behavior analysis device 100, and is, for example, an SSD (Solid State Drive) or HDD (Hard Disk Drive). Note that at least a portion of the programs and data shown in Figure 1 as being stored in the memory unit 150 may be held externally via a network and read into memory 140 as needed. Furthermore, in the following description, programs may be described as the main processing element.

[0025] The memory unit 150 stores the following programs: a data reception program 151, a synchronization calculation program 152, and a network display program 153. The memory unit 150 also stores the following data: sensor data 161, user management data 162, game management data 163, graph data 164, recognition accuracy data 165, synchronization data 166, and display setting data 167. Details of these programs and data will be described later with reference to the drawings.

[0026] The wireless communication device 200 is a communication device capable of wireless communication, and plays the role of relaying wireless communication between the wireless communication unit 310 of the wearable device 300 and the external communication unit 110 of the collective behavior analysis device 100. The communication standard used by the wireless communication device 200 is not particularly limited, as long as it conforms to the communication standard used by the external communication unit 110 and the wireless communication unit 310.

[0027] The wearable device 300 is a wearable device attached to each individual (specifically, for example, each athlete participating in a team sport) of the group behavior analysis system 1 that is the target of analysis. The wearable device 300 can utilize a general-purpose wearable device, and for example, it may have a wireless communication unit 310, a microcontroller 320, a flash memory 330, a USB communication unit 340, an acceleration sensor 350, a gyro sensor 360, a geomagnetic sensor 370, and a GNSS receiver unit 380.

[0028] The wireless communication unit 310 has wireless communication capabilities and transmits sensor data and other information collected by the wearable device 300 to the wireless communication device 200. The microcontroller 320 performs overall control of the wearable device 300. The flash memory 330 stores sensor data and location data collected by the wearable device 300. The USB communication unit 340 has USB (Universal Serial Bus) communication capabilities and, for example, transmits data stored in the flash memory 330 to a USB memory inserted into the wearable device 300.

[0029] The acceleration sensor 350 is a sensor that detects the acceleration of an individual wearing the wearable device 300, the gyro sensor 360 is a sensor that detects the angular velocity (amount of rotation) of the individual, and the geomagnetic sensor 370 is a sensor that measures the direction (body orientation) of the individual by detecting the Earth's magnetic field. The GNSS receiver 380 is a device that uses GNSS (Global Navigation Satellite System) to determine the position and velocity (or distance) of the individual. The collected data (sensor data) of the individual collected by each of the sensors 350 to 370 and the GNSS receiver 380 is stored in the flash memory 330 and also transmitted from the wireless communication unit 310 to the external communication unit 110 of the collective behavior analysis device 100 via the wireless communication device 200.

[0030] The sensors mounted on the wearable device 300 only need to be capable of collecting the sensor data necessary for network analysis in the group behavior analysis device 100, and are not limited to the configuration exemplified in Figure 1. For example, it is sufficient if at least a GNSS receiver 380 is mounted. Also, for example, a gyro sensor 360 can determine acceleration and body orientation in more detail than the accelerometer 350 and geomagnetic sensor 370, but it does not necessarily have to be mounted on the wearable device 300.

[0031] Figure 2 shows an example of how the wearable device 300 is worn. The wearable device 300 shown in Figure 2 is fixed near the center of the subject's back.

[0032] Figure 3 is an illustrative diagram illustrating an example of the installation and use of the group behavior analysis system 1. In Figure 3 and subsequent figures in the first embodiment, the group behavior targeted for network analysis is assumed to be a competitive sports game played on a court (hereinafter referred to as "game"). As shown in Figure 3, each player wears a wearable device 300 as shown in Figure 2. One or more wireless communication devices 200 are installed around the court, and each wireless communication device 200 is connected to the group behavior analysis device 100 for communication. When a game is played in this setup, sensor data is transmitted in real time from the wearable devices 300 worn by each player, and this sensor data is collected by the group behavior analysis device 100 via the wireless communication devices 200.

[0033] (1-2) Data The following provides specific examples of the various data stored in the memory unit 150 of the group behavior analysis device 100 shown in Figure 1.

[0034] Figure 4 shows an example of sensor data 161. Sensor data 161 is continuously accumulated by the data reception program 151, based on sensor data from each individual that is attached to a wearable device 300 worn by each individual performing group behavior, and received via a wireless communication device 200.

[0035] The sensor data 161 shown in Figure 4 generates a record each time sensor data is received from a wearable device 300, and consists of the following items: sensor ID 1611, date and time 1612, acceleration 1613, angular velocity 1614, geomagnetic field 1615, latitude and longitude 1616, and velocity 1617.

[0036] Sensor ID 1611 indicates the sensor ID of the wearable device 300 that transmitted the sensor data. The sensor ID is a unique identifier pre-assigned to each wearable device 300. Date and time 1612 indicates the date and time information when the sensor data was received (or transmitted).

[0037] Each item from acceleration 1613 to velocity 1617 records detection data from various sensors included in the sensor data. Specifically, acceleration 1613 shows detection data from acceleration sensor 350. Angular velocity 1614 shows detection data from gyro sensor 360. Geomagnetic field 1615 shows detection data from geomagnetic field sensor 370. Latitude and longitude 1616 shows positioning data from GNSS receiver 380. Velocity 1617 shows velocity measurement data from GNSS receiver 380.

[0038] Figure 5 shows an example of user management data 162. User management data 162 holds information about the target individuals (in this example, athletes) of the network analysis performed by the group behavior analysis device 100. The users (target individuals) managed by user management data 162 are also users of the wearable device 300.

[0039] The user management data 162 shown in Figure 5 has a record for each target individual and consists of the following items: User ID 1621, Sensor ID 1622, Team ID 1623, Start Date and Time 1624, and End Date and Time 1625.

[0040] User ID 1621 indicates the identifier (User ID) individually assigned to the target individual. Sensor ID 1622 indicates the sensor ID of the wearable device 300 attached to the target individual. Team ID 1623 indicates the identifier (Team ID) individually assigned to the team to which the target individual belongs in the group behavior being analyzed (in this example, a game). Start date and time 1624 indicates the start date and time of the group behavior being analyzed, and end date and time 1625 indicates the end date and time of the group behavior being analyzed.

[0041] Figure 6 shows an example of game management data 163. Game management data 163 holds information about the group behavior (in this example, the game) that is the target of network analysis by the group behavior analysis device 100. User management data 162 and game management data 163 are sometimes collectively referred to as "management data".

[0042] The game management data 163 shown in Figure 6 has a record for each group activity (game) being analyzed and consists of the following items: game ID 1631, start date and time 1632, end date and time 1633, participating team ID 1634, game name 1635, and participating team name 1636.

[0043] Game ID 1631 indicates the identifier (Game ID) assigned to the game in question. Start time 1632 indicates the start time of the game, and end time 1633 indicates the end time of the game. Participating team ID 1634 indicates the team ID of the team participating in the game, and participating team name 1636 indicates the team name. Game name 1635 indicates the name of the game.

[0044] Figure 7 shows an example of graph data 164. Graph data 164 is data generated during the synchronous calculation process by the synchronous calculation program 152, and defines all possible combinations of individuals that can constitute a network in the collective behavior (game) being analyzed (in this example, the total number of combinations of players). For each edge connecting two individuals, graph data 164 records information about the combinations of nodes (points, i.e., individuals) at both ends of that edge.

[0045] The graph data 164 shown in Figure 7 consists of the following items: graph ID 1641, game ID 1642, starting user ID 1643, and ending user ID 1644.

[0046] Graph ID 1641 indicates the identifier (graph ID) assigned to each edge of the network between individuals. Game ID 1642 indicates the game ID of the game being analyzed. Starting user ID 1643 indicates the user ID assigned to the individual (player) at the starting point of the edge corresponding to graph ID 1641, and ending user ID 1644 indicates the user ID assigned to the individual (player) at the ending point of that edge.

[0047] Figure 8 shows an example of recognition accuracy data 165. Recognition accuracy data 165 is data generated during the synchronous calculation process by the synchronous calculation program 152, and for each graph ID corresponding to a combination of individuals, information regarding the recognition accuracy from the starting individual to the ending individual is recorded.

[0048] The recognition accuracy data 165 shown in Figure 8 consists of the following items: graph ID 1651, angle 1652, distance between two parties 1653, and recognition accuracy 1654.

[0049] Graph ID 1651 corresponds to Graph ID 1641 of Graph Data 164. Angle 1652 indicates the angle from the center of the field of view of the starting individual (player with starting user ID 1643) to the ending individual (player with ending user ID 1644). Distance between two individuals 1653 indicates the distance between the starting individual and the ending individual. Recognition accuracy 1654 indicates the calculated value of the degree to which the starting individual recognizes the ending individual (recognition accuracy).

[0050] Figure 9 shows an example of synchronous data 166. Synchronized data 166 is data generated during the synchronous calculation process by the synchronous calculation program 152, and for each graph ID corresponding to a combination of individuals, the transfer entropy and weighted transfer entropy from the starting individual to the ending individual are recorded.

[0051] The synchronization data 166 shown in Figure 9 consists of the items graph ID 1661, moving entropy 1662, and weighted moving entropy 1663.

[0052] Graph ID 1661 corresponds to graph ID 1641 of graph data 164. Transfer entropy 1662 represents the transfer entropy for the combination of individuals shown in graph ID 1641, and weighted transfer entropy 1663 represents the weighted transfer entropy for the combination of individuals shown in graph ID 1641. Transfer entropy and weighted transfer entropy will be explained later in the explanation referring to Figures 11 to 14.

[0053] Figure 10 shows an example of display setting data 167. Display setting data 167 is data used when the network display program 153 displays the results of the network analysis of collective behavior analyzed by the synchronous calculation program 152 on the analysis result display screen, and it holds information related to the display settings of the analysis result display screen.

[0054] The display setting data 167 shown in Figure 10 consists of items such as game ID 1671, movement entropy threshold 1672, recognition accuracy threshold 1673, and network display method 1674.

[0055] Game ID 1671 corresponds to Game ID 1631 in game management data 163. The moving entropy threshold 1672 indicates the display threshold for moving entropy to be displayed on the analysis results display screen for the game indicated by Game ID 1671, and the recognition accuracy threshold 1673 similarly indicates the display threshold for recognition accuracy. As shown in Figures 16 to 18 described later, the analysis results display screen can be configured so that the user can adjust the display thresholds for moving entropy and recognition accuracy. The network display method 1674 indicates the network display method on the analysis results display screen. The network display method is not limited to a specific method. For example, in Figures 16 to 18 described later, a location-based network display method is exemplified, but a location-less network display method may be selected.

[0056] (1-3) Processing In the following sections, we will explain in detail the processes performed by the group behavior analysis system 1 (group behavior analysis device 100) using the configuration and data examples shown in Figures 1 to 10.

[0057] (1-3-1) Synchronous calculation process Figure 11 is a flowchart showing an example of the processing procedure for synchronous calculation. The synchronous calculation is performed by the synchronous calculation program 152 after the data receiving program 151 collects sensor data from the wearable device 300.

[0058] The trigger for starting the synchronous calculation process may be when a user instructs the execution of network analysis, or it may be performed periodically, for example. Furthermore, if network analysis is performed in real time during gameplay, the synchronous calculation process should be performed on sensor data for the most recent predetermined period, and if network analysis is performed after the game has ended, the synchronous calculation process should be performed on all sensor data collected during the game. In either case, before the start of the game, predetermined item information that should be set in advance is registered in user management data 162 and game management data 163, and other predetermined conditions necessary for the processing are also specified.

[0059] As shown in Figure 11, first, the synchronous calculation program 152 reads sensor data 161 in time series for the individual (player) to be analyzed (step S101). In step S101, the synchronous calculation program 152 can obtain the user ID of the individual to be analyzed and the sensor ID of the wearable device 300 worn by the individual by referring to management data (user management data 162, game management data 163). Then, the synchronous calculation program 152 generates combinations of two individuals using the obtained user IDs, assigns a graph ID to each of these generated combinations, and records them in graph data 164. For the sake of simplicity, in the following explanation, we will assume that an individual is a person, refer to "two individuals" as "two parties", refer to the individual at the starting point of the network as the "starting user", and refer to the individual at the ending point of the network as the "ending user".

[0060] Next, the synchronous calculation program 152 estimates the body orientation between two individuals for all combinations of individuals recorded in the graph data 164 in step S101, and records the body orientation from the starting user to the ending user as the angle 1652 in the recognition accuracy data 165 (step S102).

[0061] Specifically, in step S102, the synchronization calculation program 152 uses the starting user ID 1643 and ending user ID 1644 recorded in the graph data 164 as keys to obtain the sensor ID 1622 of the wearable device 300 attached to each user from the user management data 162. Next, the synchronization calculation program 152 refers to the sensor data 161 for the respective sensor IDs 1622 of the starting and ending users to obtain the position (latitude and longitude 1616) and body orientation (angular velocity 1614) of each user. Based on the above obtained information, the synchronization calculation program 152 estimates the body orientation from the starting user to the ending user (the direction of the ending user as seen from the starting user), using the starting user's body orientation as the center of the field of view, and records the estimation result in angle 1652 of the recognition accuracy data 165.

[0062] Next, the synchronization calculation program 152 estimates the comprehensive distance between the two parties, similar to step S102, and records the estimation result in the distance between the two parties 1653 of the recognition accuracy data 165 (step S103). Specifically, the synchronization calculation program 152 can estimate the distance between the two parties from the position of the starting user and the position of the ending user.

[0063] Next, the synchronous calculation program 152 calculates the accuracy of recognition of the end user by the starting user (recognition accuracy) for all possible combinations of two persons (each combination of multiple individuals), based on the body orientation from the starting user to the end user estimated in step S102 and the distance between the two persons estimated in step S103, taking into account the field of view of the starting user, and records the calculation result in the recognition accuracy 1654 of the recognition accuracy data 165 (step S104).

[0064] Figure 12 is a diagram illustrating the calculation of recognition accuracy. Figure 12(A) shows an example of the positional relationship between the starting user and the ending user, Figure 12(B) shows the relationship between the distance between the two and the recognition accuracy, and Figure 12(C) shows the relationship between the accuracy of the ending user from the starting user's field of view center and the recognition accuracy.

[0065] As shown in Figure 12(A), the starting user's field of view is within a predetermined estimated field of view angle centered on the orientation of their body. The estimated field of view angle can be predetermined to be, for example, around 120 degrees, but it may be arbitrarily changed by the user. As shown in Figure 12(B), the recognition accuracy decreases as the distance between the two parties increases. Also, as shown in Figure 12(C), the recognition accuracy decreases as the angle from the starting user to the ending user increases (i.e., as the ending user moves further away from the center of the starting user's field of view). As can be seen from Figures 12(A) to (C), the starting user's recognition accuracy of the ending user is higher when the ending user is at a close angle and distance from the starting user's field of view center, and decreases as the angle or distance increases. The synchronization calculation program 152 can calculate the recognition accuracy by using a calculation formula that quantifies this trend.

[0066] Returning to the explanation of Figure 11, following step S105, the synchronization calculation program 152 calculates the comprehensive transfer entropy between the two parties and records the calculation result in the transfer entropy 1662 of the synchronization data 166 (step S106). The method for calculating the transfer entropy will be described in detail later with reference to Figures 13 and 14.

[0067] Next, the synchronization calculation program 152 weights the transfer entropy between the two parties calculated in step S106 by the recognition accuracy calculated in step S104, records the result of the weighted transfer entropy calculation in the weighted transfer entropy 1663 of the synchronization data 166 (step S107), and then terminates the synchronization calculation process.

[0068] As described above, by executing the processes in steps S101 to S107, graph data 164, recognition accuracy data 165, and synchronization data 166 are generated using the graph ID, which represents the combination between individuals, as the key. In particular, in this embodiment, in step S106, not only is the comprehensive transfer entropy between two individuals in the population calculated to estimate the causality between the two individuals, but in step S107, the synchronization (causality) between the two individuals can be analyzed with greater accuracy by weighting using mutual or unidirectional recognition accuracy based on the distance and orientation between the two individuals. Note that the specific weighting method is not limited as long as it reflects the distance and orientation between the two individuals.

[0069] Figure 13 is a flowchart illustrating an example of the processing procedure for the movement entropy calculation process. Figure 14 is a diagram illustrating the image of calculating movement entropy from two random variables. The movement entropy calculation process is a process in which the synchronous calculation program 152 calculates the movement entropy from the starting user to the ending user, and is executed in step S105 of Figure 11. The movement entropy calculated in this embodiment targets a comprehensive relationship between two individuals, and takes acceleration data obtained from the acceleration 1613 of the sensor data 161 as input, converts it into normalized and discrete random variables for each individual, and then calculates the movement entropy between the two individuals from the random variables of the two individuals.

[0070] As shown in Figure 13, the synchronization calculation program 152 first aggregates the amount of movement per unit time for each target individual from the read sensor data 161 (see step S101 in Figure 11) and performs a histogram calculation (step S201). Graph 401 in Figure 13 shows an image of the aggregation of movement amounts in step S201, and a graph showing the change in frequency per momentum distance is generated based on a graph showing the change in momentum (amount of movement) per unit time.

[0071] Next, the synchronous calculation program 152 divides the histogram calculated in step S201 into predetermined class widths, for example, based on Sturges' formula (step S202). Graph 402 in Figure 13 shows the histogram divided into predetermined class widths. In this example, Sturges' formula "k = log2N + 1 (N: sample size, k: number of classes)" is used, but other formulas or methods for determining the class width of the histogram may also be used.

[0072] Next, the synchronous calculation program 152 normalizes and discretizes the movement based on the histogram splitting performed in step S202 (step S203). Graph 403 in Figure 13 is a graph showing the random variable for each time step, generated by the normalization and discretization. Known methods can be used for normalization and discretization.

[0073] Next, the synchronous calculation program 152 calculates the movement entropy from the time series data of the movement between each coordinate after normalization and discretization in step S203 (step S204).

[0074] For details, Figure 14(A) shows the time series random variables for point I and point J, respectively, as examples of time series data of the amount of movement between each coordinate after normalization and discretization in step S203. In other words, these time series random variables correspond to the values ​​of the random variables for each class in graph 403 in Figure 13.

[0075] The synchronization calculation program 152 can then calculate the movement entropy from point J to point I by using the two time-series random variables mentioned above and calculating the formula shown in Figure 14(B). In other words, by setting point J as the starting user and point I as the ending user, the movement entropy from the starting user to the ending user can be calculated. If the movement entropy between the two parties calculated in this way is high, it can be estimated that there is synchronization (causality) between the movements of the two parties.

[0076] Finally, the synchronization calculation program 152 records the movement entropy from the starting user to the ending user, calculated as described above, in the movement entropy 1662 of the synchronization data 166, and terminates the movement entropy calculation process.

[0077] While the method for calculating moving entropy was described above, the collective behavior analysis device 100 according to this embodiment can also employ other analytical methods that quantify the relationship between two points, such as cross-correlation functions, DTW, or mutual information, as an alternative to moving entropy. However, these analytical methods may have limitations compared to using moving entropy, such as being unable to indicate directionality or being unable to appropriately represent differences in timing (time differences).

[0078] (1-3-2) Network display processing Figure 15 is a flowchart illustrating an example of the network display processing procedure. Network display processing is a process that presents the results of the network analysis of group behavior analyzed by synchronous calculation processing to the user of the group behavior analysis device 100 by drawing them on the analysis results display screen, and is executed by the network display program 153. Before the start of network display processing, the group behavior (target game) to be displayed on the analysis results display screen is specified by the user of the group behavior analysis device 100. More specifically, the game ID of the target game may be specified directly, or the date and time of the target game, participating teams, etc., may be specified.

[0079] According to Figure 15, first, the network display program 153 searches for management data (user management data 162, game management data 163) based on the specified conditions set by the user and identifies the game ID of the target game (step S301).

[0080] Next, the network display program 153 uses the game ID identified in step S301 as a key to retrieve the display settings for the corresponding record from the display settings data 167 (step S302).

[0081] Next, the network display program 153 searches the graph data 164 using the game ID identified in step S301 as a key, and identifies all the graph IDs of the interpersonal network in the target game indicated by the game ID. Furthermore, for each identified graph ID, the network display program 153 refers to the synchronization data 166 and the recognition accuracy data 165, and obtains the index values ​​of synchronization (causality) and recognition accuracy from the record data having that graph ID (step S303). Specifically, the index value of synchronization is the transfer entropy 1662 or weighted transfer entropy 1663 held in the synchronization data 166, and the index value of recognition accuracy is the recognition accuracy 1654 held in the recognition accuracy data 165.

[0082] Next, the network display program 153 uses the index values ​​of synchronization and recognition accuracy obtained in step S303 that exceed the display thresholds (moving entropy threshold 1672, recognition accuracy threshold 1673) in the display settings obtained in step S302 to draw a network graph on the analysis results display screen (step S304). When drawing the analysis results display screen, information obtained from management data, etc., in addition to the above index values ​​may be used as appropriate.

[0083] The network display program 153 then outputs the analysis result display screen drawn in the manner described above to the user terminal's display (not shown), thereby providing the user with a network of individuals based on the causality between individuals (transfer entropy in this example) and the accuracy of recognition. The user can then visually recognize the network of individuals in the specified group behavior (target game).

[0084] Figures 16 to 18 show examples of the analysis results display screen (1 to 3).

[0085] The analysis results display screen 410 shown in Figure 16 is an example of the analysis results display screen when the display threshold has not been adjusted by the user. First, the basic screen configuration of the analysis results display screen will be explained with reference to Figure 16. In Figures 17 and 18, which will be discussed later, the explanation of the common screen configuration will be omitted.

[0086] In the analysis results display screen 410, area 411 displays the date the target game was played, and area 412 displays the name of the target game. The contents of areas 411 and 412 are specified in advance by the user. As mentioned above, in step S301 of Figure 15, the network display program 153 can identify the game ID of the corresponding target game by referring to the game management data 163 using the specified date and target game as keys.

[0087] In the analysis results display screen 410, area 413 displays the networks between individuals in the target game. The method of displaying the networks is not particularly limited, but for example, in Figure 16, each player in Team A and Team B is assigned a number, and the networks between players whose index values ​​exceed the display threshold are shown unidirectionally by straight lines with arrows. Specifically, for example, it is shown that player "2" of Team A is related to player "1" of the same Team A and player "8" of the opposing team, Team B, but on the other hand, it can be seen that player 2 is not related to any other players.

[0088] In the analysis results display screen 410, the current display threshold is displayed in area 414. In Figure 16, adjustable sliders are provided for both moving entropy and recognition accuracy. Users can freely adjust the display thresholds for moving entropy and recognition accuracy within a predetermined range by operating the sliders in area 414. When the display threshold is changed, the network display processing shown in Figure 15 is executed according to the changed display threshold, and the display content in area 413 changes.

[0089] In the analysis results display screen 410, a slider bar for adjusting the time series is displayed in area 415. By operating the slider bar in area 415, the user can specify the timing of the network between individuals to be displayed in area 413. If the timing is changed, the settings in the display setting data 167 are updated, and the network display process shown in Figure 15 is executed according to the changed timing, causing the display content in area 413 to change. The network display program 153 may also be configured to perform so-called video display, which continuously changes the network display while automatically advancing the time series in area 413 of the analysis results display screen 410. In that case, the slider bar in area 415 will also move and be displayed in accordance with the progress of the display in area 413.

[0090] The analysis results display screen 420 shown in Figure 17 is an example of the analysis results display screen after the user has adjusted the display threshold in the analysis results display screen 410 shown in Figure 16. Specifically, compared to the analysis results display screen 410 in Figure 16, in the analysis results display screen 420 in Figure 17, the display threshold for recognition accuracy has been adjusted to "strong" in the adjustable display threshold area 424. As a result of raising the display threshold for recognition accuracy, the network displayed in area 423 of the analysis results display screen 420 is a filtered version of the network displayed in area 413 of the analysis results display screen 410, and only networks with more reliable relationships are displayed.

[0091] The analysis result display screen 430 shown in Figure 18 differs from the analysis result display screens 410 and 420 described above in the configuration of the area 434 in which the display threshold can be adjusted.

[0092] In the analysis results display screen 430, area 434 is provided with a slider bar that allows adjustment of the display threshold for an index called "cooperation strength." Cooperation strength is an index that combines the causality between individuals (e.g., transfer entropy) and the accuracy of recognition, and in this example, it specifically corresponds to weighted transfer entropy weighted by the accuracy of recognition. By operating the slider bar in area 434, the user can freely adjust the display threshold for cooperation strength (weighted transfer entropy) within a predetermined range. When the display threshold for cooperation strength is changed, the network display process in Figure 15 is executed according to the changed display threshold, and the display content in area 433 changes. Specifically, in area 433, the network between individuals is displayed using the weighted transfer entropy that exceeds the display threshold for cooperation strength among the synchronization index values ​​obtained in step S303 of Figure 15.

[0093] While the aforementioned analysis result display screens 410 and 420 were configured for experts, allowing adjustment of both causality (transfer entropy) and recognition accuracy, the analysis result display screen 430 allows adjustment of the display threshold for "cooperation strength (weighted transfer entropy)," which weights causality by recognition accuracy. This simplifies operation, improves usability, and makes the display configuration more suitable for general users.

[0094] Furthermore, the types of display thresholds that can be operated in regions 414, 424, and 434 may be configured to be switchable by the user. Alternatively, the network display program 153 may change the display settings so that the types of display thresholds that can be operated differ depending on the permissions granted to the user in advance. In addition, adjustment sliders for the cooperation strength (weighted moving entropy) may be provided in regions 414 and 412.

[0095] As described above, the collective behavior analysis device 100 according to this embodiment can acquire multiple types of sensor data from multiple individuals performing collective behavior by executing a synchronous calculation process, estimate the causality (synchronization) of movement between two individuals that encompass multiple combinations of individuals based on the sensor data (e.g., movement entropy), and further estimate the accuracy of mutual or unidirectional recognition based on the distance or body orientation of the two individuals.

[0096] Furthermore, the group behavior analysis device 100, when displaying the inter-individual network using the estimation results of the synchronous calculation process through network display processing, can switch the display and output of network analysis indicators any number of times according to the user's wishes. Specifically, in the analysis result display screen 410 of Figure 16, the user can specify the group behavior (target game) for which they want to display the inter-individual network by inputting into regions 411 and 412, and can specify the time series to display by adjusting the slider bar in region 435. In addition, by adjusting the display threshold in region 414, the inter-individual network can be displayed based on filtered causality and recognition accuracy. Furthermore, the criteria for determining recognition accuracy can be changed by changing values ​​such as the estimated field of view, which can also be pre-set. In addition, as shown in the analysis result display screen 430 of Figure 18, the inter-individual network can also be displayed based on an index value (weighted moving entropy) in which causality (moving entropy) is weighted by recognition accuracy.

[0097] Thus, according to the group behavior analysis device 1 (group behavior analysis device 100) of this embodiment, the network between individuals in group behavior can be presented in an explanatory manner, making it easier for users to objectively understand the network of group behavior and enabling effective instruction in sports and other activities.

[0098] (2) Second embodiment In the second embodiment, a group behavior analysis system is described that estimates causality and recognition accuracy in the network between individuals and displays the network between individuals, without having each individual performing group behavior (for example, each player in a ball game) wear a wearable device 300.

[0099] Figure 19 is a block diagram showing an example configuration of a group behavior analysis system 2 according to a second embodiment of the present invention. The group behavior analysis system 2 is a system that analyzes the network between individuals in group behavior and presents the analysis results to the user, and as shown in Figure 19, it comprises a group behavior analysis device 500 and a camera 600. Among the components of the group behavior analysis system 2, components that are the same as those of the group behavior analysis system 1 according to the first embodiment are denoted by common reference numerals and their descriptions are omitted. Furthermore, the processing procedures (synchronous calculation processing, network display processing) performed by the group behavior analysis device 500 in the group behavior analysis system 2 are basically the same as those performed by the group behavior analysis device 100 in the first embodiment, and their detailed descriptions are omitted.

[0100] As shown in Figure 19, the group behavior analysis device 500 is connected to the camera 600. The camera 600's shooting range 601 is directed towards the court or other area where the group activity (in this example, a ball game) is taking place, and the camera 600 transmits the captured data of the group activity to the external communication unit 110 of the group behavior analysis device 500, for example, in real time. The captured data received by the external communication unit 110 of the group behavior analysis device 500 is stored in the storage unit 550 as image data 561.

[0101] The group behavior analysis device 500 is a computer such as a personal computer (PC), and has an external communication unit 110, a calculation unit 120, a control unit 130, a memory 140, and a storage unit 550. Each unit is interconnected by an internal communication line (bus).

[0102] The storage unit 550 is an auxiliary storage device that non-temporarily stores programs and data used in the group behavior analysis device 500, and is, for example, an SSD or HDD. Note that at least a portion of the programs and data shown in Figure 19 as being stored in the storage unit 550 may be held externally via a network and read into the memory 140 as needed.

[0103] The memory unit 550 stores the following programs: an image recognition program 551, a synchronization calculation program 552, and a network display program 153. In addition, the memory unit 550 stores the following data as in the first embodiment: user management data 162, game management data 163, graph data 164, recognition accuracy data 165, synchronization data 166, and display setting data 167. Furthermore, it stores image data 561, person coordinate data 562, and ball coordinate data 563.

[0104] The image recognition program 551 performs image analysis on the image data 561 collected from the camera 600, obtains the coordinates of people and balls in the image data 561 in chronological order, and records the obtained coordinates in person coordinate data 562 and ball coordinate data 563. The above processing by the image recognition program 551 is performed after the registration of the image data 561 collected from the camera 600, and before or immediately after the start of the synchronous calculation process.

[0105] Figure 20 shows an example of person coordinate data 562. The person coordinate data 562 stores the location information of a person obtained by image analysis of image data 561 in a time series. Specifically, the person coordinate data 562 shown in Figure 20 consists of the items person ID 5621, date and time 5622, and coordinates 5623 indicating the person's coordinates. The person ID 5621 is an identifier (person ID) assigned to each person identified by image analysis of image data 561, and by identifying the correspondence between each individual indicated by the person ID and the user ID managed by the user management data 162 using a predetermined method, it becomes possible to treat person ID 5621 as user ID 1621.

[0106] Figure 21 shows an example of ball coordinate data 563. The ball coordinate data 563 stores the position information of the ball obtained by image analysis of the image data 561 in a time series. Specifically, the ball coordinate data 563 shown in Figure 21 consists of the following items: game ID 5631, date and time 5632, and coordinates 5633 indicating the ball's coordinates.

[0107] Here, the human coordinate data 562 described above allows us to know the position of each individual (player) in a group activity, but it does not allow us to determine the orientation of the players' bodies. Furthermore, there is no information regarding the field of view of each individual. Therefore, the synchronous calculation program 552 of the second embodiment uses the human coordinate data 562 and ball coordinate data 563 at the same time to estimate the orientation of the players' bodies and their field of view as follows.

[0108] Figure 22 is a diagram illustrating a method for estimating body orientation using ball coordinates. Figure 22 shows an example of the positional relationship between a ball 701 and multiple players 702 during a game. The coordinates of the ball 701 can be obtained from coordinate 5633 of the ball coordinate data 563, and the coordinates of the players 702 can be obtained from coordinate 5623 of the person coordinate data 562. At this time, the synchronous calculation program 552 estimates the orientation of the individual's body and the range of their field of view, assuming that the center of the individual's field of view is at the position (direction) of the ball. Specifically, for player "1" 702, the direction of the arrow is the center of the field of view (i.e., the orientation of the body), and region 703 is the range of the field of view.

[0109] Based on the above estimation, the synchronization calculation program 552 can calculate the position and body orientation of each individual in group behavior. This means the same result as the process in step S102 of the synchronization calculation process in the first embodiment shown in Figure 11. Therefore, the synchronization calculation program 552 in the second embodiment can perform the same synchronization calculation process as in the first embodiment by utilizing the above estimation result. Furthermore, the network display process performed using the results of the synchronization calculation process is the same in the second embodiment as in the first embodiment.

[0110] As described above, the group behavior analysis system 2 (group behavior analysis device 500) according to the second embodiment can analyze the network between individuals in the same way as the first embodiment, without requiring each individual to wear a wearable device 300 as in the first embodiment. This is achieved by estimating the orientation and field of view of each individual based on images captured by the camera 600, using a specific object (a ball in this example) that each individual is paying attention to as a reference, and displaying explanatory analysis results to the user.

[0111] As a variation of the second embodiment, a wearable device 300 may be mounted on the ball. In this case, the group behavior analysis device 500 can be configured to acquire sensor data from the wearable device 300, thereby obtaining more accurate ball coordinates.

[0112] Furthermore, as another modification of the second embodiment, if it is difficult to obtain ball coordinates from image data 561 due to reasons such as the small size of the ball, or if the target game is a sport that does not use a ball, the synchronization calculation program 552 may consider the direction of movement of an individual (player) as the center of their field of view and estimate the orientation of the individual's body and the range of their field of view. In addition, as another estimation method when ball coordinates cannot be obtained, the center of each individual's (player's) field of view may be estimated based on specific situations during group action. Specifically, for example, the ball coordinates may be estimated from the degree of density of multiple players, and the center of each individual's (player's) field of view may be estimated based on these estimated ball coordinates, or the direction of the center of a player's field of view may be estimated based on data indicating situations such as changes in offense and defense during the game.

[0113] (3) Third Embodiment The third embodiment is an embodiment specifically focused on group behavior performed by two individuals, and provides a more precise analysis than the first and second embodiments, by analyzing the networks between the various parts of the two individuals' skeletons.

[0114] Figure 23 is a block diagram showing an example configuration of a group behavior analysis system 3 according to a third embodiment of the present invention. The group behavior analysis system 3 is a system that analyzes the network between individuals in group behavior and presents the analysis results to the user, and as shown in Figure 23, comprises a group behavior analysis device 800 and a camera 600. The camera 600 has the same configuration as the camera 600 described in the second embodiment, but it is preferable that it is capable of capturing images at a higher resolution than the second embodiment. In addition, in the description of the third embodiment, the description of things that are the same as in the first or second embodiment will be omitted.

[0115] The group behavior analysis device 800 is a computer such as a personal computer (PC), and has an external communication unit 110, a calculation unit 120, a control unit 130, a memory 140, and a storage unit 850. Each unit is interconnected by an internal communication line (bus).

[0116] The memory unit 850 is an auxiliary storage device that non-temporarily stores programs and data used in the group behavior analysis device 800, and is, for example, an SSD or HDD. Note that at least a portion of the programs and data shown in Figure 23 as being stored in the memory unit 850 may be held externally via a network and read into the memory 140 as needed.

[0117] The memory unit 850 stores the following programs: an image recognition program 851, a synchronization calculation program 852, and a network display program 153. The memory unit 850 also stores the same data as in the first embodiment: game management data 163, graph data 164, recognition accuracy data 165, synchronization data 166, and display setting data 167. It also stores the same data as in the second embodiment: image data 561, and further, skeletal coordinate data 861 and user management data 862.

[0118] The image recognition program 851 performs image analysis on the image data 561 collected from the camera 600, and for each person pictured in the image data 561, it obtains the coordinates of predetermined parts of the skeleton in a time series, and records the obtained coordinates of each part in the skeleton coordinate data 861. The above processing by the image recognition program 551 is performed after the registration of the image data 561 collected from the camera 600, and before or immediately after the start of the synchronous calculation process.

[0119] Figure 24 is a diagram illustrating the skeletal parts to be analyzed. The multiple parts 901 indicated by circles in Figure 24 represent typical parts of the human skeleton (e.g., joints). The points to be designated as parts 901 can be set in advance. In the image analysis of the image data 561 by the image recognition program 851, the coordinates of each part 901 are analyzed from the human image, and the obtained coordinates of each part are recorded in the skeletal coordinate data 861.

[0120] Figure 25 shows an example of skeletal coordinate data 861. The skeletal coordinate data 861 shown in Figure 25 consists of the following items: skeletal ID 8611, date and time 8612, and part coordinates 8613. The skeletal ID 8611 indicates an identifier (skeletal ID) assigned to each person (a group of skeletons) identified by image recognition. The date and time 8612 indicates the date and time the image analysis was performed by the image recognition program 851. The part coordinates 8613 indicate the coordinates of each part (part 901 in Figure 24) obtained by image recognition. In Figure 25, the right shoulder, right elbow, and left ankle are shown as specific parts, but of course, it is not limited to these parts alone.

[0121] Figure 26 shows an example of user management data 862. The user management data 862 shown in Figure 26 has a record for each target individual and consists of the following items: user ID 8621, skeleton ID 8622, team ID 8623, start date and time 8624, and end date and time 8625. Comparing the user management data 862 with the user management data 162 in Figure 5 of the first embodiment, the sensor ID 1622 has been replaced with the skeleton ID 8622.

[0122] Skeleton ID 8622 indicates the skeleton ID assigned to the skeleton of the person (athlete) indicated by User ID 8621, and corresponds to Skeleton ID 8611 in the skeleton coordinate data 861 in Figure 25. The other items in User Management Data 862 (User ID 8621, Team ID 8623, Start Date and Time 8624, End Date and Time 8625) are the same as the items with the same names in User Management Data 162 in Figure 5, so their explanation is omitted.

[0123] The synchronous calculation program 852, like the synchronous calculation program 152 in the first embodiment and the synchronous calculation program 552 in the second embodiment, uses various data stored in the storage unit 850 to perform the same synchronous calculation processing as in Figure 4. However, in the third embodiment, since the inter-individual network is analyzed with respect to the skeletal parts of two individuals, in the synchronous calculation processing by the synchronous calculation program 852, where it is written as "extensive between two individuals" in Figure 4, it is necessary to read it as "extensive between the skeletal parts of each individual".

[0124] When this synchronous calculation process is performed, the causality of skeletal movement (e.g., movement entropy) and recognition accuracy of the skeletal parts of the two individuals can be comprehensively estimated. Then, using these estimation results, the network display program 153 performs network display processing, allowing the inter-individual network based on the skeletal parts of the two individuals to be displayed on the analysis results display screen.

[0125] Figure 27 shows an example of the display of the analysis results display screen in the third embodiment. The analysis results display screen 910 shown in Figure 27 has a display configuration of areas 911 to 915, but these areas 911 to 915 correspond to areas 411 to 415 of the analysis results display screen 410 shown in Figure 16, so a detailed explanation is omitted.

[0126] Area 911 displays the date the game was played, and area 912 displays the name of the game.

[0127] Region 913 displays the network between individuals in the target game. In Figure 27, the causality (synchronization) between the skeletal parts of two individuals is shown by a line with an arrow based on the transfer entropy. Note that if a line with an arrow is displayed in region 913 from skeletal part X of person A to skeletal part Y of person B, it means that the movement of skeletal part Y of person B caused the movement of skeletal part X of person A, representing the causality (synchronization) in which the movement of person B influenced the movement of person A.

[0128] Area 914 displays display thresholds that can be adjusted by the user. By operating the slider bar in area 914, the user can freely adjust the display thresholds for moving entropy and recognition accuracy within a predetermined range. In Figure 27, moving entropy and recognition accuracy are displayed as adjustable items, but as in the first and second embodiments, weighted moving entropy may also be displayed.

[0129] Area 915 displays a slider bar for adjusting the time series. By manipulating the slider bar in area 915, the user can specify the timing of the inter-individual network displayed in area 913.

[0130] As described above, the group behavior analysis system 3 (group behavior analysis device 800) according to the third embodiment recognizes the skeletons of two individuals from images captured by the camera 600 for group behavior performed by two individuals, and analyzes the network between individuals (between two individuals) with greater accuracy than the first and second embodiments based on the movement and orientation of the skeletons, and can display explanatory analysis results to the user.

[0131] Although the computational processing load increases, the group behavior analysis system 3 according to the third embodiment can also analyze the inter-individual network of group behavior involving three or more individuals by recognizing the coordinates of skeletal parts in each person and analyzing the causality (synchronization) of the skeletal movements of each person.

[0132] Furthermore, while the above description describes obtaining skeletal coordinates from images captured by camera 600, as a modification of the third embodiment, the wearable device 300 described in the first embodiment may be attached to the skeletal parts (each part 901 shown in Figure 24) of each person performing group actions. In this modification, as with the first embodiment, image recognition of the skeletal parts by the image recognition program 851 becomes unnecessary, and the coordinates and movements of each skeletal part can be obtained with high accuracy. [Explanation of symbols]

[0133] 1,2,3 Group Behavior Analysis System 100, 500, 800 Group Behavior Analysis Device 110 External Communications Department 120 Arithmetic section 130 Control Unit 140 memory 150,550,850 storage section 151 Data Reception Program 152,552,852 Synchronous Calculation Programs 153 Network Display Program 161 Sensor Data 162,862 user management data 163 Game Management Data 164 graph data 165 Recognition Accuracy Data 166 Synchronized Data 167 Display Settings Data 200 Wireless communication devices 300 wearable devices 310 Wireless Communication Section 320 Microcontrollers 330 Flash Memory 340 USB communication unit 350 Accelerometer 360 Gyroscope Sensor 370 Geomagnetic Sensor 380 GNSS receiver 410,420,430,910 Analysis result display screen 561 Image Data 562 people coordinate data 563 Ball Coordinate Data 600 Camera 861 Skeletal coordinate data

Claims

1. A group behavior analysis device that analyzes the network between individuals in group behavior involving multiple individuals, A synchronization calculation unit capable of performing the following: a causality estimation process that estimates the causality between individuals in each combination of the multiple individuals from sensor data acquired from the multiple individuals or video data of the multiple individuals; and a recognition accuracy estimation process that estimates the distance between individuals and the orientation of the individuals (the direction of one individual as seen from the other individual) from the sensor data or video data for each combination of the multiple individuals, and estimates the accuracy of recognition between individuals based on the estimated distance between individuals and the orientation of the individuals; A network display unit that displays the network between individuals in the group behavior based on the causality between individuals and the accuracy of recognition in each combination of the plurality of individuals estimated by the synchronization calculation unit, Equipped with, The synchronization calculation unit, in the causality estimation process, uses the inter-individual movement entropy calculated using time-series data of the amount of movement per unit time for each individual as an index value indicating causality between the individuals. The synchronization calculation unit, in the recognition accuracy estimation process, estimates a higher accuracy of recognition of the other individual by the first individual the closer the distance from the first individual to the other individual is, and estimates a higher accuracy of recognition of the other individual by the first individual the closer the direction of the other individual as seen from the first individual is to the center of the first individual's field of view. A group behavior analysis device characterized by the following features.

2. The synchronization calculation unit weights the causality between the individuals for each combination of the plurality of individuals based on the accuracy of the recognition between those individuals. The network display unit displays the network between individuals in the collective behavior based on the weighted causality between the individuals for each combination of the plurality of individuals. The group behavior analysis device according to feature 1.

3. The aforementioned network display unit is Regarding the display parameters related to the causality between individuals and the accuracy of recognition, the system will allow users to adjust the display threshold. The display parameters exceeding the aforementioned display threshold are used to display the network between individuals in the collective behavior. A group behavior analysis device according to claim 1 or 2.

4. The aforementioned network display unit is If the display threshold is changed by an operation from the user after the network of individuals in the group behavior has been displayed, the network of individuals in the group behavior will be redisplayed based on the changed display threshold. The group behavior analysis device according to feature 3.

5. The system further comprises position estimation means for acquiring or estimating the position of a specific object that is expected to attract the attention of multiple individuals in the aforementioned group behavior, The synchronization calculation unit considers the position of the specific object obtained or estimated by the position estimation means as the direction of the center of each individual's field of view, and estimates the orientation of each individual. The group behavior analysis device according to feature 1.

6. The synchronization calculation unit estimates the orientation of the individual by considering the direction of movement of the individual as the direction of the center of the individual's field of view. The group behavior analysis device according to feature 1.

7. The group of individuals performing the aforementioned group behavior are multiple people, The synchronization calculation unit estimates the causality between individuals by estimating the causality between one or more parts of each skeleton of two individuals from sensor data acquired from the multiple individuals or video data of the multiple individuals, and estimates the accuracy of recognition between the two individuals. The network display unit displays the network between the two individuals in the group activity based on the causal relationship between the skeletal parts of the two individuals and the accuracy of the recognition, which are estimated by the synchronization calculation unit. The group behavior analysis device according to feature 1.

8. A group behavior analysis system that analyzes the networks between individuals in group behavior involving multiple individuals, A wearable device with a sensor attached to each of the aforementioned multiple individuals, or a camera for photographing the aforementioned multiple individuals, A group behavior analysis device connected to the wearable device so as to be able to receive sensor data detected by the wearable device's sensor or video data captured by the camera, Equipped with, The aforementioned group behavior analysis device is A synchronization calculation unit capable of performing the following: causality estimation process for estimating the causality between individuals in each combination of the plurality of individuals from the sensor data or the video data; and recognition accuracy estimation process for each combination of the plurality of individuals, estimating the distance between individuals and the orientation of the individuals (the direction of one individual as seen from the other individual) from the sensor data or the video data, and estimating the accuracy of recognition between individuals based on the estimated distance between individuals and the orientation of the individuals; A network display unit that displays the network between individuals in the group behavior based on the causality between individuals and the accuracy of recognition in each combination of the plurality of individuals estimated by the synchronization calculation unit, It has, The synchronization calculation unit, in the causality estimation process, uses the inter-individual movement entropy calculated using time-series data of the amount of movement per unit time for each individual as an index value indicating causality between the individuals. In the recognition accuracy estimation process, the synchronization calculation unit estimates a higher accuracy of recognition of the other individual by the first individual the closer the distance from the first individual to the other individual is, and estimates a higher accuracy of recognition of the other individual by the first individual the closer the direction of the other individual as seen from the first individual is to the center of the first individual's field of view. A group behavior analysis system characterized by the following features.

9. A method for analyzing group behavior using a group behavior analysis device that analyzes the network between individuals in group behavior involving multiple individuals, The group behavior analysis device performs a causality estimation step of estimating the causality between individuals in each combination of the multiple individuals from sensor data acquired from the multiple individuals or video data of the multiple individuals, The group behavior analysis device estimates, for each combination of the plurality of individuals, the distance between individuals and the orientation of the individuals, which is the direction of one individual as seen from the other individual, from the sensor data or the video data, and estimates the accuracy of recognition between individuals based on the estimated distance between individuals and the orientation of the individuals in the recognition accuracy estimation step. The group behavior analysis device includes a network display step that displays the network between individuals in the group behavior based on the causality estimation step and the recognition accuracy estimation step, and the causality between individuals and recognition accuracy in each combination of the plurality of individuals estimated in the causality estimation step and the recognition accuracy estimation step. Equipped with, In the causality estimation step, the group behavior analysis device uses the inter-individual movement entropy calculated using time-series data of the amount of movement per unit time of each individual as an index value indicating causality between the individuals. In the recognition accuracy estimation step, the group behavior analysis device estimates a higher accuracy of recognition by one individual of the other individual the closer the distance from one individual to the other individual is, and estimates a higher accuracy of recognition by one individual of the other individual the closer the direction of the other individual as seen from one individual is to the center of the field of view of one individual. A group behavior analysis method characterized by the following.

Images