Action state determination program, action state determination system, and action state determination method

Abstract

To provide an action state determination program that can cause a computer to execute processing for appropriately determining an action state of a group of persons present in a predetermined space, and further provide an action state determination system and an action state determination method.SOLUTION: An action state determination program for determining a state of a group of persons present in a predetermined space, causes a computer to execute the steps of: dividing a group into a plurality of groups including at least a first group and a second group so as to satisfy the relation expression [1] below; extracting state information for each person in at least the first group and the second group resulting from the division; and determining an action state of the group by comparing the extracted state information with evaluation reference information. The relation expression [1]: 0.23≤x / w≤0.77, where x is the number of persons in the first group, and w is the total number of persons in the space.SELECTED DRAWING: Figure 28

Description

[Technical field]

[0001] The present invention relates to a behavior state determination program, a behavior state determination system, and a behavior state determination method for causing a computer to execute a process for determining the behavior state of a group of people present in a specified space, such as participants in a lecture, class, or lecture meeting. [Background technology]

[0002] Attempts are being made to photograph participants in lectures, classes, and conferences, etc., and then use the images to analyze and evaluate them.

[0003] As one such example, Patent Document 1 discloses the following information processing device. This information processing device extracts actions defined as evaluation targets for predetermined evaluation items based on body parts and their movements detected from video data capturing a lecturer and students. Then, the actions of the lecturer and students are evaluated according to predetermined evaluation criteria. Specifically, the degree to which the lecturer turns to the students during the lecture (the degree to which they pay attention to the students) and the degree to which the students turn to the lecturer during the lecture (the degree to which they pay attention to the lecturer) are evaluated based on the frequency of occurrence and the length of duration. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-49481 A Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, in recent years, traditional lecture styles have been reevaluated, and more modern and advanced lecture styles are becoming popular. Specifically, instead of passively receiving lectures, a style in which students actively take the initiative in the lecture (active learning) has been adopted. More specifically, examples of such lecture styles include group discussions and debates conducted by students in rooms, project-based learning, creative learning (practical training and fieldwork), and sharing what has been learned with multiple people. Furthermore, there are also lecture styles that focus on multiple units rather than individual lectures, forming groups of three to six people to work together, and sharing impressions after watching a video.

[0006] As a result, the style of lectures where the students are actively led is becoming more and more popular. For this reason, there is a demand for a method to properly evaluate the behavioral state of students, such as how much they are engaging with the class.

[0007] However, the evaluation of the degree of interaction between the lecturer and the students shown in Patent Document 1 cannot properly evaluate the degree to which the students and the students are interacting with each other.

[0008] An object of the present invention is to provide a behavioral state determination program that can cause a computer to execute a process for appropriately determining the behavioral state of a group of people present in a specified space, such as a situation in which participants face each other in a lecture, class, or presentation meeting.

[0009] Another object of the present invention is to provide a behavior state determination system and a behavior state determination method that can properly determine the behavior state of a group of people present in a specified space. [Means for solving the problem]

[0010] The above object can be achieved by the following means. (1) A behavior state determination program for determining a behavior state of a group of people present in a predetermined space, A division step of dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational formula [1]; an extraction step of extracting status information of each person in at least the first group and the second group divided by the division step; a determination step of determining a behavioral state of the group by comparing the state information extracted by the extraction step with evaluation criterion information; A behavioral state determination program that causes a computer to execute the above steps.

[0011] 0.23≦x / w≦0.77 [1] Where x is the number of people in the first group w: total number of people in the space (2) The behavioral state determination program according to the preceding paragraph 1, wherein in the dividing step, the group of people lined up facing forward is divided in the left-right direction of the space. (3) The behavioral state determination program according to the preceding paragraph 1 or 2, wherein the state information is information on any one of a person's facial direction, a person's body direction, and a person's gaze direction. (4) The behavioral state determination program according to the preceding paragraph 3, wherein the direction of the person's face, the direction of the person's body, and the direction of the person's gaze are either forward, backward, left, or right. (5) A selection step of selecting a boundary group in a boundary region between adjacent groups obtained by the division step so as to satisfy the following relational expression [2], In the extraction step, status information of each person in the border group is extracted; 3. The behavioral state determination program according to claim 1 or 2, wherein in the determination step, the behavioral state of the group is determined taking into consideration the state information of each person in the boundary group extracted in the extraction step.

[0012] 0.33≦m / w≦0.75 [2] where m is the number of people in the border group w: total number of people in the space (6) A behavioral state determination program according to the preceding paragraph 1 or 2, in which position information of a person in the space is acquired from an image captured by a photographing device or from a position information acquisition device. (7) The behavioral state determination program according to the preceding paragraph 1 or 2, wherein the total number of people in the space is six or more. (8) The behavioral state determination program according to paragraph 1 or 2, wherein the evaluation criteria information is information set at an arbitrary threshold value for the minimum value when comparing the number or percentage of people in one direction and the number or percentage of people in the opposite direction. (9) A behavioral state determination program according to any one of the preceding paragraphs 1 to 2, in which, in the determination step, state information of people belonging to each group, or information on the ratio of the number of people in each posture to the total number of people, is machine-learned to determine the behavioral state of the groups. (10) The behavioral state determination program according to the preceding paragraph 1 or 2, wherein the determination step determines the behavioral state of the group at predetermined time intervals. (11) The state determination program according to the preceding paragraph 1 or 2, further causing the computer to execute an output step of outputting the result of the behavioral state of the group determined in the determination step. (12) A behavior state determination system for determining a behavior state of a group of people present in a predetermined space, comprising: A division means for dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational expression [1]; an extraction means for extracting status information of each person in at least the first group and the second group divided by the division means; a determination means for determining a state of the group by comparing the state information extracted by the extraction means with evaluation criterion information; A behavioral state determination system comprising:

[0013] 0.23≦X / W≦0.77 [1] Where, X is the number of people in the first group. W: Total number of people in the space (13) A behavioral state determination method for determining a behavioral state of a group of people present in a predetermined space, the method being executed in a behavioral state determination system, A division step of dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational expression [1]; an extraction step of extracting status information of each person in at least the first group and the second group divided by the division step; a determination step of determining a state of the group by comparing the state information extracted by the extraction step with evaluation criterion information; A behavioral state determination method comprising:

[0014] 0.23≦X / W≦0.77 [1] Where, X is the number of people in the first group. W: Total number of people in the space Effect of the Invention

[0015] According to the behavioral state determination program of the present invention, a computer divides a group into a plurality of groups including at least a first group and a second group so as to satisfy the relational expression [1]. The computer also extracts state information of each person in at least the first group and the second group. The computer then compares the extracted state information with the evaluation criterion information to determine the group state. This allows the computer to properly determine the behavioral state of a group of people present in a specified space.

[0016] According to the behavioral state determination system and behavioral state determination method of the present invention, a group is divided into a plurality of groups including at least a first group and a second group so as to satisfy the relational expression [1]. Furthermore, state information of each person in at least the first group and the second group is extracted. Then, the extracted state information is compared with the evaluation criterion information to determine the behavioral state of the group. Therefore, the behavioral state of a group of people existing in a specified space can be appropriately determined. [Brief description of the drawings]

[0017] [Figure 1] 1 is a configuration diagram of a behavior state determination system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing examples A to L of a virtual group W of students. [Diagram 3] FIG. 13 is a diagram showing examples P to U of a group W of virtual students. [Figure 4] FIG. 4 is a diagram showing the state in which the groups illustrated in FIG. 2 and FIG. 3 are divided in a different manner. [Diagram 5] FIG. 4 is a diagram showing the state in which the groups illustrated in FIG. 2 and FIG. 3 are divided in a different manner. [Figure 6] FIG. 13 is a diagram illustrating an example of evaluation criteria. [Figure 7] FIG. 13 is a diagram showing another example of the evaluation criteria. [Figure 8] 1 is a table showing the results of Example 1. [Figure 9] 1 is a table showing the results of Example 2. [Figure 10] 1 is a table showing the results of Example 3. [Figure 11] 1 is a table showing the results of Comparative Example 1. [Figure 12] 1 is a table showing the results of Example 4. [Figure 13] 1 is a table showing the results of Comparative Example 2. [Figure 14] 1 is a table showing the results of Example 5. [Figure 15] 1 is a table showing the results of Example 6. [Figure 16] 1 is a table showing the results of Example 7. [Figure 17] 1 is a table showing the results of Example 8. [Figure 18] 1 is a table showing the results of Example 9. [Figure 19] 1 is a table showing the results of Example 10. [Figure 20] 13 is a table showing the results of Comparative Example 3. [Figure 21] 1 is a table showing the results of Example 11. [Figure 22] 1 is a table showing the results of Comparative Example 4. [Figure 23] 1 is a table showing the results of Example 12. [Figure 24] 1 is a table showing the results of Example 13. [Diagram 25] 1 is a table showing the results of Example 14. [Figure 26] 11 is a table showing the results of Comparative Example 5. [Figure 27] 1 is a table showing the results of Comparative Example 6. [Figure 28] 1 is a table summarizing the results of Examples and Comparative Examples. [Figure 29] 11 is a table showing output results when state determination is performed at regular time intervals. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0019] 1 is a configuration diagram of a behavioral state determination system according to an embodiment of the present invention. The behavioral state determination system includes an information processing device 1 and an imaging device 2.

[0020] The information processing device 1 is a personal computer. The information processing device 1 includes a CPU 11, a RAM 12, a storage unit 13, a display device 14, an input device 15, a network interface unit (network I / F unit) 16, etc. These are connected to each other via a system bus 17.

[0021] The CPU 11 executes an operating program stored in the storage unit 13 or the like and deployed in the RAM 13 to centrally control the entire behavioral state determination system. In particular, in this embodiment, the CPU 11 acquires image data transmitted from the photographing device 2 and determines the behavioral state of a predetermined group. The behavioral state determination process will be described later.

[0022] The RAM 12 is a storage medium that provides a working area when the CPU 11 operates according to an operating program.

[0023] The storage unit 13 is composed of a storage medium such as a hard disk. The storage unit 13 stores the operation program of the CPU 11, various management data, application programs, etc. In this embodiment, the behavioral state information of the group determined by the information processing device 1 is stored in chronological order.

[0024] The display device 14 is composed of a CRT, a liquid crystal display device, etc. The display device 14 displays various messages, an input acceptance screen for the user, a selection screen, etc.

[0025] The input device 15 is used for input operations by the user, and includes a keyboard, a mouse, and the like.

[0026] The network interface unit 16 functions as a communication means for transmitting and receiving data between the image capturing apparatus 2 and other external devices via a network.

[0027] The image capturing device 2 is a camera that captures images of a group W of participants in an indoor or outdoor space during a lecture, class, or lecture meeting, etc., and captures the participants or lecturer in chronological order.

[0028] Image data (video) captured by the imaging device 2 is transmitted to the information processing device 1. The information processing device 1 receives the image data transmitted from the imaging device 2. The CPU 11 of the information processing device 1 analyzes the image data and identifies the number and positions of participants in two-dimensional coordinates. The CPU 11 further extracts at least one of the body direction, face direction, and line of sight direction of each participant as status information, and performs analysis, evaluation, etc. as described below. The imaging device 2 may be a single device or multiple devices.

[0029] In this embodiment, the students are assumed to be facing the lecturer on the podium. The lecturer's side is the front, the students' backs are the back, the right side (left side in the figure) from the students' perspective is the right, and the left side (right side in the figure) is the left. Therefore, when the students are facing the lecturer, they are facing forward. Also, during a lecture, there are situations where the students on the left and right face each other and have a discussion. In such cases, the students on the left and right often face each other in the left and right directions. Also, when one of the students speaks, the other student often faces the speaker.

[0030] In this embodiment, when a large proportion of students in group W are facing each other on the left and right, the behavioral state of group W is called a "consultation state." On the other hand, when a large proportion of students are facing the direction of one speaker, the behavioral state of group W is called a "speaking state."

[0031] Accurately grasping whether the behavioral state of group W is in a "consultation state" is extremely important in evaluating the behavior of the lecturer and the students.

[0032] The position information and number of participants may be obtained by other methods than the two-dimensional coordinates of the image data of the photographing device 2. For example, the position information may be obtained by two-dimensional coordinates using a position information acquisition device such as a Global Positioning System (GPS). Each participant may wear a device capable of detecting the direction of the participant's body, face, and gaze, and the direction of the participant's body, face, and gaze may be detected based on data transmitted from the device. The speaker may be identified by extracting speaker characteristics, such as holding a microphone or standing, from the image data. Alternatively, the speaker may be identified by having each participant hold a microphone.

[0033] In this embodiment, when the left and right students face each other, it is assumed that the direction of the students' bodies, faces, and gazes are all directed toward the student on the other side. Therefore, whether the left and right students face each other is determined based on whether the direction of the students' bodies is directed toward the student on the other side. However, there are cases where only the faces and gazes are directed toward the other side, and the bodies are not. There are also cases where only the gazes are directed toward the other side. Therefore, it may be possible to determine whether the students are facing each other by judging the direction of the faces and gazes, without judging the direction of the bodies. It may be determined from at least one of the direction of the bodies, the direction of the faces, and the direction of the gazes.

[0034] Next, a behavior state determination process for determining whether the behavior state of group W is in a "consultation state" will be described.

[0035] This behavior state determination process is executed by the CPU 11 of the information processing device 1 operating in accordance with a behavior state determination program, which is an operating program. The behavior state determination program may be stored in advance in a recording medium such as the storage unit 13. Alternatively, the behavior state determination program stored in a CD-ROM or other portable recording medium may be installed in the information processing device. Alternatively, the behavior state determination program may be downloaded from a server or the like via a network and installed.

[0036] The behavior state determination process is performed in the order of division process, extraction process, and determination process. (1) Division processing A group of participants in an indoor or outdoor space is divided into multiple groups including at least a first group and a second group so as to satisfy the following relational expression [1].

[0037] 0.23≦x / w≦0.77 [1] Here, x is the number of people in the first group, and w is the total number of people in the space, i.e., the total number of participants.

[0038] The boundary between the first and second groups when dividing the group may be formed by a straight line or a curved line. Taking into consideration the shape of the group and the arrangement of people, desks, chairs, etc., an inflection point may be provided midway along the boundary.

[0039] Note that the first group X when group W is divided into multiple groups is not limited. Any group among the multiple divided groups may be set as the first group X. Therefore, when dividing into two groups, for example, a state in which the group is divided so that x / w is 0.25 is the same as a state in which the group is divided so that x / w is 0.75.

[0040] By dividing the data into multiple groups so as to satisfy the relational expression [1], the accuracy of the judgment can be improved. This will be described later.

[0041] In order to further improve the accuracy of the determination, the boundary group may be selected so as to satisfy the following relational expression [2] in the boundary region between adjacent groups when the group is divided.

[0042] 0.33≦m / w≦0.75 [2] Here, m is the number of people in the boundary group, and w is the total number of people in the space. (2) Extraction process Next, state information of each person in each divided group is extracted. In this example, the state information is the orientation of the person.

[0043] If the border group M is selected, status information of each person is extracted from the border group M in addition to the divided groups. (3) Judgment process Next, the extracted state information of the students is compared with preset evaluation criteria information to determine the behavioral state of the group, in other words, whether it is in a "consultation state" or not.

[0044] When a borderline group is selected, the state of the group is determined taking into consideration the state information of the students in the borderline group as well.

[0045] In order to verify the above processes (1) to (3), several specific judgment processes shown below were carried out using a group W of multiple virtual students.

[0046] FIG. 2 shows examples A to L of a group W of virtual students, and FIG. 3 shows examples P to U of the group W of virtual students.

[0047] In each group W, one black dot represents one student, and the students are arranged vertically and horizontally. In each example in Figure 2, the number of students w in each group W is 24, and in each example in Figure 3, the number of students w in each group W is 6.

[0048] In each group W in Figures 2 and 3, students facing forward or backward and students facing left or right are distinguished by two types of black marks. The arrows accompanying each student indicate the direction of the student. Therefore, in example A in Figure 2, there are three students facing right, three students facing left, 18 students facing forward, and no students facing backward. In other examples B to L in Figure 2 and examples M to S in Figure 3, the number and positions of students facing each direction are set appropriately.

[0049] In addition, in examples G to L of FIG. 2 and examples T and U of FIG. 3, the students surrounded by a rectangular frame are speakers. That is, in examples G to L of FIG. 2 and examples T and U of FIG. 3, the behavioral state of group W is classified as "speaking state". In examples B, C, D, and E of FIG. 2 and examples Q, R, and S of FIG. 3, the proportion of students facing each other in the left and right direction is large. Therefore, the behavioral state of group W is classified as "consulting state". In examples A and F of FIG. 2 and example P of FIG. 3, the proportion of students facing each other in the left and right direction is small, and there is no speaker. Therefore, the behavioral state of group W is classified as "other". In this embodiment, group W, which is in the "speaking state", is also classified as "other".

[0050] Next, consider a situation in which the group W of all students is divided into multiple groups. [Example 1] In this Example 1, as shown in Fig. 4 (Example 1), a boundary line was set in the front-rear direction at the center in the left-right direction, and group W was divided equally into a first group X located on the right side (left side in the drawing) and a second group located on the left side (right side in the drawing). There were 12 students belonging to each of the first group X and the second group, and the number ratio of the first group X to group W was 0.5.

[0051] In the example of FIG. 4, the group W shown in example A of FIG. 2 is illustrated, but each of the groups W in examples B to L of FIG. 2 is also divided in the same manner.

[0052] Then, for each group W in examples A to L in Figure 2, we investigated the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left-right pair ratio for group W, the first group X, and the second group. The results are shown in the table in Figure 8.

[0053] The leftmost item in the table in Fig. 8, "Behavioral state 1 (example A)," indicates the values ​​examined for group W, the first group, and the second group shown in example A in Fig. 2. The same applies to "Behavioral state 2 (example B)" and onwards.

[0054] The left-right pair ratio is the proportion of students facing left and right, and was calculated using the following formula [2].

[0055] Left-right pair ratio = MIN (number of people facing left, number of people facing right) × 2 / total number of people in all directions [2] In formula [2], "MIN(number of people facing left, number of people facing right)" is the smaller of the number of people facing left or right in the group. "Total number of people in all directions" is equal to the total number of students in the group.

[0056] Next, based on the left-right pair ratios calculated for group W, the first group X, and the second group, we determined whether the behavioral state of group W was a "consultation state" or "other" (including a "speaking state") other than a "consultation state." The determination was made according to evaluation criterion 1 shown in the table in Figure 6.

[0057] Evaluation criterion 1 in Figure 6 is to determine whether the left-right pair ratio is less than 0.4 or greater than 0.4 for each of group W, first group X, and second group, and to determine whether the state is a “consultation state” (simply marked as “consultation” in the table) or “other” based on the combination of the judgment results.

[0058] The judgment results for each group W are also shown in the table in Figure 8. The "Judgment" item in the table in Figure 8 is the judgment result determined based on the left-right pair ratio and evaluation criterion 1 in Figure 6. The "Correct Answer" item is the actual behavioral state for each group W in Examples A to L in Figure 2. As mentioned above, each group W in Examples B, C, D, and E in Figure 2 is classified as a "consultation state," and each group W in Examples A and F to L in Figure 2 is classified as "other."

[0059] From a comparison of the "Judgment" and "Correct" items in the table of FIG. 8, the overall correct answer rate and the utterance state correct answer rate were calculated and listed in the table. The overall correct answer rate indicates the proportion of correct judgments for each group W from behavior state 1 (example A) to behavior state 12 (example L in FIG. 2). In this Example 1, the judgment results for behavior state 4 (example D in FIG. 2) and behavior state 6 (example F in FIG. 2) differ from the actual behavior states. For this reason, the overall correct answer rate was 10÷12 ≒ 83%.

[0060] The utterance state correct answer rate indicates the proportion of correctly judged utterance states for each of the six groups W of "utterance states" shown in Figures 2G to 2L. In this Example 1, all were judged as "other," which matched the actual state, and there were no erroneous judgments, so the utterance state correct answer rate was 100%. [Example 2] In Example 2, as shown in FIG. 4 (Example 2), a boundary line was set in the left-right direction so that the ratio of the number of people in the first group X to the number of people in group W was 0.5, and group W was divided in the front-back direction into the first group X and the second group.

[0061] FIG. 4 (Example 2) illustrates the group W shown in Example A of FIG. 2, but each of the groups W in Examples B to L of FIG. 2 was divided in the same manner.

[0062] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 9.

[0063] Next, similarly to Example 1, from the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was a "consultation state" or "other" (including a "speaking state") other than the "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 9.

[0064] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 9, the judgment results for behavior state 7 (Figure 2 Example G) and behavior state 10 (Figure 2 Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 83%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 3] In this Example 3, as shown in Fig. 4 (Example 3), a boundary line is set in the front-rear direction so that the ratio of the number of people in the first group to the number of people in the group W is 0.33, and the group W is divided into the first group and the second group in the left-right direction. Note that the first group and the second group may be interchanged, and the ratio of the number of people in the first group X to the number of people in the group W may be 0.67.

[0065] In the example of FIG. 4 (Example 3), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0066] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 10.

[0067] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 10.

[0068] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 10, the judgment result for behavior state 6 (Example F) differs from the actual behavior state. For this reason, the overall correct answer rate was 92%. In addition, for group W in the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, all judgment results were "Other," and there were no erroneous judgments, so the utterance state correct answer rate was 100%. [Comparative Example 1] In this comparative example 1, as shown in Figure 4 (Comparative Example 1), a boundary line was set in the front-to-back direction so that the ratio of the number of people in the first group to group W was 0.17, and group W was divided in the left-to-right direction into a first group and a second group.

[0069] In the example of FIG. 4 (Comparative Example 1), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0070] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 11.

[0071] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 11.

[0072] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 11, the judgment results for behavior state 3 (Example C), behavior state 4 (Example D), behavior state 5 (Example E), behavior state 7 (Example G), and behavior state 10 (Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 58%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 4] The fourth embodiment is an example in which state judgment is performed on a group W of six students as shown in FIG.

[0073] As shown in Figure 4 (Example 4), a boundary line was set in the front-to-back direction so that the ratio of the number of people in the first group to the number of people in group W of 6 was 0.33, and group W was divided in the left-to-right direction into the first group and the second group.

[0074] In the example of FIG. 4 (Example 4), the group W shown in Example P of FIG. 3 is illustrated, but each of the groups W in Examples Q to U of FIG. 3 was also divided in the same manner.

[0075] Then, similarly to the first embodiment, for each group W in examples P to U in Fig. 3, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 12.

[0076] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 12.

[0077] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 12, the judgment result for behavior state 3 (Example R) differs from the actual behavior state. For this reason, the overall correct answer rate was 83%. In addition, for the two "utterance states" group W shown in Figure 3 Example T and Figure 3 Example U, all were judged as "Other," and there were no erroneous judgments, so the utterance state correct answer rate was 100%. [Comparative Example 2] In Comparative Example 2, as shown in Figure 4 (Comparative Example 2), a boundary line having an inflection point in the front-to-back direction was set so that the ratio of the number of people in the first group to group W was 0.21, and group W was divided in the left-to-right direction into a first group X and a second group.

[0078] In the example of FIG. 4 (Comparative Example 2), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0079] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 13.

[0080] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 13.

[0081] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 13, the judgment results for behavior state 3 (Example C), behavior state 5 (Example E), behavior state 6 (Example F), behavior state 7 (Example G), and behavior state 10 (Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 58%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 5] In this Example 5, as shown in Figure 5 (Example 5), a boundary line having an inflection point in the front-to-back direction was set so that the ratio of the number of people in the first group to group W was 0.5, and group W was divided in the left-to-right direction into a first group and a second group.

[0082] In the example of FIG. 5 (Example 5), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0083] Then, similarly to Example 1, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were investigated for group W, the divided first group X, and the second group. The results are shown in the table of FIG.

[0084] Next, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the left / right pair ratios obtained for group W, the first group X, and the second group were used to determine whether the behavior state of that group W was a "consultation state" or "other" other than a "consultation state" according to Evaluation Criteria 1 in Fig. 6. The determination results are also shown in the table in Fig. 14.

[0085] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 14, the judgment result for behavior state 6 (Example F) differs from the actual behavior state. For this reason, the overall correct answer rate was 92%. In addition, for group W in the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, all judgment results were "Other," and since there were no erroneous judgments, the utterance state correct answer rate was 100%. [Example 6] In this Example 6, as shown in Figure 5 (Example 6), a boundary line having an inflection point in the front-to-back direction was set so that the ratio of the number of people in the first group to group W was 0.25, and group W was divided in the left-to-right direction into a first group and a second group.

[0086] In the example of FIG. 5 (Example 6), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0087] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 15.

[0088] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was in a "consultation state" or was in "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 15.

[0089] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 15, the judgment result for behavior state 7 (Example G) differs from the actual behavior state. For this reason, the overall correct answer rate was 92%. In addition, for group W of the "utterance state" shown in Examples G to L of Figure 2, the correct answer rate was 83% because there was one incorrect judgment. [Example 7] In this Example 7, as shown in Figure 5 (Example 7), a boundary line having an inflection point in the front-to-back direction was set so that the ratio of the number of people in the first group to group W was 0.54, and group W was divided in the left-to-right direction into a first group and a second group.

[0090] In the example of FIG. 5 (Example 7), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner.

[0091] Then, similarly to Example 1, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the divided first group X, and the second group. The results are shown in the table in Fig. 16.

[0092] Next, similarly to Example 1, based on the left / right pair ratios calculated for group W, first group X, and second group, it was determined whether the behavior state of group W was in a "consultation state" or was in "other" other than a "consultation state" according to evaluation criterion 1 in Fig. 6. The determination results are also shown in the table in Fig. 16.

[0093] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 16, the judgment results for behavioral state 6 (Example F) and behavioral state 12 (Example L) differ from the actual behavioral states. For this reason, the overall correct answer rate was 83%. In addition, for group W of the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, the correct answer rate was 83% because there was one incorrect judgment. [Example 8] In this Example 8, as shown in Fig. 4 (Example 8), a boundary line (the same as in Example 1) was set in the front-back direction so that the ratio of the number of people in the first group to the number of people in group W was 0.5, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the border group M. In Fig. 4 (Example 8), the group shown in gray is border group M.

[0094] In the example of FIG. 4 (Example 8), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0095] Then, for each group W in examples A to L in Figure 2, we investigated the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left-right pair ratio for group W, the first group X, the second group, and the border group M. The results are shown in the table in Figure 17.

[0096] Next, based on the left-right pair ratios calculated for group W, the first group X, the second group, and the border group M, we determined whether the behavioral state of group W was a “consultation state” or “other” other than a “consultation state” in accordance with evaluation criterion 2 in Figure 7.

[0097] Evaluation criterion 2 in Figure 7 is to determine whether the left-right pair ratio is less than 0.4 or greater than 0.4 for each of group W, first group X, second group, and border group M, and to determine whether the group is “consultation” or “other” based on the combination of the judgment results.

[0098] The results of the determination are also shown in the table of FIG.

[0099] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 17, only the judgment result for behavior state 6 (Example F) differed from the actual behavior state. Therefore, the overall correct answer rate was 92%. In addition, for group W in the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, all were judged as "Other," and there were no erroneous judgments, so the utterance state correct answer rate was 100%. [Example 9] In this Example 9, as shown in Fig. 4 (Example 9), a boundary line (the same as in Example 2) was set in the left-right direction so that the ratio of the number of people in the first group X to the number of people in the group W was 0.5, and the group W was divided in the front-back direction into the first group X and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the boundary group M. In Fig. 4 (Example 9), the group shown in gray is the boundary group M.

[0100] In the example of FIG. 4 (Example 9), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0101] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 18.

[0102] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 18.

[0103] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 18, the judgment results for behavior state 7 (Example G) and behavior state 10 (Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 83%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 10] In this Example 10, as shown in Fig. 4 (Example 10), a boundary line (the same as in Example 2) was set in the front-back direction so that the ratio of the number of people in the first group to the number of people in group W was 0.33, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the border group M. In Fig. 4 (Example 10), the group shown in gray is the border group M.

[0104] In the example of FIG. 4 (Example 10), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0105] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 19.

[0106] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was in a "consultation state" or in another state other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 19.

[0107] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 19, the judgment result differed from the actual behavior state only for behavior state 6 (Example F). Therefore, the overall correct answer rate was 92%. Furthermore, for group W in the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were no erroneous judgments and the correct answer rate was 100%. [Comparative Example 3] In this Comparative Example 3, as shown in Fig. 4 (Comparative Example 3), a boundary line (the same boundary line as in Comparative Example 1) was set in the front-back direction so that the ratio of the number of people in the first group to group W was 0.17, and group W was divided in the left-right direction into a first group and a second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select border group M. In Fig. 4 (Comparative Example 3), the group shown in gray is border group M.

[0108] In the example of FIG. 4 (Comparative Example 3), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0109] Then, similarly to Example 8, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were investigated for group W, first group X, second group, and border group M. The results are shown in the table in FIG.

[0110] Next, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the left / right pair ratios obtained for group W, the first group X, the second group, and the border group M were used to determine whether the behavior state of that group W was a "consultation state" or "other" other than a "consultation state" according to Evaluation Criteria 2 in Fig. 7. The results of the determination are also shown in the table in Fig. 20.

[0111] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 20, the judgment results for behavior state 4 (Example D), behavior state 6 (Example F), behavior state 7 (Example G), and behavior state 10 (Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 67%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 11] In this Example 11, a border group M was selected from a group W of six students, and a state judgment was performed.

[0112] As shown in Fig. 4 (Example 11), a boundary line (the same as in Example 4) was set in the front-back direction so that the ratio of the number of people in the first group to the number of people in group W of six people was 0.33, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from first group X and second group to select border group M. In Fig. 4 (Example 11), the group shown in gray is border group M.

[0113] In the example of FIG. 4 (Example 11), the group W shown in Example P of FIG. 3 is illustrated, but each group W in Examples Q to U of FIG. 3 was also divided in the same manner and a boundary group M was selected.

[0114] Then, similarly to Example 8, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were investigated for group W, first group X, second group, and border group M. The results are shown in the table in FIG.

[0115] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 21.

[0116] As can be seen from a comparison of the "Judgment" and "Correct" items in the table of FIG. 21, there were no erroneous judgments, and both the overall correct answer rate and the utterance state correct answer rate were 100%. [Comparative Example 4] In this comparative example 4, as shown in Fig. 4 (Comparative Example 4), a boundary line (the same boundary line as in Comparative Example 2) having an inflection point in the front-back direction was set so that the ratio of the number of people in the first group to the number of people in group W was 0.21, and group W was divided in the left-right direction into a first group X and a second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select a boundary group M. In Fig. 4 (Comparative Example 4), the group shown in gray is boundary group M.

[0117] In the example of FIG. 4 (Comparative Example 4), the group W shown in Example A of FIG. 2 is illustrated, but each of the groups W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0118] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 22.

[0119] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 22.

[0120] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 22, the judgment results for behavioral state 6 (Example F), behavioral state 7 (Example G), and behavioral state 10 (Example J) differ from the actual behavioral states. For this reason, the overall correct answer rate was 75%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Example 12] In Example 12, as shown in Fig. 5 (Example 12), a boundary line (the same as that in Example 5) having an inflection point in the front-back direction was set so that the ratio of the number of people in the first group to the number of people in group W was 0.5, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the boundary group M. In Fig. 5 (Example 12), the group shown in gray is the boundary group M.

[0121] In the example of FIG. 5 (Example 12), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0122] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 23.

[0123] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 23.

[0124] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 23, the judgment result differed from the actual behavior state only for behavior state 6 (Example F). Therefore, the overall correct answer rate was 92%. Furthermore, for group W in the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were no erroneous judgments and the utterance state correct answer rate was 100%. [Example 13] In Example 13, as shown in Fig. 5 (Example 13), a boundary line (the same boundary line as in Example 6) having an inflection point in the front-back direction was set so that the ratio of the number of people in the first group to the number of people in group W was 0.25, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the boundary group M. In Fig. 5 (Example 13), the group shown in gray is the boundary group M.

[0125] In the example of FIG. 5 (Example 13), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0126] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 24.

[0127] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 24.

[0128] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 24, the judgment result differed from the actual behavior state only for behavior state 7 (Example G). Therefore, the overall correct answer rate was 92%. Also, for group W of the "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there was one incorrect judgment, so the utterance state correct answer rate was 83%. [Example 14] In Example 14, as shown in Fig. 5 (Example 14), a boundary line (the same as in Example 7) having an inflection point in the front-back direction was set so that the ratio of the number of people in the first group to the number of people in group W was 0.54, and group W was divided in the left-right direction into the first group and the second group. Furthermore, people closest to the set boundary line were extracted from the first group X and the second group to select the boundary group M. In Fig. 5 (Example 14), the group shown in gray is the boundary group M.

[0129] In the example of FIG. 5 (Example 14), the group W shown in Example A of FIG. 2 is illustrated, but each group W in Examples B to L of FIG. 2 was also divided in the same manner and a boundary group M was selected.

[0130] Then, similarly to Example 8, for each group W in Examples A to L in Fig. 2, the number of students belonging to each group, the number of students in each direction (front, back, left, right), and the left / right pair ratio were examined for group W, the first group X, the second group, and the border group M. The results are shown in the table in Fig. 25.

[0131] Next, similarly to Example 8, based on the left / right pair ratios calculated for group W, first group X, second group, and border group M, it was determined whether the behavior state of group W was a "consultation state" or "other" other than a "consultation state" according to evaluation criterion 2 in Fig. 7. The determination results are also shown in the table in Fig. 25.

[0132] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 25, the judgment results differed from the actual behavioral states for behavioral state 6 (Example F), behavioral state 7 (Example G), and behavioral state 10 (Example J). For this reason, the overall correct answer rate was 75%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Comparative Example 5] In this Comparative Example 5, the number of students belonging to group W, the number of students in each direction (front, back, left, right), and the left / right pair ratio were investigated in the same manner as in Example 1, without dividing group W of 24 students shown in Examples A to L in Fig. 2. The results are shown in the table in Fig. 26.

[0133] Next, similarly to Example 1, for each group W in Examples A to L in FIG. 2, it was judged from the left-right pair ratio whether the behavior state of the group W was "consultation state" or "other" other than "consultation state" according to Evaluation Criteria 1 in FIG. 6. In applying Evaluation Criteria 1, the left-right pair ratio of the first group X was set to the same value as the left-right pair ratio of group W, and the left-right pair ratio of the second group was set to 0. The judgment results are also shown in the table in FIG. 26.

[0134] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 26, the judgment results for behavior state 6 (Example F), behavior state 7 (Example G), and behavior state 10 (Example J) differ from the actual behavior states. For this reason, the overall correct answer rate was 75%. In addition, for group W of "utterance state" shown in Figure 2 Example G to Figure 2 Example L, there were two incorrect judgments, so the utterance state correct answer rate was 67%. [Comparative Example 6] In this Comparative Example 6, the group W of six students shown in Examples P to U in Fig. 3 was not divided, and the number of students belonging to group W, the number of students in each direction (front, back, left, right), and the left / right pair ratio were investigated in the same manner as in Example 1. The results are shown in the table in Fig. 27.

[0135] Next, similarly to Example 1, for each group W in Examples P to U of FIG. 3, it was judged from the left-right pair ratio whether the behavior state of the group W was "consultation state" or "other" other than "consultation state" according to Evaluation Criteria 1 of FIG. 6. In applying Evaluation Criteria 1, the left-right pair ratio of the first group X was set to the same value as the left-right pair ratio of group W, and the left-right pair ratio of the second group was set to 0. The judgment results are also shown in the table of FIG. 27.

[0136] As can be seen from a comparison of the "Judgment" and "Correct" items in the table in Figure 27, the judgment result for behavior state 6 (example U) differs from the actual behavior state. For this reason, the overall correct answer rate was 83%. Also, for group W of the "utterance state" shown in examples T and U in Figure 3, there was one incorrect judgment, so the utterance state correct answer rate was 50%.

[0137] A table summarizing the results of the above-mentioned Examples and Comparative Examples is shown in Fig. 28. The symbols D, C, B, and A shown in the judgment items in Fig. 28 indicate that the judgment accuracy improves in this order, and therefore A indicates that it has the best accuracy. [Evaluation when boundary group M is not set] The reference values ​​are an overall correct answer rate of 75% and a speech state correct answer rate of 67% in Comparative Example 5 in which group X of 24 people is not divided, and an overall correct answer rate of 83% and a speech state correct answer rate of 50% in Comparative Example 6 in which group X of 6 people is not divided.

[0138] As can be seen from FIG. 28, in Examples 1 to 7 where the ratio x / w of the number of people in the first group X to the number of people in the group W was 0.23≦x / w≦0.77, both the overall correct answer rate and the utterance state correct answer rate exceeded the reference value, or even if one of the overall correct answer rate and the utterance state correct answer rate was equal to the reference value, the other exceeded the reference value, confirming that the behavioral state was judged with high accuracy.

[0139] Moreover, in Example 2, in which the first and second groups are divided in the front-rear direction, the determination accuracy is inferior to that in Example 1, and Examples 3 to 7, in which the first and second groups are divided in the left-right direction. Therefore, although the first and second groups may be divided in the front-rear direction, it is found that dividing them in the left-right direction is more preferable.

[0140] It can also be seen that highly accurate judgment can be made by preferably setting the total number of people in group W to six or more.

[0141] In this way, by dividing group X so that the ratio of the number of people in the first group X to the number of people in group W (x / w) is 0.23≦x / w≦0.77, the accuracy of determining the behavioral state of group X is improved. This is presumably for the following reasons.

[0142] In other words, when the students are facing each other on the left and right, the number or ratio of people facing left and right in the first and second groups is considered to be roughly the same. In contrast, when the students are facing the speaker, there is a bias in the number or ratio of people facing left and right in the first and second groups. Therefore, because there is a difference in the data when the students are facing each other and when they are facing the speaker, it is possible to distinguish between a "consultation state" and a "speaking state," and it is considered that the "consultation state" was evaluated appropriately. [Evaluation when boundary group M is selected] From the results of Examples 8-14, in which the border group M defined by 0.33≦m / w≦0.75 was selected, it can be seen that the behavioral state of group W can be determined with high accuracy. Note that it is preferable that the number of people m in the border group M is smaller than the number of people in each of the first and second groups. [others] However, the behavioral state of the group W is not always constant but changes over time. For this reason, it is desirable to have a configuration in which the group W is continuously photographed by the photographing device 2, the behavioral state is judged by the information processing device 1 at regular intervals, and the judgment results are recorded in the storage unit 13 or can be output to the display device 14, a printer (not shown), an external device, etc.

[0143] An example of time-series output is shown in Figure 29. In the example of Figure 29, Group X is judged every minute to be in a "consultation state" or "other" state. By judging the behavioral state in time series in this way, it is possible to grasp the behavioral state of the participants in more detail.

[0144] Furthermore, data such as the number of people in group W, first group X, second group, and border group M, the ratio of the number of people in first group X to group W x / w, the ratio of the number of people in border group M to group W m / w, the number of people in each direction in each group, the left-right pair ratio, and the results of the behavioral state determination may be accumulated and machine-learned. In this case, there is an advantage that the behavioral state can be easily determined by searching for data having the same conditions as the group W to be determined from the accumulated data.

[0145] Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment.

[0146] For example, although group W was divided into two groups, the first group X and the second group, it may be divided into three or four or more groups. In this case, the left-right pair ratio in each group is calculated, and the behavioral state of group X is judged based on an evaluation criterion according to the number of groups into which it is divided. Furthermore, when dividing into three or more groups and selecting a border group M, the border group M may be selected for all of the boundary lines, or may be selected for only some of the boundary lines.

[0147] In addition, it is preferable that there is no overlap of people in the divided groups, but overlap is acceptable as long as it is 10% or less, more preferably 5% or less overlap, and most preferably there is no overlap. [Explanation of symbols]

[0148] 1. Information processing device 2. Imaging Device 11 CPU 12 RAM 13 Storage section 14 Display device 15 Input Devices 16 Network Interface Section

Claims

1. A behavior state determination program for determining a behavior state of a group of people present in a predetermined space, A division step of dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational formula [1]; an extraction step of extracting status information of each person in at least the first group and the second group divided by the division step; a determination step of determining a behavioral state of the group by comparing the state information extracted by the extraction step with evaluation criterion information; A behavioral state determination program that causes a computer to execute the above steps. 0.23≦x / w≦0.77... [1] Where x is the number of people in the first group w: total number of people in the space

2. 2. The behavioral state determination program according to claim 1, wherein in the dividing step, the group of people lined up facing forward is divided in the left and right direction of the space.

3. 3. The behavioral state determination program according to claim 1, wherein the state information is information on any one of a person's face direction, a person's body direction, and a person's line of sight direction.

4. 4. The behavioral state determination program according to claim 3, wherein the direction of the person's face, the direction of the person's body, and the direction of the person's gaze are any of forward, backward, left, and right directions.

5. a selection step of selecting a boundary group in a boundary region between adjacent groups obtained by dividing the boundary group in the dividing step so as to satisfy the following relational expression [2], In the extraction step, status information of each person in the border group is extracted; 3. The behavioral state determination program according to claim 1, wherein in the determination step, the behavioral state of the group is determined taking into consideration state information of each person in the boundary group extracted in the extraction step. 0.33≦m / w≦0.75 [2] Where m is the number of people in the border group w: total number of people in the space

6. 3. The behavioral state determination program according to claim 1, wherein the position information of the person in the space is acquired from an image captured by a camera device or from a position information acquisition device.

7. 3. The behavioral state determination program according to claim 1, wherein the total number of people in the space is six or more.

8. 3. The behavioral state determination program according to claim 1, wherein the evaluation criterion information is information set as an arbitrary threshold value for a minimum value when comparing the number or ratio of people in one direction and the number or ratio of people in the opposite direction.

9. The state determination program according to claim 1 or 2, wherein in the determination step, the behavioral state of the group is determined by machine learning state information of people belonging to each group, or information on the ratio of the number of people in each posture to the total number of people.

10. 3. The behavioral state determination program according to claim 1, wherein the behavioral state of the group is determined at predetermined time intervals in the determination step.

11. 3. The behavioral state determination program according to claim 1, further comprising an output step of outputting a result of the behavioral state of the group determined in the determination step.

12. A behavior state determination system for determining a behavior state of a group of people present in a predetermined space, comprising: A division means for dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational expression [1]; an extraction means for extracting status information of each person in at least the first group and the second group divided by the division means; a determination means for determining a behavioral state of the group by comparing the state information extracted by the extraction means with evaluation criterion information; A behavioral state determination system comprising: 0.23≦X / W≦0.77... [1] Where, X is the number of people in the first group. W: Total number of people in the space

13. A behavioral state determination method for determining a behavioral state of a group of people present in a predetermined space, the method being executed in a behavioral state determination system, A division step of dividing the group into a plurality of groups including at least a first group and a second group so as to satisfy the following relational expression [1]; an extraction step of extracting status information of each person in at least the first group and the second group divided by the division step; a determination step of determining a behavioral state of the group by comparing the state information extracted by the extraction step with evaluation criterion information; A behavioral state determination method comprising: 0.23≦X / W≦0.77... [1] Where, X is the number of people in the first group. W: Total number of people in the space

Images