Information processing method, information processing device, and information processing program

Abstract

This information processing method involves: collecting verbal information and non-verbal information regarding a plurality of users constituting a community; evaluating, on the basis of the verbal information and / or the non-verbal information, a power level indicating the magnitude of power of each of the users in the community; acquiring, on the basis of the power level of each of the users, relationship information indicating the relationship between users among the plurality of users; setting, on the basis of the relationship information, detection sensitivity for detecting inappropriate speech and behavior from among communications made between said users among the plurality of users; determining, by using the detection sensitivity as a reference, whether or not the communications include the inappropriate speech and behavior; and outputting information for reporting that the inappropriate speech and behavior have been detected when the communications include the inappropriate speech and behavior.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to a technique for detecting inappropriate speech and behavior.

[0002] The technology described in Patent Document 1 discloses a management system for inappropriate postings that, in order to eliminate inappropriate postings on the Internet, when a request for investigation is received from a file distribution system regarding a reporter who has reported that a posted file is an inappropriate post, conducts an investigation into the reliability of the reporter using a prohibited information statistics database, and based on this investigation, sends a notification to the file distribution system to reject, approve, or suspend the report.

[0003] However, the technology described in Patent Document 1 was developed with the aim of eliminating inappropriate posts on the Internet, and no consideration has been given to appropriately detecting inappropriate speech and behavior in communities.

[0004] JP 2013-178706 A

[0005] The present disclosure has been made to solve such problems, and aims to provide a technology that enables appropriate detection of inappropriate speech and behavior in a community.

[0006] An information processing method in one aspect of the present disclosure is an information processing method in a computer, which collects verbal information and non-verbal information of multiple users who make up a community, evaluates a power level indicating the strength of each of the multiple users within the community based on at least one of the verbal information and the non-verbal information, obtains relationship information indicating the relationships between users among the multiple users based on the power level of each of the multiple users, sets a detection sensitivity for detecting inappropriate behavior in communications between users among the multiple users based on the relationship information, determines whether the communication includes the inappropriate behavior based on the detection sensitivity, and, if the communication includes the inappropriate behavior, outputs information to notify that the inappropriate behavior has been detected.

[0007] According to the present disclosure, it is possible to appropriately detect inappropriate speech and behavior.

[0008] FIG. 1 is a block diagram showing an example of the configuration of an information processing system according to embodiment 1. FIG. 2 is a diagram schematically showing an example of level correspondence information. FIG. 3 is a diagram showing an example of the data configuration of inappropriate speech and behavior information. FIG. 4 is a flowchart showing an overall view of the processing of the information processing system according to embodiment 1. FIG. 5 is a block diagram showing an example of the configuration of an information processing system according to embodiment 2. FIG. 6 is a flowchart showing an overall view of the processing of the information processing system according to embodiment 2. FIG. 7 is a block diagram showing an example of the configuration of an information processing system according to embodiment 3. FIG. 8 is a flowchart showing an overall view of the processing of the information processing system according to embodiment 3. FIG. 9 is a block diagram showing an example of the configuration of an information processing system according to embodiment 4. FIG. 10 is a flowchart showing an overall view of the processing of the information processing system according to embodiment 4.

[0009] (Background to the present disclosure) When a specific member of a community behaves in an inappropriate manner toward another member, the relationship between the members may deteriorate, and the productivity of community activities may decrease. In order to prevent this, it is conceivable to monitor communication within the community and detect inappropriate behavior.

[0010] Whether or not a certain behavior is deemed inappropriate is influenced in part by the relationships between the members of the community. For example, whether a certain behavior is considered humorous or inappropriate can vary depending on the relationships between the members. If this point is not taken into consideration and behavior that may be deemed inappropriate is uniformly detected, free communication within the community may be hindered, which could result in a decrease in the productivity of community activities.

[0011] The present invention has been made in view of the above-mentioned problems, and aims to provide a technology that enables appropriate detection of inappropriate speech and behavior in a community.

[0012] In order to solve the above problems, the following techniques are disclosed.

[0013] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, which collects verbal information and nonverbal information of multiple users who make up a community, evaluates a power level indicating the strength of each of the multiple users within the community based on at least one of the verbal information and the nonverbal information, obtains relationship information indicating the relationships between users among the multiple users based on the power level of each of the multiple users, sets a detection sensitivity for detecting inappropriate behavior in communications between users among the multiple users based on the relationship information, determines whether the communication includes the inappropriate behavior based on the detection sensitivity, and, if the communication includes the inappropriate behavior, outputs information to notify that the inappropriate behavior has been detected.

[0014] According to this information processing method, the detection sensitivity for detecting inappropriate behavior is set based on relationship information. That is, the detection sensitivity is adjusted according to the relationship between users. Then, based on the detection sensitivity, it is determined whether or not communication contains inappropriate behavior. In this way, inappropriate behavior is detected while taking into account the relationship between users. Therefore, inappropriate behavior in a community can be appropriately detected.

[0015] (2) In the information processing method described in (1) above, the collecting of the nonverbal information may involve collecting attribute information indicating the attributes of each of the multiple users, and the evaluating of the power levels may involve acquiring level correspondence information indicating the correspondence between the attributes and the power levels, and evaluating the power levels of each of the multiple users based on the attribute information and the level correspondence information.

[0016] According to this configuration, attribute information indicating the attributes of the user is used to evaluate the power level of the user, so that the power level can be evaluated with higher accuracy.

[0017] (3) In the information processing method described in (1) or (2) above, the verbal information may be collected by collecting the volume of speech of each of the plurality of users, and the power level may be evaluated based on the volume of speech of each of the plurality of users.

[0018] According to this configuration, the amount of speech of a user is used to evaluate the power level of the user, so that the power level can be evaluated more accurately.

[0019] (4) In the information processing method described in any one of (1) to (3) above, when communication is performed between a first user and a second user having a lower power level than the first user, the detection sensitivity may be set such that a level difference indicating the difference in power level between the first user and the second user is calculated, and the detection sensitivity when communication is performed from the first user to the second user is set high according to the magnitude of the level difference.

[0020] According to this configuration, when a first user communicates with a second user having a lower power level than the first user, the detection sensitivity can be increased according to the magnitude of the level difference. That is, when a user with a high power level communicates with a user with a low power level, the greater the difference in power levels between the two users, the higher the detection sensitivity. In this way, the behavior of a user with a high power level is more likely to be detected as inappropriate behavior, and therefore, when a user with a high power level communicates with a user with a low power level, the user with a high power level can be urged to be careful about his or her behavior.

[0021] (5) In the information processing method described in (4) above, the detection sensitivity may be set low when communication from the second user to the first user occurs, depending on the magnitude of the level difference.

[0022] According to this configuration, the detection sensitivity when a communication is made from the second user to the first user can be lowered according to the magnitude of the level difference. That is, when a communication is made from a user with a low power level to a user with a high power level, the detection sensitivity becomes lower as the difference in power levels between the two users increases. In this way, the behavior of a user with a low power level is less likely to be detected as inappropriate behavior, which encourages users with low power levels to actively communicate with users with high power levels.

[0023] (6) In the information processing method described in any one of (1) to (5) above, the amount of communication between users among the plurality of users may be calculated based on at least one of the verbal information and the non-verbal information, and the detection sensitivity may be adjusted according to the amount of communication.

[0024] With this configuration, the detection sensitivity is adjusted according to the amount of communication, so that the detection sensitivity can be set to a level that more accurately reflects the relationships within the community, making it possible to more appropriately detect inappropriate speech and behavior within the community.

[0025] (7) In another aspect of the present disclosure, an information processing device includes: a collection unit that collects verbal information and nonverbal information of multiple users that constitute a community; a level evaluation unit that evaluates a power level indicating the strength of each of the multiple users within the community based on at least one of the verbal information and the nonverbal information; an acquisition unit that acquires relationship information indicating the relationships between users among the multiple users based on the power level of each of the multiple users; a sensitivity setting unit that sets a detection sensitivity for detecting inappropriate behavior in communication between the users based on the relationship information; a determination unit that determines whether the communication includes the inappropriate behavior based on the detection sensitivity; and an output unit that outputs information to notify that the inappropriate behavior has been detected if the communication includes the inappropriate behavior.

[0026] According to this configuration, it is possible to provide an information processing device that can obtain the same effects as the above-described information processing method.

[0027] (8) In another aspect of the present disclosure, an information processing program causes a computer to function as an information processing device, and causes the computer to perform the following processes: collect verbal information and nonverbal information of multiple users who constitute a community; evaluate a power level indicating the strength of each of the multiple users within the community based on at least one of the verbal information and the nonverbal information; obtain relationship information indicating the relationships between the multiple users based on the power level of each of the multiple users; set a detection sensitivity for detecting inappropriate behavior in communication between the users based on the relationship information; determine whether the communication includes the inappropriate behavior based on the detection sensitivity; and, if the communication includes the inappropriate behavior, output information to notify that the inappropriate behavior has been detected.

[0028] According to this configuration, it is possible to provide an information processing program that can achieve the same effect as the above-described information processing method.

[0029] The present disclosure can also be realized as an information processing system operated by such an information processing program. Needless to say, such a computer program can be distributed on a non-transitory computer-readable recording medium such as a CD-ROM or via a communication network such as the Internet.

[0030] Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and step orders shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all of the embodiments, the respective contents can be combined.

[0031] (First Embodiment) FIG. 1 is a diagram showing an example of an information processing system 100 according to a first embodiment. The information processing system 100 is a system for appropriately detecting inappropriate speech and behavior in a community. Note that a community refers to an association made up of multiple members, and examples of communities include a workplace, a home, a local group, and a sports club. In addition, a community made up of members who provide services and members who receive the services, such as a hospital or a supermarket, also falls under the category of a community. Furthermore, inappropriate speech and behavior refers to remarks or actions that may cause pain to at least one of the mind and body of a member.

[0032] The following describes an example in which the information processing system 100 is applied to a company 1 (an example of a community). In the following example, members of the company 1 are referred to as users.

[0033] 1, an information processing system 100 according to this embodiment includes a plurality of user terminals 2, a plurality of sensors 3, and a server 10 (an example of an information processing device). The plurality of user terminals 2, the plurality of sensors 3, and the server 10 are connected to each other so as to be able to communicate with each other via a network NW. The network NW is, for example, a public communication line such as the Internet. However, the network NW may also be a local area network.

[0034] The multiple user terminals 2 are multiple information processing devices operated by multiple users, and are configured, for example, as multiple desktop computers, laptop computers, or mobile terminals such as smartphones and tablet terminals. Each of the multiple users can send and receive messages using online chat tools such as email by operating the user terminal 2. Furthermore, each of the multiple users can participate in an online conference by operating the user terminal 2.

[0035] The multiple sensors 3 include one or more microphone devices and one or more imaging devices. The one or more microphone devices record sounds generated within the company 1 as sound data. For example, the one or more microphone devices record the user's voice as sound data. In addition, the one or more microphone devices collect various sounds such as the sound of clicking one's tongue or the sound of tapping on a desk. The one or more imaging devices record the user's actions in the company 1 in the form of image data or video data. This data is transmitted to the server 10 via the network NW.

[0036] The multiple sensors 3 are installed, for example, on the walls or ceiling of the company 1. The multiple sensors 3 may be mounted on the above-mentioned multiple user terminals 2. That is, if the user terminal 2 is equipped with a microphone device and an imaging device, these may be used as the multiple sensors 3.

[0037] The server 10 is, for example, a cloud server configured with one or more computers. As shown in FIG. 1, the server 10 includes a communication circuit 20, a memory 30, and a processor 40.

[0038] The communication circuit 20 is a communication interface circuit compatible with a communication method using a network such as Ethernet (registered trademark), and connects the server 10 to the network NW.

[0039] The memory 30 is configured by a non-volatile rewritable storage device such as a solid state drive, a hard disk drive, etc. As shown in Fig. 1 , the memory 30 according to this embodiment has an attribute information storage unit 31, a level correspondence information storage unit 32, and an inappropriate speech and behavior information storage unit 33.

[0040] The attribute information storage unit 31 stores attribute information indicating the attributes of each of a plurality of users. The attribute information includes, for example, the user's age, the user's job title, the length of time the user has belonged to a community, and the blood relationship between a certain user and other users.

[0041] The level correspondence information storage unit 32 stores level correspondence information D1 indicating the correspondence between user attributes and user power levels. Power levels will be described later. FIG. 2 is a diagram schematically illustrating an example of the level correspondence information D1. As shown in FIG. 2, the level correspondence information D1 according to the present embodiment indicates that the power level value assigned to the position of trainee is 2, the power level value assigned to the position of general manager is 6, and the power level value assigned to the position of director is 8. The level correspondence information D1 also indicates that the power level value assigned to the position of assistant manager is 4, the power level value assigned to the position of manager is 5, the power level value assigned to the position of executive director is 9, and the power level value assigned to the position of president is 10. In addition, the level correspondence information D1 may store correspondence between various attributes and power levels.

[0042] The inappropriate speech and behavior information storage unit 33 stores inappropriate speech and behavior information D2 indicating the correspondence between examples of speech and behavior that may be considered inappropriate, inappropriate speech and behavior levels, and detection sensitivity levels. Fig. 3 is a diagram showing an example of the data configuration of the inappropriate speech and behavior information D2. As shown in Fig. 3, the inappropriate speech and behavior information D2 has a first column CL1 in which examples of speech and behavior that may be considered inappropriate are registered, a second column CL2 in which inappropriate speech and behavior levels are registered, and a third column CL3 in which detection sensitivity levels are registered.

[0043] In the first column CL1, various statements and actions are registered as examples of speech and behavior that may be considered inappropriate. Specifically, as shown in Figure 3, statements such as "You idiot," violent behavior, statements such as "Go home," and clicking one's tongue are registered as examples of speech and behavior that may be considered inappropriate.

[0044] The second column CL2 registers inappropriate behavior levels corresponding to examples of behavior that may be considered inappropriate. The inappropriate behavior levels are indicators of the likelihood that a certain behavior or speech constitutes inappropriate behavior. In this example, the inappropriate behavior levels are divided into 10 levels, from level 1 to level 10. The magnitude of the inappropriate behavior level corresponds to the likelihood that the behavior or speech constitutes inappropriate behavior. For example, as shown in FIG. 3 , the inappropriate behavior level for behavior such as "Get a grip" that is relatively unlikely to be considered inappropriate behavior is 1. On the other hand, the inappropriate behavior level for behavior such as "You idiot" that is relatively likely to be considered inappropriate behavior is 10. Note that the configuration of the first column CL1 and the second column CL2 shown in FIG. 3 is merely an example, and the level of inappropriate behavior level assigned to each behavior or speech can be changed as appropriate. In addition, the level of inappropriate speech and behavior may be determined based on the harm score output using the AI ​​model disclosed in Non-Patent Document 1, "Kobayashi Kouga, et al., [online], "Proposal and Evaluation of a Japanese Harmful Expression Schema," Association for Natural Language Processing 2023, [searched March 18, 2024], Internet <URL: https: / / www.anlp.jp / proceedings / annual_meeting / 2023 / pdf_dir / D4-1.pdf>."

[0045] The third column CL3 registers detection sensitivity levels corresponding to the inappropriate speech and behavior levels. The detection sensitivity level is an index indicating the level of detection sensitivity for detecting inappropriate speech and behavior. In this example, the detection sensitivity level is divided into 10 levels, from level 1 to level 10. The sensitivity setting unit 44, which will be described later, sets the detection sensitivity level value in the range of 1 to 10. The value set by the sensitivity setting unit 44 is used as a threshold for the detection unit 45, which will be described later, to detect inappropriate speech and behavior. For example, the sensitivity setting unit 44 sets the detection sensitivity level value to 3. In this case, the detection unit 45, which will be described later, detects speech and behavior with an inappropriate speech and behavior level value of 3 or greater as inappropriate speech and behavior. On the other hand, the detection unit 45 does not detect speech and behavior with an inappropriate speech and behavior level value less than 3 as inappropriate speech and behavior. 3, when the detection sensitivity level is set to 3, the detection unit 45 detects, as inappropriate behavior, speech and behavior for which the inappropriate behavior level value is set to 3 or higher, such as a statement like "You idiot," a statement like "You're stealing my salary," or clicking one's tongue. On the other hand, the detection unit 45 does not detect, as inappropriate behavior, speech and behavior for which the inappropriate behavior level value is set to 1 or 2, such as "Is that okay?" or "Please pull yourself together." In this way, by adjusting the detection sensitivity level, the range of speech and behavior detected as inappropriate behavior can be changed.

[0046] Generally speaking, the detection sensitivity can be increased by setting the detection sensitivity level to a small value, while the detection sensitivity can be decreased by setting the detection sensitivity level to a large value.

[0047] The processor 40 is configured by, for example, a CPU. The processor 40 has a collection unit 41, a level evaluation unit 42, a relationship information acquisition unit 43 (an example of an acquisition unit), a sensitivity setting unit 44, a detection unit 45 (an example of a determination unit), and an output unit 46. The collection unit 41 to the output unit 46 may be realized by the processor 40 executing a predetermined program stored in the memory 30, or may be configured by dedicated hardware circuits.

[0048] The collection unit 41 acquires verbal information and non-verbal information. Verbal information refers to linguistic information used when users communicate with each other. Examples of verbal information include words spoken by users and text included in messages sent and received by users. Non-verbal information refers to non-verbal information related to communication between users. Examples of non-verbal information include a user's facial expression, gaze, posture, gestures, body language, and attribute information. Other examples of non-verbal information include the tone of voice and the volume of the voice. The collection unit 41 acquires verbal information and non-verbal information from, for example, the multiple sensors 3 described above.

[0049] The level evaluation unit 42 evaluates the power level of each of the multiple users, which indicates the strength of that user's influence within the community, based on at least one of the verbal information and the non-verbal information. Note that power within the community refers to, for example, authority, influence, speaking ability, political power, etc.

[0050] The level evaluation unit 42 in this embodiment acquires level correspondence information D1 indicating the correspondence between user attributes and power level levels from the level correspondence information storage unit 32, and evaluates the power level levels of each of multiple users based on the attribute information and the level correspondence information D1.

[0051] The relationship information acquisition unit 43 acquires power balance information (an example of relationship information) that indicates the power balance (an example of relationships) between multiple users based on the power levels of each of the multiple users. Power balance refers to the distribution of power within a community. A power balance becomes unbalanced when a specific individual or group has more power than other members. If this unbalance continues, problems such as power mismatch, a sense of unfairness, and biased decision-making may occur.

[0052] The sensitivity setting unit 44 sets a detection sensitivity for detecting inappropriate speech or behavior in communications between users among a plurality of users based on the power balance information. Specifically, the sensitivity setting unit 44 sets the detection sensitivity by setting a value for the detection sensitivity level. Details will be described later.

[0053] Furthermore, when communication is performed between a first user and a second user having a lower power level than the first user, the sensitivity setting unit 44 calculates a level difference indicating the difference in power level between the first user and the second user, and then sets the detection sensitivity when communication is performed from the first user to the second user to a higher level depending on the magnitude of the level difference.

[0054] Note that setting the detection sensitivity high means setting the detection sensitivity level to a value smaller than the standard value of the detection sensitivity level. In this embodiment, the standard value of the detection sensitivity level is set to 5. Therefore, setting the detection sensitivity high means setting the detection sensitivity level to a value smaller than 5. However, the standard value of the detection sensitivity level is not limited to 5 and can be changed as appropriate.

[0055] Furthermore, the sensitivity setting unit 44 sets the detection sensitivity when communication is made from the second user to the first user to a low value according to the magnitude of the level difference.

[0056] Setting the detection sensitivity low means setting the detection sensitivity level to a value greater than the standard value of the detection sensitivity level.

[0057] The detection unit 45 determines whether or not inappropriate words or actions are included in the communication between users based on the detection sensitivity set by the sensitivity setting unit 44 .

[0058] The output unit 46 outputs alert information when communication contains inappropriate speech or behavior. The alert information is information for notifying the user that inappropriate speech or behavior has been detected. For example, the output unit 46 outputs a message stating "Inappropriate speech or behavior" to the user terminal 2 as alert information. Alternatively, the output unit 46 may output the alert information by vibrating a wearable device (e.g., a smart watch) worn by the user. In addition, the output unit 46 may output an indicator image resembling an indicator having a needle and a scale with numbers as alert information to the user terminal 2. For example, if the inappropriate speech or behavior level set for the speech or behavior detected as inappropriate speech or behavior is 5, the output unit 46 may output an indicator image in which a needle points to the scale with 5 as alert information to the user terminal 2.

[0059] The output unit 46 according to this embodiment outputs the alert information only to the user terminal 2 of the user who has uttered inappropriate words or behavior. However, the output unit 46 may also output the alert information to the user terminals 2 of all of the multiple users. Alternatively, if there is a user who is the administrator of the user who has uttered inappropriate words or behavior, the output unit 46 may output the alert information only to the user terminal 2 of the administrator.

[0060] The processing of the information processing system 100 configured as described above will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an overview of the processing of the information processing system 100 according to the first embodiment.

[0061] In step S1, the collection unit 41 collects verbal information and nonverbal information of each user. That is, the collection unit 41 collects the words and actions of each user. For example, the collection unit 41 collects audio data including the speech of "Good morning" and messages including text such as "Thank you for your hard work" as verbal information. Furthermore, for example, the collection unit 41 collects the gaze and facial direction of each user as nonverbal information. The gaze and facial direction of each user are information represented by three-dimensional vectors. Note that a technique for detecting the gaze and facial direction of each user is disclosed, for example, in Non-Patent Document 2, "Yoshio Matsumoto, et al., "Development of a Real-Time Face and Gaze Measurement System and Its Application to Intelligent Interfaces," Transactions of the Information Processing Society of Japan, Computer Vision and Image Media, October 2006, vol. 47, no. SIG15 (CVIM16), pp. 10-21." Furthermore, for example, the collection unit 41 collects image data or video data in which the action of "bowing one's head" is recorded as non-verbal information.

[0062] Furthermore, in step S1, the collection unit 41 collects attribute information of each user as nonverbal information. For example, the collection unit 41 collects attribute information indicating that the job title of user A is trainee, the job titles of users B and C are general managers, and the job title of user D is director. As described above, this attribute information is stored in the attribute information storage unit 31 of the memory 30.

[0063] In step S2, the collection unit 41 identifies a user. That is, the collection unit 41 identifies the subject and object of each speech and behavior collected in step S1. In other words, the collection unit 41 identifies which user uttered each speech and behavior collected in step S1 and to which user. The collection unit 41 performs the above-mentioned identification process using, for example, non-verbal information such as the gaze and facial direction of each user collected in step S1, and verbal information such as the content of each user's voice. Furthermore, if text is collected as verbal information in step S1 and the text includes mention information, the collection unit 41 may identify the user by referring to the mention information. Mention information is information including an identifier for identifying a user, and is composed of a character string such as "@ + user name," for example.

[0064] Hereinafter, a user who is the subject of a predetermined speech or behavior will be referred to as a subject user, and a user who is the object of a predetermined speech or behavior will be referred to as an object user.

[0065] In step S3, the detection unit 45 detects words and actions that may be considered inappropriate from the words and actions of each user collected by the processing in step S1. If the detection unit 45 detects words and actions that may be considered inappropriate (YES in step S3), the processing proceeds to step S4. If the detection unit 45 does not detect words and actions that may be considered inappropriate (NO in step S3), the processing returns to step S1.

[0066] The processing of step S3 will be described in detail. The detection unit 45 according to this embodiment detects potentially inappropriate behavior by comparing the behavior of each user collected in step S1 with the first column CL1 shown in FIG. 3 . For example, assume that sound data including the speech "Stop messing around" is collected as verbal information in step S1. In this case, the detection unit 45 first converts the speech "Stop messing around" into text using existing speech recognition technology. Then, if a word with a similarity higher than the reference value for the converted speech is registered in the first column CL1, the detection unit 45 determines YES in step S3. Here, as shown in FIG. 3 , the word "Stop messing around" is registered in the first column CL1. Therefore, in this example, the detection unit 45 determines that inappropriate behavior has been detected (YES in step S3).

[0067] Returning to Fig. 4 , in step S4, the detection unit 45 stores the value of the inappropriate speech and behavior level set for the behavior detected in step S3 as the value of the comparison level in a predetermined storage area of ​​the memory 30. For example, assume that the behavior "Stop messing around" is detected in step S3. Here, as shown in the first column CL1 and the second column CL2 of Fig. 3 , the behavior "Stop messing around" is a behavior for which the inappropriate speech and behavior level value is set to 4. Therefore, in this case, the detection unit 45 stores that the value of the comparison level is 4.

[0068] In step S5, the level evaluation unit 42 evaluates the power level of the target user of the inappropriate behavior. That is, the level evaluation unit 42 evaluates the power levels of the subject user who uttered the behavior that may be considered inappropriate and the object user who is the object of the behavior.

[0069] The processing of step S5 will be described below using an example in which the power levels of the subject user and the object user of the verbal behavior "Stop messing around" are evaluated. It is assumed that the subject user of the verbal behavior "Stop messing around" is identified as user A and the object user is identified as user B by the processing of step S2. In this case, the level evaluation unit 42 evaluates the power levels of user A and user B according to the following procedure. First, the level evaluation unit 42 references the attribute information collected by the collection unit 41 in step S1 to determine that user A's job title is trainee and user B's job title is general manager. Next, the level evaluation unit 42 acquires level correspondence information D1 shown in FIG. 2 from the level correspondence information storage unit 32 of the memory 30. Then, the level evaluation unit 42 references the level correspondence information D1 to determine that the power level of a trainee is assigned "2" and the power level of a general manager is assigned "6." Based on this information, the level evaluation unit 42 evaluates the power level of user A (trainee) as 2 and the power level of user B (general manager) as 6.

[0070] Returning to FIG. 4 , in step S6, the relationship information acquisition unit 43 acquires power balance information. The power balance information is a data set of the power level values ​​of the subject user and the power level values ​​of the object user. For example, if the subject user is user A and the object user is user B, the power balance information is expressed as "2:6." If the subject user is B and the object user is C, the power balance information is expressed as "6:6." If the subject user is A and the object user is D, the power balance information is expressed as "2:8."

[0071] In step S7, the sensitivity setting unit 44 determines whether the power balance is balanced. That is, it determines whether the power level value of the subject user and the power level value of the object user are the same. If the power balance is balanced between the subject user and the object user (YES in step S7), the process proceeds to step S11. If the power balance is not balanced between the subject user and the object user (NO in step S7), the process proceeds to step S8.

[0072] In step S8, the sensitivity setting unit 44 determines whether the power level of the subject user is higher than the power level of the object user. If the power level of the subject user is higher than the power level of the object user (YES in step S8), the process proceeds to step S9. If the power level of the subject user is lower than the power level of the object user (NO in step S8), the process proceeds to step S10.

[0073] In step S9, the sensitivity setting unit 44 calculates the following formula 1 to set the value of the detection sensitivity level.

[0074] L1=s-p·(u1-u2) (Equation 1) L1 indicates the value of the detection sensitivity level. In other words, L1 is a threshold for detecting inappropriate speech and behavior. s indicates the standard value of the detection sensitivity level. As described above, the standard value in this embodiment is set to 5. p indicates the first correction coefficient for adjusting the value of the detection sensitivity level. The first correction coefficient is, for example, 0.5. Note that the first correction coefficient can be changed in the range of 0<p<1. u1 indicates the value of the power level of the user with the higher power level in the power balance information acquired in step S6. u2 indicates the value of the power level of the user with the lower power level in the power balance information acquired in step S6.

[0075] The detection sensitivity levels according to this embodiment are divided into levels with 1 as the lower limit and 10 as the upper limit. Therefore, when the value of L1 is below 1, the sensitivity setting unit 44 obtains "L1=1" as the solution to Equation 1. Similarly, when the value of L1 is above 1, the sensitivity setting unit 44 obtains "L1=10".

[0076] In step S10, the sensitivity setting unit 44 calculates the following equation 2 to set the value of the detection sensitivity level.

[0077] L1=s+p·(u1−u2) (Equation 2) Since L1, s, p, u1, and u2 are as explained in Equation 1, explanation will be omitted.

[0078] In step S11, the sensitivity setting unit 44 sets the standard value of the detection sensitivity level as the value of the detection sensitivity level.

[0079] In step S12, the detection unit 45 determines whether the value of the comparison level stored in step S4 is equal to or greater than the value of the detection sensitivity level set in step S9, step S10, or step S11. If the value of the comparison level is equal to or greater than the value of the detection sensitivity level (YES in step S12), the process proceeds to step S13. If the value of the comparison level is smaller than the value of the detection sensitivity level (NO in step S12), the process returns to step S1.

[0080] In step S13, the output unit 46 outputs the alert information.

[0081] The processing of steps S7 to S13 will be described below using a specific example.

[0082] (Example 1) In Example 1, a situation is assumed in which the processing from step S1 to step S6 has detected the statement "Stop messing around" as a potentially inappropriate statement, and it has been determined that this statement was made by user A to user B.

[0083] In Example 1, since the power level of user A (power level 2) and the power level of user B (power level 6) are not the same value, a NO determination is made in the processing of step S7, and the processing proceeds to step S8. Since the power level of user A is lower than the power level of user B, a NO determination is made in the processing of step S8, and the processing proceeds to step S10.

[0084] In step S10, Equation 2 is calculated. As described above, in this embodiment, the standard value of the detection sensitivity level is set to 5, and the first correction coefficient p is set to 0.5. Therefore, 5 is substituted for s in Equation 2, and 0.5 is substituted for p. Furthermore, since user B (power level 6) corresponds to a user with a high power level and user A (power level 2) corresponds to a user with a low power level, 6 is substituted for u1 and 2 is substituted for u2. Therefore, by calculating Equation 2, the sensitivity setting unit 44 obtains the solution "L1 = 7." Based on this solution, the sensitivity setting unit 44 sets the detection sensitivity level when user A communicates with user B to 7. That is, the sensitivity setting unit 44 sets the value of the detection sensitivity level so that only behaviors with an inappropriate behavior level value of 7 or higher are detected as inappropriate behaviors when user A communicates with user B.

[0085] Then, in step S12, the detection unit 45 determines whether the value of the inappropriate speech level (comparison level) set for the speech and behavior of "Stop messing around" is larger than the value of the detection sensitivity level set in step S10. As described above, the value of the inappropriate speech level set for the speech and behavior of "Stop messing around" is 4, and the value of the detection sensitivity level set in step S10 is 7. Therefore, in this example, a determination of "NO" is made in step S12, and the processing returns to step S1. That is, in example 1, the speech and behavior of "Stop messing around" uttered by user A to user B is not detected as inappropriate speech and behavior.

[0086] (Example 2) In Example 2, a situation is assumed in which the processing from step S1 to step S6 has detected the remark "Stop messing around" as a potentially inappropriate remark, and it has been determined that this remark was made by user B toward user A.

[0087] In Example 2, the power level of User A (power level 2) and the power level of User B (power level 6) are not the same value, so the determination in step S7 is NO and the process proceeds to step S8. Then, because the power level of User B is higher than the power level of User A, the determination in step S8 is YES and the process proceeds to step S9.

[0088] In step S9, Equation 1 is calculated. As in Example 1, 5 is substituted for s in Equation 1, 0.5 is substituted for p, 6 is substituted for u1, and 2 is substituted for u2. Therefore, by calculating Equation 1, the sensitivity setting unit 44 obtains the solution "L1 = 3." Based on this solution, the sensitivity setting unit 44 sets the value of the detection sensitivity level when communication is performed from user B to user A to 3. In other words, the sensitivity setting unit 44 sets the value of the detection sensitivity level so that when communication is performed from user B to user A, only speech or behavior with an inappropriate speech or behavior level value of 3 or higher is detected as inappropriate speech or behavior.

[0089] Then, in step S12, the detection unit 45 determines whether the value of the inappropriate speech and behavior level set for the speech and behavior "Stop messing around" is larger than the value of the detection sensitivity level set in step S9. The value of the inappropriate speech and behavior level set for the speech and behavior "Stop messing around" is 4, and the value of the detection sensitivity level set in step S9 is 3, so in this example, a determination of YES is made in step S12. That is, in example 2, the speech and behavior of "Stop messing around" uttered by user B toward user A is detected as inappropriate speech and behavior. Then, in step S13, a message such as "This is inappropriate speech and behavior" is output as alert information to the user terminal 2 of user B.

[0090] (Example 3) In Example 3, a situation is assumed in which the processing of steps S1 to S6 has detected the statement "Stop messing around" as a potentially inappropriate statement, and it has been determined that this statement was made by user B toward user C.

[0091] In Example 3, since the power level values ​​of user B (power level 6) and user C (power level 6) are the same, a YES determination is made in the processing of step S7, and the processing proceeds to step S11. Then, in step S11, the sensitivity setting unit 44 sets the detection sensitivity level to 5, which is the standard value of the detection sensitivity level. That is, the sensitivity setting unit 44 sets the detection sensitivity level value so that when user B communicates with user C, only speech or behavior with an inappropriate speech or behavior level value of 5 or higher is detected as inappropriate speech or behavior.

[0092] Then, in step S12, the detection unit 45 determines whether the value of the inappropriate speech level set for the speech and behavior "Stop messing around" is larger than the value of the detection sensitivity level set in step S11. Since the value of the inappropriate speech and behavior set for the speech and behavior "Stop messing around" is 4 and the value of the detection sensitivity level set in step S11 is 5, in this example, the determination in step S12 is NO, and the processing returns to step S1. That is, in example 3, the speech and behavior of "Stop messing around" uttered by user B to user C is not detected as inappropriate speech and behavior.

[0093] As described above, according to the information processing system 100 of embodiment 1, the value of the detection sensitivity level for detecting inappropriate behavior is set based on power balance information. Specifically, in the above-described formulas 1 and 2 for setting the value of the detection sensitivity level, the numerical values ​​substituted for u1 and u2 are determined based on the power balance information. Then, in the information processing system 100, inappropriate behavior is detected based on the detection sensitivity level value determined based on the power balance information. In this way, by setting the detection sensitivity based on the power balance information, the detection sensitivity is set to a sensitivity that reflects the human relationships within the community. This makes it possible to appropriately detect inappropriate behavior.

[0094] Furthermore, in the information processing system 100 according to the first embodiment, when a first user communicates with a second user having a lower power level than the first user, the above formula 1 is calculated. Here, in formula 1, "u1 - u2" is calculated to calculate the level difference between the first user and the second user. In formula 1, the larger the value of the level difference calculated by "u1 - u2", the smaller the value of the detection sensitivity level. In other words, the detection sensitivity is set higher. In this way, the larger the difference in power level between the first user, i.e., the user with the higher power level, and the second user, i.e., the user with the lower power level, the more likely the behavior of the user with the higher power level is to be detected as inappropriate behavior. As a result, when a user with a higher power level communicates with a user with a lower power level, the user with the higher power level can be urged to be careful about his or her behavior.

[0095] Furthermore, in the information processing system 100 according to the first embodiment, when a second user communicates with a first user, the above-described formula 2 is calculated. Here, in formula 2, "u1 - u2" is calculated to calculate the level difference between the first user and the second user. In formula 2, the larger the value of the level difference calculated by "u1 - u2", the larger the value of the inappropriate speech and behavior level set as the detection sensitivity. In other words, the detection sensitivity is set lower. In this way, the larger the level difference, the more difficult it is for the speech and behavior of the second user, i.e., the user with the lower power level, to be detected as inappropriate speech and behavior. As a result, users with lower power levels are encouraged to actively communicate with users with higher power levels.

[0096] (Embodiment 2) An information processing system 100S according to embodiment 2 will be described. The information processing system 100S adjusts the value of the detection sensitivity level in accordance with the amount of communication between users over a certain period of time. Note that in embodiment 2, the same components as in embodiment 1 are denoted by the same reference numerals, and their description will be omitted.

[0097] Fig. 5 is a block diagram showing an example of the configuration of an information processing system 100S. As shown in Fig. 5, a server 10S of the information processing system 100S has a processor 40S. The processor 40S has a collection unit 41S instead of the above-mentioned collection unit 41, and has a sensitivity setting unit 44S instead of the above-mentioned sensitivity setting unit 44.

[0098] The collection unit 41S according to the second embodiment collects verbal information and nonverbal information, similar to the collection unit 41 described above. The collection unit 41S then acquires first information based on at least one of the verbal information and the nonverbal information. The first information is information used to calculate the amount of communication between users over a certain period of time, and includes, for example, the participation time of each user in online conferences, the time each user spent exchanging messages using online chat tools such as e-mail, the participation time of each user in real-world conferences, and the time each user spent interacting in the real world. A real-world conference refers to a conference held in the real world by at least three or more users. A real-world conversation refers to a one-on-one discussion between users in the real world. When acquiring the first information, the collection unit 41S may refer to schedule information recording the planned activities of each user. The schedule information is stored, for example, in the user terminal 2.

[0099] The sensitivity setting unit 44S according to the second embodiment sets the value of the detection sensitivity level based on the power balance information, similar to the sensitivity setting unit 44 according to the first embodiment.

[0100] Furthermore, the sensitivity setting unit 44S according to the second embodiment calculates the amount of communication between users based on the first information acquired based on at least one of verbal information and non-verbal information. Specifically, the sensitivity setting unit 44S calculates the amount of communication between users among the plurality of users based on at least one of the participation time of each user in an online conference, the time required for each user to communicate using an online chat tool, the participation time of each user in a conference in the real world, and the conversation time of each user in the real world.

[0101] The sensitivity setting unit 44S according to the second embodiment adjusts the value of the detection sensitivity level in accordance with the amount of communication.

[0102] The processing of the information processing system 100S configured as above will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an overall view of the processing of the information processing system 100S according to the second embodiment. Note that the following describes an example in which the processing shown in Fig. 6 is performed once a day. However, the timing at which the processing shown in Fig. 6 is performed can be changed as appropriate.

[0103] In step S101, the collection unit 41S collects verbal information and nonverbal information. Then, the collection unit 41S acquires first information based on at least one of the verbal information and the nonverbal information. When collecting the first information, the collection unit 41S may acquire the number of characters of text sent by each user in their exchange using the online chat tool, instead of the time required for each user to exchange using the online chat tool.

[0104] In step S102, the sensitivity setting unit 44S calculates the amount of communication c(t) in a certain period of time by calculating the following formula 3. The certain period of time is, for example, one day.

[0105] c(t) = c1 · k1 + c2 · k2 + c3 · k3 + c4 · k4 ... (Equation 3) In the above equation 3, c1 represents the total participation time of a specific user in an online conference. k1 represents a first weighting coefficient for correcting c1. c2 represents the total participation time of a specific user in interactions using an online chat tool. k2 represents a second weighting coefficient for correcting c2. c3 represents the total participation time of a specific user in a real-world conference. k3 represents a third weighting coefficient for correcting c3. c4 represents the total dialogue time of a specific user in the real world. k4 represents a fourth weighting coefficient for correcting c4. In this embodiment, the values ​​of k1 to k4 are set so that k1 < k2 < k3 < k4. However, the magnitude relationship between k1 to k4 can be changed as appropriate.

[0106] As an example, assume that the amount of communication c(t) between user A and user B is calculated. In this case, c1 is substituted with a value indicating the sum of user A's participation time and user B's participation time in the online conference. c2 is substituted with a value indicating the total time required for user A and user B to interact using online chat. c3 is substituted with a value indicating the sum of user A's participation time and user B's participation time in the real-world conference. c4 is substituted with a value indicating the total dialogue time in the real world between user A and user B.

[0107] In step S101, if the collection unit 41S acquires the number of characters in the text sent by each user in an exchange using the online chat tool, the sensitivity setting unit 44S converts the number of characters in the text into the time required for the exchange using the online chat tool. For example, the collection unit 41S calculates the average number of characters spoken by a user per minute based on the verbal information collected in step S101. The collection unit 41S then performs a conversion process based on the average number of characters spoken and the number of characters in the text. Specifically, if the average number of characters spoken by user A per minute is 100 characters and user A sends 300 characters of text using the online chat tool, the collection unit 41S calculates that the time required for user A to exchange using the online chat tool is three minutes.

[0108] In step S103, the sensitivity setting unit 44S adjusts the value of the detection sensitivity level based on the amount of communication c(t) over a certain period of time. Specifically, the sensitivity setting unit 44S calculates the following equation 4.

[0109] L3 = L1 + l c(t) (Equation 4) L3 indicates the value of the detection sensitivity level adjusted based on the amount of communication over a certain period of time. L1 indicates the value of the detection sensitivity level set based on power balance information. An example of setting the value of the detection sensitivity level based on power balance information is as described in steps S1 to S11 of embodiment 1, so a description thereof will be omitted. l indicates a second correction coefficient for correcting the value of L3. c(t) indicates the amount of communication over a certain period of time, as described above.

[0110] Then, in step S103, the sensitivity setting unit 44S sets L3 calculated by calculating Equation 4 as the value of the detection sensitivity level.

[0111] As described above, the information processing system 100S according to the second embodiment can dynamically adjust the detection sensitivity level, which is statically set based on power balance information, based on the amount of communication c(t) over a certain period of time. That is, the detection sensitivity level set by the processing of step S9, step S10, or step S11 according to the first embodiment can be adjusted based on the amount of communication c(t) over a certain period of time. Specifically, in the above equation 4, the larger the value of the communication amount c(t) over a certain period of time, the larger the value of L3. In other words, the detection sensitivity decreases. In this way, the detection sensitivity can be lowered between users who have a large value of the communication amount c(t) over a certain period of time and are considered to have a close relationship. This allows the detection sensitivity to be set to a level that more accurately reflects the interpersonal relationships within the community. As a result, it becomes possible to more appropriately detect inappropriate behavior.

[0112] (Embodiment 3) An information processing system 100T according to embodiment 3 will be described. The information processing system 100T analyzes user emotions based on the facial expressions of users when communicating with each other, and adjusts detection sensitivity based on the analysis results. Note that in embodiment 3, the same components as in embodiment 2 are assigned the same reference numerals, and descriptions thereof will be omitted.

[0113] Fig. 7 is a block diagram showing the configuration of an information processing system 100T. As shown in Fig. 7, a server 10T of the information processing system 100T has a memory 30T and a processor 40T. The memory 30T has a facial expression information storage unit 34. The processor 40T has a collection unit 41T instead of the collection unit 41 according to the first embodiment, and a sensitivity setting unit 44T instead of the sensitivity setting unit 44 according to the first embodiment. The processor 40T also has an emotion analysis unit 47T.

[0114] The facial expression information storage unit 34 stores various template images showing samples of human facial expressions. Specifically, the facial expression information storage unit 34 stores template images showing facial expressions such as anger, crying, laughter, and joy. In addition, the facial expression information storage unit 34 may further store template images showing surprised expressions, troubled expressions, and the like.

[0115] The collection unit 41T collects user facial expression information, which indicates the facial expressions of users when communicating with each other, as non-verbal information. The user facial expression information is configured, for example, by image data or video data showing the user's facial expression.

[0116] The emotion analysis unit 47T analyzes the user's emotions when communication is taking place based on the user's facial expression information, and evaluates whether the communication that took place over a certain period of time was positive or negative based on the analysis results.

[0117] The sensitivity setting unit 44T according to the third embodiment sets the value of the detection sensitivity level based on the power balance information, similar to the sensitivity setting unit 44T according to the first embodiment. Then, the sensitivity setting unit 44T adjusts the value of the detection sensitivity level based on the evaluation result of the emotion analysis unit 47T.

[0118] The processing of the information processing system 100T configured as described above will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an overview of the processing of the information processing system 100T according to the third embodiment.

[0119] In step S201, the collection unit 41T collects verbal information and non-verbal information. In particular, the collection unit 41T collects user facial expression information as non-verbal information.

[0120] In step S202, the emotion analysis unit 47T analyzes the emotion of the user when communication is taking place based on the user's facial expression information. An example of the processing by the emotion analysis unit 47T will be described below.

[0121] For example, assume that a three-minute conversation is taking place between user A and user B, and that the conversation is collected as verbal information by the collection unit 41T. Also assume that user facial expression information indicating the facial expressions of both users during the conversation is collected as non-verbal information by the collection unit 41T. In this case, the emotion analysis unit 47T divides the conversation into one-minute intervals and classifies it into conversation sections A, B, and C.

[0122] Next, the emotion analysis unit 47T extracts feature amounts of the template image stored in the facial expression information storage unit 34 and feature amounts of the user A's user facial expression information in the conversation section A, and calculates the similarity between the two based on these feature amounts. The emotion analysis unit 47T then analyzes the user's emotion based on the template image that is most similar to the user facial expression information. For example, if the similarity between a template image showing an angry expression and the user A's user facial expression information is highest, the emotion analysis unit 47T analyzes that user A is angry. Alternatively, if the similarity between a template image showing a smiling expression and the user A's user facial expression information is highest, the emotion analysis unit 47T analyzes that user A is smiling. Using a similar method, the emotion analysis unit 47T analyzes the emotion of user B. If emotions such as anger or crying are most frequently observed in the conversation section A, the emotion analysis unit 47T evaluates that negative communication took place in the conversation section A. Conversely, if emotions such as laughter and joy are most frequently observed in conversation section A, the emotion analysis unit 47T evaluates that positive communication took place in conversation section A. Similar processing is performed on conversation sections B and C.

[0123] The emotion analysis unit 47T performs the above process on all communications that have taken place between user A and user B over a certain period of time (for example, one day).

[0124] In step S203, the sensitivity setting unit 44T calculates the following equation 5.

[0125] L4 = L1 + c1(t) m1 - c2(t) m2 ... (Equation 5) L4 indicates the value of the detection sensitivity level adjusted based on the user's emotions. L1 indicates the value of the detection sensitivity level set based on the power balance information. c1(t) indicates the total time of positive communication that occurred during a certain period of time. m1 indicates the third correction coefficient for correcting c1(t). c2(t) indicates the total time of negative communication that occurred during a certain period of time. m2 indicates the fourth correction coefficient for correcting c2(t).

[0126] Then, in step S203, the sensitivity setting unit 44T sets L4 calculated by the above equation 5 as the value of the detection sensitivity level.

[0127] As described above, the information processing system 100T according to the third embodiment can dynamically adjust the detection sensitivity level value, which is statically set based on power balance information, based on the emotion indicated by the user's facial expression. That is, the detection sensitivity level value set by the processing of step S9, step S10, or step S11 according to the first embodiment can be adjusted based on the user's emotion. Specifically, in the above equation 5, the value of L4 increases between users who are engaged in positive communication and who are considered to have positive emotions toward each other. That is, the detection sensitivity decreases. On the other hand, the value of L4 decreases between users who are engaged in negative communication and who are considered to have negative emotions toward each other. That is, the detection sensitivity increases. This configuration allows the detection sensitivity to be set to a level that more accurately reflects the interpersonal relationships within the community. As a result, inappropriate speech and behavior can be more appropriately detected.

[0128] (Embodiment 4) An information processing system 100U according to embodiment 4 will be described. The information processing system 100U analyzes user emotions based on messages exchanged between users, and adjusts detection sensitivity based on the emotions. Note that in embodiment 4, the same components as in embodiment 1 are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0129] 9 is a block diagram showing an example of the configuration of an information processing system 100U. As shown in FIG. 9, a server 10U of the information processing system 100U includes a memory 30U and a processor 40U. The memory 30U includes an emotion dictionary storage unit 35. The processor 40U also includes a collection unit 41U instead of the collection unit 41 according to the first embodiment, and a sensitivity setting unit 44U instead of the sensitivity setting unit 44 according to the first embodiment. The processor 40U also includes an emotion analysis unit 47U.

[0130] The emotion dictionary storage unit 35 stores an emotion dictionary that contains words that express human emotions. Each word contained in the emotion dictionary is associated with emotion information. Emotion information is information that indicates whether a word is a positive word or a negative word, and in this example is represented by -1, 0, or +1. -1 indicates that the word is a negative word, +1 indicates that the word is a positive word, and 0 indicates that the word is neutral.

[0131] The collection unit 41U collects text included in messages sent and received between users as verbal information.

[0132] The emotion analysis unit 47U analyzes the emotions of users based on the text included in messages exchanged between users.

[0133] The sensitivity setting unit 44U sets the value of the detection sensitivity level based on the power balance information, similar to the sensitivity setting unit 44 according to embodiment 1. Then, the sensitivity setting unit 44U adjusts the value of the detection sensitivity level based on the analysis result of the emotion analysis unit 47U.

[0134] The processing of the information processing system 100U configured as described above will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an overview of the processing of the information processing system 100U according to the fourth embodiment.

[0135] In step S301, the collection unit 41U collects verbal information and nonverbal information. In this case, the collection unit 41U collects text included in messages exchanged between users via an online chat tool over a certain period (e.g., one day). For example, the collection unit 41U collects messages exchanged between user A and user B.

[0136] In step S302, the emotion analysis unit 47U analyzes the user's emotions when communication is taking place based on the text. An example of the processing by the emotion analysis unit 47U will be described below.

[0137] The emotion analysis unit 47U first performs morphological analysis on the text and converts the resulting words into numerical vectors using a vector conversion method such as Word2Vec. Next, the words registered in the emotion dictionary stored in the emotion dictionary storage unit 35 are vectorized using a similar method. Next, the similarity between the words extracted from the text and the words registered in the emotion dictionary is calculated. Distances such as cosine distance and Euclidean distance can be used to calculate the similarity. Then, based on the similarity, the word most similar to the word extracted from the text is searched for from among the words registered in the emotion dictionary. Then, based on the emotion information associated with the most similar word, the word extracted from the text is evaluated as being positive or negative.

[0138] For example, assume that a text sent from user A to user B contains the word "good morning" and the word "hello" is registered in the emotion dictionary. Here, assume that the word with the highest similarity (closest cosine distance, etc.) to "good morning" is "hello." Furthermore, assume that the word "hello" is associated with emotion information of "+1" in the emotion dictionary. In other words, assume that the word "hello" is registered in the emotion dictionary as a word that indicates a positive emotion. In this case, the emotion analysis unit 47U evaluates the word "good morning" as a positive word. The emotion analysis unit 47U performs the above process on all words included in all messages exchanged between user A and user B over a certain period of time. The emotion analysis unit 47U then calculates the following equation 6.

[0139] L5 = L1 + d1(t) n1 - d2(t) n2 ... (Equation 6) L5 indicates the value of the detection sensitivity level adjusted based on the user's emotions. L1 indicates the value of the detection sensitivity level set based on the power balance information. d1(t) indicates the total number of positive words detected from the text. n1 indicates the fifth correction coefficient for correcting d1(t). d2(t) indicates the total number of negative words detected from the text. n2 indicates the sixth correction coefficient for correcting d2(t). According to Equation 6 above, the more positive words contained in the text exchanged between users over a certain period of time, the larger the value of L5 becomes. In other words, the lower the detection sensitivity becomes. On the other hand, the more negative words contained in the text exchanged between users over a certain period of time, the smaller the value of L5 becomes. In other words, the higher the detection sensitivity becomes.

[0140] Then, in step S303, the sensitivity setting unit 44U sets L5 calculated by the above equation 6 as the value of the detection sensitivity level.

[0141] According to the above configuration, the detection sensitivity can be adjusted to be lower for users who are likely to have positive feelings toward each other because the text exchanged between them contains many positive words. On the other hand, the detection sensitivity can be adjusted to be higher for users who are likely to have negative feelings toward each other because the text exchanged between them contains many negative words. This allows the detection sensitivity to be set to a level that more accurately reflects the relationships within the community. As a result, inappropriate speech and behavior can be more appropriately detected.

[0142] The present disclosure can employ the following modifications.

[0143] (Modification)

[0144] (1) The information processing system 100 can also be used to determine whether a user's behavior corresponds to inappropriate speech or behavior. For example, the user's behavior may be recorded in the form of video data by one or more of the imaging devices described above, and the detection unit 45 may analyze the video data using existing video recognition technology. If the video data includes any of the behaviors registered in the first column CL1 of Fig. 3 (such as a violent behavior, clicking one's tongue, or hitting a table), the detection unit 45 may determine YES in step S3 of Fig. 4.

[0145] The information processing system 100 can also be used to determine whether words contained in text sent and received via online chat tools or the like correspond to inappropriate speech or behavior.

[0146] (2) In the first embodiment, an example of calculating a user's power level based on the user's job title has been described, but a user's power level may also be calculated based on the amount of speech the user makes. For example, the total speech time of each user over a certain period may be calculated based on the amount of speech the user makes, and the power level may be evaluated in descending order of the total speech time.

[0147] Alternatively, the power level of each user may be evaluated based on the volume of the user's voice. For example, the average volume of the voice of each user may be calculated, and the power level may be evaluated in descending order of the average volume.

[0148] Additionally, the power level of a user may be evaluated based on the content of the user's utterances.

[0149] (3) Although a power balance has been given as an example of a relationship between users, the relationship is not limited to this. For example, the relationship may be a hierarchical relationship.

[0150] (4) The method of analyzing a user's emotions by the emotion analysis unit is not limited to the above example. For example, the emotion analysis unit may analyze a user's emotions based on the content of the user's comments.

[0151] The present disclosure is useful in the technical field of detecting inappropriate speech and behavior.

Claims

1. An information processing method on a computer, comprising: collecting verbal information and non-verbal information of a plurality of users constituting a community; evaluating a power level indicating the strength of each of the plurality of users within the community based on at least one of the verbal information and the non-verbal information; obtaining relationship information indicating the relationships between the plurality of users based on the power level of each of the plurality of users; setting a detection sensitivity for detecting inappropriate behavior in communications between the plurality of users based on the relationship information; determining whether the communication contains the inappropriate behavior based on the detection sensitivity; and, if the communication contains the inappropriate behavior, outputting information to notify that the inappropriate behavior has been detected.

2. The information processing method of claim 1, wherein the collecting of the nonverbal information comprises collecting attribute information indicating the attributes of each of the plurality of users, and the evaluating of the power levels comprises acquiring level correspondence information indicating the correspondence between the attributes and the levels of the power levels, and evaluating the levels of the power levels of each of the plurality of users based on the attribute information and the level correspondence information.

3. An information processing method according to claim 1 or 2, wherein the collecting of the verbal information comprises collecting the volume of speech of each of the plurality of users, and the evaluating of the power level comprises evaluating the level of the power level of each of the plurality of users based on the volume of speech.

4. The information processing method of claim 1 or 2, wherein when communication is performed between a first user and a second user having a lower power level than the first user, the detection sensitivity is set by calculating a level difference indicating the difference in power level between the first user and the second user, and setting the detection sensitivity when communication is performed from the first user to the second user high depending on the magnitude of the level difference.

5. The information processing method according to claim 4, wherein the detection sensitivity setting includes setting the detection sensitivity when communication from the second user to the first user is performed low in accordance with the magnitude of the level difference.

6. An information processing method according to claim 1 or 2, further comprising: calculating the amount of communication between users among the plurality of users based on at least one of the verbal information and the non-verbal information; and adjusting the detection sensitivity in accordance with the amount of communication.

7. An information processing device comprising: a collection unit that collects verbal information and non-verbal information of a plurality of users that constitute a community; a level evaluation unit that evaluates a power level indicating the strength of each of the plurality of users within the community based on at least one of the verbal information and the non-verbal information; an acquisition unit that acquires relationship information indicating the relationships between users among the plurality of users based on the power level of each of the plurality of users; a sensitivity setting unit that sets a detection sensitivity for detecting inappropriate behavior in communications between users among the plurality of users based on the relationship information; a determination unit that determines whether the communication includes the inappropriate behavior based on the detection sensitivity; and an output unit that outputs information to notify that the inappropriate behavior has been detected if the communication includes the inappropriate behavior.

8. An information processing program that causes a computer to function as an information processing device, the information processing program causing the computer to perform the following processes: collect verbal information and non-verbal information of multiple users that make up a community; evaluate a power level indicating the strength of each of the multiple users within the community based on at least one of the verbal information and the non-verbal information; obtain relationship information indicating the relationships between the multiple users based on the power level of each of the multiple users; set a detection sensitivity for detecting inappropriate behavior in communications between the multiple users based on the relationship information; determine whether the communication contains inappropriate behavior based on the detection sensitivity; and, if the communication contains inappropriate behavior, output information to notify that the inappropriate behavior has been detected.

Images