Evaluation device, evaluation method, and program

Abstract

The present disclosure is an evaluation device that evaluates the level of satisfaction a person belonging to a predetermined organization has with the organization, the evaluation device comprising: an acquisition unit that acquires posted information indicating the content of postings to a community site within the predetermined organization; an item extraction unit that performs sentence delimitation on the posted information by natural language processing, and extracts, from each sentence delimited by the sentence delimitation, items relating to a satisfaction level and being set according to evaluation items on a job search site; an emotion type derivation unit that analyzes the emotion type of each delimited sentence by emotion analysis through natural language processing, and derives the number of occurrences of each emotion type for each item relating to the satisfaction level; and a satisfaction level score calculation unit that calculates a score for each emotion type for each item relating to the satisfaction level, on the basis of the number of occurrences of each emotion type for each item relating to the satisfaction level, and calculates a satisfaction level score, which is the sum of the calculated scores.

Description

Evaluation device, evaluation method, and program

[0001] The present disclosure relates to a technique for evaluating the degree of satisfaction of people who belong to a given organization with the organization.

[0002] In recent years, companies have been expected to fulfill their social responsibilities and improve their corporate value by engaging in ESG (Environment, Social, and Governance) initiatives. ESG is an acronym that combines the first letters of the English words E (Environment), S (Social), and G (Governance).

[0003] Furthermore, currently, evaluations of corporate ESG initiatives are only conducted by rating agencies (investment institutions, securities companies, etc.), and are based on the values ​​of stakeholders. These evaluations are made from an investment perspective, including evaluations from customers, suppliers, the local community, and society. Various technologies are also being considered that utilize artificial intelligence (AI) to incorporate evaluations from various stakeholders (Non-Patent Documents 1 and 2).

[0004] Henrik Skaug Satra, "The AI ​​ESG protocol: Evaluating and disclosing the environmental, social, and governance implications of artificial intelligence capabilities, assets, and activities," Sustainable Development, Volume 31, Issue 2, https: / / doi.org / 10.1002 / sd.2438, October 2022. Yuriko Nakao, "Research on the evaluation of ESG information using AI," Bulletin of the Faculty of Information Science, Kansai University, December 20, 2022. <https: / / kansai-u.repo.nii.ac.jp / ?action=repository_action_common_download&item_id=24144&item_no=1&attribute_id=19&file_no=1>

[0005] However, these previous studies are based on numerical calculations made on information disclosed by companies and other organizations (environmental reports, CSR reports, CO2 emissions, environmental contribution activities, the proportion of female managers, diversity, and other numerical data), and therefore are unable to evaluate satisfaction levels that are close to the actual situation within the organization.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to perform an evaluation of satisfaction that is closer to the actual situation within an organization than conventional methods.

[0007] In order to achieve the above object, the invention of claim 1 is an evaluation device for evaluating the satisfaction of a person belonging to a specified organization with the organization, the evaluation device comprising: an acquisition unit that acquires posted information indicating the content of posts to a community site within the specified organization; an item extraction unit that performs sentence segmentation on the posted information by natural language processing and extracts items related to the satisfaction set by the evaluation items on the job change site from each sentence separated by the sentence segmentation; an emotion type derivation unit that analyzes the emotion type of each separated sentence by emotion analysis in natural language processing and derives the number of each emotion type for each of the satisfaction-related items; and a satisfaction score calculation unit that calculates a score for each emotion type for each of the satisfaction-related items based on the number of each emotion type for each of the satisfaction-related items, and calculates a satisfaction score that is the sum of the scores.

[0008] As described above, the present disclosure has the effect of enabling evaluation of satisfaction levels that are closer to the actual situation within an organization than conventional methods.

[0009] FIG. 1 is an overall configuration diagram of a communication system according to an embodiment. FIG. 2 is an electrical hardware configuration diagram of an evaluation device according to an embodiment. FIG. 3 is an electrical hardware configuration diagram of a server according to an embodiment. FIG. 4 is a diagram illustrating the functional configuration of the evaluation device. FIG. 5 is a diagram illustrating an example of posted information acquired by an acquisition unit. FIG. 6 is a conceptual diagram illustrating results extracted by an item extraction unit as delimited sentences and items related to satisfaction. FIG. 7 is a conceptual diagram illustrating emotion types derived for each satisfaction item by an emotion type derivation unit from the results extracted by the item extraction unit. FIG. 8 is a conceptual diagram illustrating results calculated by a satisfaction score calculation unit as a score for each satisfaction item and a satisfaction score which is the sum of the scores for each item. FIG. 9 is a flowchart illustrating processing or operations executed by the evaluation device.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [System Configuration of the Embodiment] First, the overall configuration of a communication system according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to the embodiment.

[0012] 1, a communication system 10 of this embodiment is constructed by an evaluation device 30 and a server 50 that operates a community site such as an in-house SNS (Social Networking Service). The evaluation device 30 and the server 50 can communicate with each other via a communication network 100 such as a LAN (Local Area Network) or the Internet.

[0013] The Internet includes satellite Internet. The communication network 100 may include a dedicated network such as an ISP (Internet Service Provider) network managed and / or operated by a telecommunications carrier. The connection form of the communication network 100 may be either wireless or wired.

[0014] The evaluation device 30 is configured by one or more computers. When the evaluation device 30 is configured by multiple computers, it may be referred to as an "evaluation device" or an "evaluation system."

[0015] The evaluation device 30 quantitatively evaluates the degree of satisfaction of people who belong to a predetermined organization such as a company with (regarding) the organization.

[0016] The server 50 is configured by one or more computers. The server 50 can store and publish content posted to the community site.

[0017] [Hardware Configuration] <Hardware Configuration of Evaluation Device> Next, the electrical hardware configuration of the evaluation device 30 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the evaluation device.

[0018] As shown in FIG. 2 , the evaluation device 30 is a computer and includes a CPU 301, a ROM 302, a RAM 303, an SSD 304, an external device connection I / F (Interface) 305, a network I / F 306, a display 307, an operation unit 308, a media I / F 309, and a bus line 510.

[0019] Of these, the CPU 301 controls the overall operation of the evaluation device 30. The ROM 302 stores programs such as IPL used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301.

[0020] The SSD 304 reads or writes various data under the control of the CPU 301. Note that instead of the SSD 304, a hard disk drive (HDD) may be used.

[0021] The external device connection I / F 305 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB memory, and a printer.

[0022] The network I / F 306 is an interface for performing data communication via the communication network 100 .

[0023] The display 307 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays various images.

[0024] The operation unit 308 is a keyboard, a pointing device, or the like, and is an example of an input unit for accepting input operations such as characters, numbers, and various instructions.

[0025] The media I / F 309 controls reading and writing (storing) of data from and to a recording medium 309m such as a flash memory, etc. The recording medium 309m includes DVDs and Blu-ray Discs (registered trademarks).

[0026] The bus line 310 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 301 shown in FIG.

[0027] <Hardware Configuration of Server> Next, the electrical hardware configuration of the server 50 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the electrical hardware configuration of the server.

[0028] As shown in FIG. 3, the server 50 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an SSD (Solid State Drive) 504, an external device connection I / F (Interface) 505, a network I / F 506, a media I / F 509, and a bus line 310.

[0029] Of these, the CPU 501 controls the overall operation of the server 50. The ROM 502 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501.

[0030] The SSD 504 reads or writes various data under the control of the CPU 501. Note that instead of the SSD 504, a hard disk drive (HDD) may be used.

[0031] The external device connection I / F 505 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0032] The network I / F 506 is an interface for performing data communication via the communication network 100 .

[0033] The media I / F 509 controls reading and writing (storing) of data from and to a recording medium 509m such as a flash memory, etc. The recording medium 509m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0034] The bus line 510 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 501 shown in FIG.

[0035] 4, the evaluation device 30 has an acquisition unit 31, an item extraction unit 32, an emotion type derivation unit 33, a satisfaction score calculation unit 34, and an output unit 35. Each of these units has a function realized by an instruction from the CPU 301 in FIG. 2 based on a program.

[0036] The acquisition unit 31 acquires, from the server 50, posted information indicating the content posted to the community site.

[0037] The item extraction unit 32 performs preprocessing and sentence segmentation in natural language processing on the posted information as text acquired from the acquisition unit 31. The preprocessing includes cleaning, such as removing noise from the text, and normalization, such as standardizing full-width characters, half-width characters, uppercase characters, and lowercase characters. The sentence segmentation is a process of detecting and dividing sentences.

[0038] 5 is a diagram illustrating an example of posted information acquired by the acquisition unit. In the posted information shown in FIG. 5, sentences are segmented into "The company's salary is high," "The overtime premium is high, too," "I don't agree with their inappropriate waste disposal," "They regularly clean up around the company to contribute to the community, but," and "There is still a culture of discrimination against women, so I feel like it's a Showa-era company."

[0039] Furthermore, the item extraction unit 32 performs natural language processing to extract sentence features (distributed representation) for each of the separated sentences, calculates the cosine similarity with the items (names) related to the satisfaction of people belonging to the organization, and extracts items from each separated sentence whose similarity to the items (names) related to the satisfaction of people belonging to the organization is higher than a threshold. In this case, the item extraction unit 32 temporarily stores the items whose similarity is higher than the threshold in RAM 303 or the like as items related to the satisfaction of people belonging to the organization. The item extraction unit 32 acquires in advance the evaluation items on the job-seeking site and the predetermined items in ESG social, and selects from these items as items related to the satisfaction of people belonging to the organization. Note that either the evaluation items on the job-seeking site or the predetermined items in ESG social may be used, rather than both. In other words, at least one of the evaluation items on the job-seeking site and the predetermined items in ESG social may be used.

[0040] 6 is a conceptual diagram showing the results extracted by the item extraction unit as delimited sentences and items related to satisfaction. Evaluation items on a job-hunting site include, for example, "salary," "paid vacation," and "future prospects of the organization." Predetermined items in the Social section of ESG include, for example, "social contribution" and "gender-free."

[0041] The emotion type derivation unit 33 analyzes each sentence separated by the item extraction unit 32 using emotion analysis in natural language processing to determine whether the meaning of the sentence corresponds to a positive (P), neutral (-), or negative (N) emotion type. Furthermore, the emotion type derivation unit 33 derives the number of emotion types for each satisfaction-related item from the emotion analysis results. In this case, a method such as BERT (Bidirectional Encoder Representations from Transformers) is used for the emotion analysis. Following the item extraction unit 32, the emotion type derivation unit 33 temporarily stores the emotion types for each separated sentence and satisfaction-related item in the RAM 303 or the like, as shown in FIG. 7 .

[0042] 7 is a conceptual diagram showing the results extracted by the item extraction unit as a segmented sentence and items related to satisfaction. For example, if the segmented sentence contains the phrase "the salary is high," the emotion type derivation unit 33 analyzes the emotion type of the satisfaction-related item "salary" as positive (P). Also, if the segmented sentence contains the phrase "the overtime pay is also high," the emotion type derivation unit 33 analyzes the emotion type of the satisfaction-related item "salary" as positive (P). As a result, the emotion type derivation unit 33 derives the number of emotion types "P" for the satisfaction-related item "salary" as "2."

[0043] On the other hand, if the delimited sentence contains the phrase "I don't agree," the emotion type derivation unit 33 analyzes the emotion type of the satisfaction-related item "social contribution" as negative (N). Furthermore, if the delimited sentence contains the phrase "contributing," the emotion type derivation unit 33 analyzes the emotion type of the satisfaction-related item "social contribution" as positive (P). As a result, the emotion type derivation unit 33 derives the number of emotion types "N" for the satisfaction-related item "social contribution" as "1," and also derives the number of emotion types "P" for the satisfaction-related item "social contribution" as "1."

[0044] Furthermore, if the separated sentence contains the phrase "the culture of discrimination against women remains," the emotion type derivation unit 33 analyzes the emotion type of the satisfaction-related item "gender-free" as negative (N).

[0045] As shown in FIG. 8, the satisfaction score calculation unit 34 calculates the satisfaction score by using a coefficient K i The score for each emotion type is calculated by multiplying the number of each emotion type for each item derived by the emotion type derivation unit 33 by . FIG. 8 is a conceptual diagram showing the results of calculation by the satisfaction score calculation unit of the score for each satisfaction item and the satisfaction score, which is the sum of the scores for each item. Here, as an example, each coefficient is set so that the sum of the coefficients is "1," but this is not limited to this. Note that the satisfaction score calculation unit 34 may calculate the score for each emotion type without using a coefficient. Furthermore, the "satisfaction-related items" shown in FIG. 8 are only an example, and items other than those shown in FIG. 8 (e.g., "work-life balance") may be used. If other items are used, some or all of the items shown in FIG. 8 may not be used.

[0046] In Fig. 8, in the case of a positive (P: Positive) result, "1" is multiplied, and in the case of a negative (N: Negative) result, "-1" is multiplied. Note that in the case of a neutral (-: Neutral) result, "0" is multiplied, and therefore is not shown in Fig. 8.

[0047] For example, if the coefficient of the satisfaction item "salary" is "0.3" and the number of emotion types with a positive "1" is "2," the emotion type score will be 0.6 (= 0.3 x 1 x 2). Also, if the coefficient of the satisfaction item "social contribution" is "0.2" and the number of emotion types with a positive "1" is "1," the emotion type score will be 0.2 (= 0.2 x 1 x 1). However, even if the coefficient of the satisfaction item "social contribution" is "0.2," if the number of emotion types with a negative "-1" is "1," the emotion type score will be -0.2 (= 0.2 x -1 x 1).

[0048] The satisfaction score calculation unit 34 further calculates a satisfaction score, which is the sum of the scores of each emotion type. In Figure 8, the satisfaction score calculation unit 34 calculates the satisfaction score as 0.5 (= 0.6 + 0.2 - 0.2 - 0.1).

[0049] The output unit 36 ​​outputs the satisfaction score calculated for each organization by the satisfaction score calculation unit 34. The output method may be output to a printer or the like via the external device connection I / F 305, output to an external computer via the network I / F 306, or output and display on the display 307.

[0050] [Processing or Operation of Evaluation Device] Next, processing or operation executed by the evaluation device 30 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing processing or operation executed by the evaluation device.

[0051] S11: Posting information indicating the content posted to the community site is acquired from the server 50.

[0052] S12: Preprocessing and sentence segmentation in natural language processing are performed on the posted information as text acquired from the acquisition unit 31. Furthermore, the item extraction unit 32 performs feature extraction (distributed representation) of the sentences in natural language processing on the segmented sentences to calculate cosine similarity with items related to the satisfaction of people who belong to the organization, and extracts items (names) from the segmented sentences whose similarity to items related to the satisfaction of people who belong to the organization is higher than a threshold.

[0053] S13: The emotion type derivation unit 33 analyzes the meaning of the sentence separated by the item extraction unit 32 by emotion analysis in natural language processing to determine whether the emotion type is positive (P), neutral (-), or negative (N). Furthermore, the emotion type derivation unit 33 derives the number of each emotion type for each satisfaction-related item from the result of the emotion analysis.

[0054] S14: The satisfaction score calculation unit 34 calculates the score for each emotion type by multiplying the coefficient set for each satisfaction item by the number of each emotion type for each item derived by the emotion type derivation unit 33. The satisfaction score calculation unit 34 further calculates a satisfaction score, which is the sum of the scores for each emotion type.

[0055] S15: The output unit 36 ​​outputs the satisfaction score for each organization.

[0056] [Major Effects of the Embodiment] As described above, according to the present embodiment, by using a community site such as an in-house SNS (Social Networking Service) as the source of information and by using satisfaction-related items set by at least the evaluation items on the job-changing site from among the evaluation items on the job-changing site and the predetermined items in ESG Social, it is possible to evaluate satisfaction within an organization that is closer to the actual situation than satisfaction evaluations using conventional methods that use information disclosed by the organization as satisfaction-related items.

[0057] Furthermore, methods such as questionnaire surveys are used to evaluate the satisfaction of people (employees, etc.) belonging to an organization, but a problem is that it takes time to analyze the survey results. Also, in questionnaire surveys, employees, etc. may not be able to express their opinions frankly because respondents are defensive and do not express their true feelings. In contrast, in this embodiment, by analyzing posted information acquired from a community site such as an in-house SNS (Social Networking Service), it is possible to evaluate the satisfaction of employees, etc., more quickly and more easily than conventional methods that use questionnaire surveys, and which allow employees, etc. to express their true feelings.

[0058] [Supplementary Note] The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations), for example.

[0059] (1) The evaluation device 30 and the server 50 can be realized by a computer and a program, and the program can be recorded on a (non-transitory) recording medium or provided via a communication network 100 such as the Internet.

[0060] (2) The CPUs 301 and 501 serving as processors may each be single or multiple.

[0061] 10 Communication system 30 Evaluation device 50 Server 31 Acquisition unit 32 Item extraction unit 33 Emotion type derivation unit 34 Satisfaction score calculation unit 35 Output unit

Claims

1. An evaluation device for evaluating the satisfaction of a person belonging to a specific organization with the organization, comprising: an acquisition unit that acquires posted information indicating the content of posts to a community site within the specific organization; an item extraction unit that performs sentence division on the posted information by natural language processing and extracts items related to the satisfaction set by the evaluation items on the job change site from each sentence separated by the sentence division; an emotion type derivation unit that analyzes the emotion type of each separated sentence by emotion analysis in natural language processing and derives the number of each emotion type for each item related to the satisfaction; and a satisfaction score calculation unit that calculates a score for each emotion type for each item related to satisfaction based on the number of each emotion type for each item related to satisfaction, and calculates a satisfaction score that is the sum of the scores.

2. The evaluation device according to claim 1, wherein the items relating to satisfaction are set based on evaluation items on the job-changing site and predetermined items in ESG social.

3. The evaluation device according to claim 1, wherein the satisfaction score calculation unit calculates a score for each emotion type for each satisfaction item by multiplying the number of each emotion type for each satisfaction item by a coefficient that is preset for each satisfaction item.

4. The evaluation device according to any one of claims 1 to 3, wherein the items related to satisfaction indicate salary, paid vacation, the future prospects of the organization, social contribution, or gender-free.

5. An evaluation device for evaluating the satisfaction of a person belonging to a specific organization with the organization, the evaluation device performing an evaluation method comprising: an acquisition process for acquiring posted information indicating the content posted to a community site within the specific organization; an item extraction process for dividing the posted information into sentences by natural language processing and extracting items related to the satisfaction set by the evaluation items on the job change site from each sentence separated by the sentence separator; an emotion type derivation process for analyzing the emotion type of each separated sentence by emotion analysis in natural language processing and deriving the number of each emotion type for each item related to the satisfaction; and a satisfaction score calculation process for calculating a score for each emotion type for each item related to satisfaction based on the number of each emotion type for each item related to satisfaction, and calculating a satisfaction score that is the sum of the scores.

6. A program for causing a computer to execute the method according to claim 5.

Images