Program, method, and information processing apparatus

Abstract

To provide a technique for facilitating overcoming challenges in carrying out operations in an organization composed of multiple human resources, in accordance with actual behaviors of users.SOLUTION: A program for operating a computer causes a processor of the computer to execute the steps of: presenting, to a user, a situation related to unconscious bias and presenting a screen for receiving an input of an answer about an action of the user in the situation; receiving, from the user, an input of a free-text answer; analyzing the free-text answer received from the user and evaluates a degree of unconscious bias of the user; and presenting an evaluated result to the user.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, and an information processing device. [Background technology]

[0002] It is becoming increasingly important for organizations engaged in business activities to achieve organizational diversity. For example, an organization that is expected to generate innovation is one in which people with various backgrounds come together and can work smoothly. Specifically, it is desirable to have gender diversity and to be able to overcome the challenges that come with bringing together people with these different backgrounds to work together.

[0003] In order to carry out business activities, employees may undergo training to acquire the necessary knowledge and carry out their work, which may change their thinking patterns.

[0004] Patent Document 1 describes a technology for the early diagnosis and prevention of mental illness, which "has the objective of contributing to mental health care and welfare by encouraging potential clients who have not yet developed symptoms or visited a mental health institution to seek mental health care through a self-diagnosis program in an integrated environment that combines the Internet and software, and by using self-treatment software to bring about changes in the client's thought patterns without the intervention of a counselor or psychiatrist, thereby providing a wide range of early diagnosis of mental illness and primary and secondary prevention of illness."

[0005] According to the technology in Patent Document 1, this has the effect of "maintaining anonymity because it is limited to self-diagnosis and self-preventive psychological education, and the exchange of personal privacy is kept to a minimum." [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-004398 Summary of the Invention [Problem to be solved by the invention]

[0007] In the case of the technology of Patent Document 1, in which answer options are prepared in advance and presented to the user, and then the user is asked to select one option, the options are presented without the user having had a chance to think about the actions that the user would actually take in a specific situation, such as what to say to a subordinate, etc. The actions that the user will take are not evaluated based on the results of the user's own thoughts.

[0008] There is a need for technology that makes it easier to overcome the challenges of working in an organization with multiple people, while taking into account the actual actions of the users themselves. [Means for solving the problem]

[0009] According to one embodiment of the present disclosure, a program for operating a computer is provided that causes a processor of the computer to perform the following steps: presenting a situation related to an unconscious belief to a user and presenting a screen for accepting input of an answer as to what action the user will take in response to the situation; accepting input of the answer in free text from the user; analyzing the free text answer accepted from the user to evaluate the degree of the user's unconscious belief; and presenting the evaluated result to the user. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to more easily overcome the challenges involved in carrying out work in an organization with multiple personnel. [Brief explanation of the drawings]

[0011] [Figure 1]FIG. 1 is a diagram showing the configuration of the system 1. [Figure 2] FIG. 2 is a diagram showing the configuration of the server 20. As shown in FIG. [Figure 3] FIG. 3 is a diagram showing the configuration of the terminal 10. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing the data structure of the user database 211. As shown in FIG. [Figure 5] FIG. 5 is a diagram showing the data structure of the question database 212. As shown in FIG. [Figure 6] FIG. 6 is a diagram showing the data structure of the prompt database 213. As shown in FIG. [Figure 7] FIG. 7 is a diagram showing the data structure of the answer history database 216. As shown in FIG. [Figure 8] FIG. 8 is a diagram showing the data structure of the training content attendance history database 217. [Figure 9] FIG. 9 is a diagram showing the flow of a process for accepting a user's response to a situation presented to the user and presenting an evaluation result for encouraging the user to change his or her behavior. [Figure 10] FIG. 10 is a diagram showing the flow of processing for presenting evaluation results in response to an inquiry from another service. [Figure 11] FIG. 11 shows an example of a screen for presenting a question to the user and receiving an answer from the user. [Figure 12] FIG. 12 shows an example of a screen for displaying the results of evaluating the user's input content. [Figure 13] FIG. 13 shows an example of a screen for presenting evaluation results based on multiple evaluation axes and training content. [Figure 14] FIG. 14 shows an example of a screen for evaluating a user based on data from other services and presenting the results. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0013] <Outline of embodiment> <1.1 Overall system configuration> FIG. 1 is a diagram showing the configuration of the system 1.

[0014] System 1 is a system that supports organizations such as business corporations, which have one or more employees and are expected to create innovation (diverse organizations). In aiming to become such an organization, the degree of achievement can be estimated along the following axes. Promoting diversity: There is little gender disparity in the ratio of managers (typically, the ratio of women in managerial positions is used as an indicator) There is little gender difference in the rate of parental leave taken (typically, the rate of parental leave taken by men is used as an indicator) Improved employee engagement As mentioned above, one of the factors that hinders the promotion of diversity and the transition to a state of improved engagement is the unconscious assumptions (unconscious bias) held by human resources. For example, gender bias can include unconscious assumptions that make it difficult for women to work, and unconscious assumptions that prevent men from taking parental leave (such as assumptions about roles within the home). There are also unconscious assumptions based on roles, such as position, within a business company, and there can be unconscious biases on the part of managers and subordinates (for example, managers should be like this, subordinates should be like that).

[0015] Therefore, by providing an opportunity for employees to become aware of these unconscious assumptions, it is possible to encourage behavioral change among the employees of business companies. Furthermore, by providing training, behavioral change can be further promoted.

[0016] 1 includes a check service server 20, an employee terminal 10, a large-scale language model service server 92, a training service server 94, a human resources and labor service server 96, a messaging service server 98, and a human resources personnel terminal 12. These devices are connected to each other via a network 80.

[0017] In the illustrated example, one terminal 10 is shown as the terminal used by users of the check service provided by the server 20, and each user operates their own terminal.

[0018] In this embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, a collection of devices can be considered as one "information processing device," and system 1 can be formed as a collection of multiple devices. The way in which multiple functions required to realize system 1 according to this embodiment are allocated to one or multiple pieces of hardware can be determined appropriately in consideration of the processing capacity of each piece of hardware and / or the specifications required for system 1.

[0019] The terminal 10 is a device operated by a user. The terminal 10 is realized, for example, as follows. · Handheld devices such as smartphones and tablets Desktop PCs (Personal Computers), laptop PCs Wearable devices worn by users (wristwatches, glasses, etc.) The terminal 10 includes a communication IF (Interface) 12 , an input device 13 , an output device 14 , a memory 15 , a storage 16 , and a processor 19 .

[0020] The communication IF 12 is an interface for inputting and outputting signals so that the terminal 10 can communicate with an external device.

[0021] The input device 13 is a device for receiving input operations from a user (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).

[0022] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0023] The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0024] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0025] The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0026] The server 20 is a device for providing users with a service for checking their unconscious beliefs.

[0027] The server 20 includes a communication IF 22 , an input / output IF 23 , a memory 25 , a storage 26 , and a processor 29 .

[0028] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0029] The input / output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user.

[0030] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0031] The storage 26 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0032] The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0033] The large-scale language model service server 92 is a server that executes language processing tasks using a language model constructed by a learning process including artificial intelligence (AI). An LLM (Large Language Model) is a server that has previously learned a large amount of large-scale data (text data, etc.), for example, web content on the Internet, or a large amount of data stored in a predetermined database, and can execute various language processing tasks by giving it tasks.

[0034] The large-scale language model service server 92 accepts prompt inputs such as text, images, and voice, and generates and responds to the prompts. Examples of LLMs include GPT-3 and GPT-4 developed by OpenAI, and BERT developed by Google. In this embodiment, the large-scale language model service server 92 may use an LLM that has been trained in advance using training content on the following topics, documents published on the Internet, or a database of case studies (cases where problems have occurred, their improvement measures, and exemplary behaviors) accumulated within a business company. - To raise awareness of unconscious biases (unconscious assumptions). Examples of undesirable communication situations resulting from the imposition of unconscious assumptions. It may also include ways to deal with such cases (such as examples of desirable responses). - Understanding employees' needs. Examples of problems arising from not understanding employees' needs (for example, problems arising from not recognizing a request to take parental leave to spend time with family) and how to deal with them. Diversity-related issues: Examples of problems arising from not embracing the importance of diversity (for example, problems arising from gender bias and preconceived notions about how certain genders should be) and how to address them. The training service server 94 is a system that stores training content and provides the training content in response to user operations.

[0035] The personnel and labor service server 96 is a system that provides employee management and labor management services.

[0036] The personnel and labor service server 96 has a list of employees and manages information such as joining, resignation, arrival time, work schedule, and vacation application.

[0037] The messaging service server 98 is a system that supports communication between users.

[0038] The messaging service server 98 accepts posts of text, audio, images, etc., and provides a web system and email system that can be shared both inside and outside the business company.

[0039] The personnel manager's terminal 12 is a terminal operated by the personnel manager.

[0040] <1.2 Functional configuration of server 20> 2 is a diagram showing the configuration of the server 20. As shown in FIG. 2, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0041] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0042] The storage unit 202 stores various databases such as a user database 211, a question database 212, a prompt database 213, an answer history database 216, and a training content attendance history database 217.

[0043] The user database 211 is a database for managing users.

[0044] The user database 211 contains information such as the user's name, employee number, etc., for using the services provided by the server 20. Details will be described later.

[0045] The question database 212 is a database for managing questions to be presented to the user. The question database 212 has, for example, questions for evaluating the following various attributes, which will be described in detail later. · Matters related to the imposition of unconscious beliefs · Understanding employee needs · Understanding diversity

[0046] The prompt database 213 is a database for managing prompt templates to be sent to the large-scale language model service server 92 based on the user's answers to questions. The prompt database 213 manages, for example, the prompt text, etc. Details will be described later.

[0047] The answer history database 216 is a database for managing user answers to questions and the evaluation results by the large-scale language model service server 92. The answer history database 216 manages, for example, the date and time when the evaluation was performed, the evaluation results such as the score generated by the large-scale language model service server 92, etc. Details will be described later.

[0048] The training content attendance history database 217 is a database for managing the history of training sessions taken by users. For example, the training content attendance history database 217 manages the date and time when the training content was taken, the evaluation results that prompted the user to take the training content, etc. Details will be described later.

[0049] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. By operating in accordance with the program, the control unit 203 performs functions indicated as a reception control module 2041, a transmission control module 2042, a user management module 2043, a question presentation module 2044, a prompt creation module 2045, and an evaluation result presentation module 2046.

[0050] The reception control module 2041 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0051] The transmission control module 2042 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0052] The user management module 2043 is a module for managing information about each user who uses the system 1. Specifically, the user management module 2043 accepts registration of information about each user and updates the user database 211.

[0053] The question presentation module 2044 is a module that performs processing to present a question to the user based on the question database 212 and receive an answer from the user.

[0054] The prompt generation module 2045 is a module that performs processing to generate a prompt to be sent to the service server 92 for the large-scale language model, based on the answer received from the user, by referring to the prompt database 213 .

[0055] The evaluation result presentation module 2046 is a module that receives the evaluation results from the large-scale language model service server 92 and performs processing to present the evaluation results to the user.

[0056] <1.3 Configuration of Terminal 10> FIG. 3 is a diagram showing the configuration of the terminal 10. As shown in FIG.

[0057] As shown in FIG. 3, the terminal 10 includes multiple antennas (antenna 111, antenna 112), communication units (first communication unit 120, second communication unit 121) corresponding to the respective antennas, an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a memory unit 180, and a control unit 190. The terminal 10 also has functions and configurations (e.g., a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not specifically shown in FIG. 3. As shown in FIG. 3, the blocks included in the terminal 10 are electrically connected by a bus or the like.

[0058] The antenna 111 emits a signal emitted by the terminal 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first communication unit 120.

[0059] The antenna 112 emits a signal emitted by the terminal 10 as a radio wave. The antenna 112 also receives a radio wave from space and provides the received signal to the second communication unit 121.

[0060] The first communication unit 120 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal 10 can communicate with other wireless devices. The second communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal 10 can communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are communication modules including a tuner, a received signal strength indicator (RSSI) calculation circuit, a cyclic redundancy check (CRC) calculation circuit, a high-frequency circuit, and the like. The first communication unit 120 and the second communication unit 121 perform modulation / demodulation, frequency conversion, and the like for wireless signals transmitted and received by the terminal 10, and provide the received signals to the control unit 190.

[0061] Input device 130 has a mechanism for accepting input operations by a user. Specifically, input device 130 is configured as a touch screen and includes touch-sensitive device 131. Touch-sensitive device 131 accepts input operations by a user of terminal 10. Touch-sensitive device 131 detects the user's touch position on the touch panel, for example, by using a capacitive touch panel. Touch-sensitive device 131 outputs a signal indicating the user's touch position detected by the touch panel to control unit 190 as an input operation.

[0062] The display 132 displays data such as images, videos, and text under the control of the control unit 190. The display 132 is realized by, for example, an LCD, an organic EL display, or the like.

[0063] The audio processing unit 140 modulates and demodulates audio signals. The audio processing unit 140 modulates a signal provided from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized, for example, by a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal 10.

[0064] The location information sensor 150 is a sensor that detects the location of the terminal 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. The satellite positioning system receives signals from at least three or four satellites, and detects the current location of the terminal 10 equipped with the GPS module based on the received signals.

[0065] The camera 160 is a device that receives light with a light receiving element and outputs the received light as a captured image. The camera 160 is, for example, a depth camera that can detect the distance from the camera 160 to a subject being photographed.

[0066] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10 .

[0067] The storage unit 180 is configured with, for example, a flash memory or the like, and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.

[0068] The control unit 190 controls the operation of the terminal 10 by reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 is, for example, an application processor. By operating in accordance with the program, the control unit 190 fulfills the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a storage control unit 195.

[0069] Operation acceptance unit 191 performs processing to accept a user's input operation to an input device such as touch-sensitive device 131. Operation acceptance unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation, based on information about the coordinates where the user has touched touch-sensitive device 131 with a finger or the like.

[0070] The transmitting / receiving unit 192 performs processing for the terminal 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.

[0071] The data processing unit 193 performs calculations on data that the terminal 10 has received as input in accordance with a program, and outputs the calculation results to a memory or the like.

[0072] The notification control unit 194 performs processing for displaying a display image on the display 132, processing for outputting sound from the speaker 142, and processing for generating vibrations.

[0073] The storage control unit 195 controls the storage of data in the storage unit 180 .

[0074] A description will be given of various types of information stored in storage unit 180. In one aspect, storage unit 180 stores various types of information such as user information 181, answer history information 182, training attendance history information 183, and the like.

[0075] The user information 181 is information about a user who uses the services of the server 20. The user information 181 includes, for example, information such as the employee number and name of the user.

[0076] The answer history information 182 is information indicating the history of answers to questions using the services of the server 20.

[0077] The training attendance history information 183 is information indicating the history of training that the user has attended using the services provided by the training service server 94, etc.

[0078] <2 Data Structure> 4 is a diagram showing the data structure of the user database 211. The user database 211 includes the following items: "User ID," "Name," "Employee Number," "Department," "Position," "Superior," "Subordinate," "Joining Date," "Resignation Date," "Number of Working Days," and "Work Style." By communicating with the personnel and labor service server 96, the server 20 can obtain information about each employee managed by the personnel and labor service server 96 and update the user database 211.

[0079] The item "user ID" is information for identifying a user.

[0080] The item "Name" is information about the user's name.

[0081] The item "employee number" is information about the employee number of the user.

[0082] Specifically, the item "employee number" is information used to identify an employee when a user logs in to use a service provided by the server 20.

[0083] The item "department" is information about the division or department to which the user belongs in an organization such as a business company.

[0084] The item "position" is information indicating the position of the user in the organization to which the user belongs.

[0085] The item "Superior" is information indicating a user who is at a higher level (for example, a direct superior) than the user in the chain of command of the organization.

[0086] The item "Subordinate" is information indicating a user who is at a lower level (for example, a direct subordinate) from the user's position in the chain of command of the organization.

[0087] The item "time of joining" is information indicating when the user joined an organization such as a business company (for example, when the user became an employee).

[0088] The item "time of retirement" is information indicating the time when the user no longer belonged to an organization such as a business company (for example, the employment relationship ended, or the user was no longer a contract worker).

[0089] The item "number of working days" is information indicating the amount of work a user performs (number of days) in a certain period (for example, weekly).

[0090] The item "work style" is information indicating the manner in which a user performs work.

[0091] Specifically, the item "work style" includes the following information as the manner in which a user performs work. · Will you come to the office? Remote work? · Whether you work in the office and remotely at the same time (breakdown of the number of days)

[0092] 5 is a diagram showing the data structure of question database 212. Question database 212 includes the following items: "Question ID," "Question Text," "Attributes," "Creator," "Creation Date," "Use Start Date," "Use End Date," "Related Laws and Regulations," and "Usability." Server 20 updates question database 212, for example, when a business providing the service of server 20 or a user adds a question.

[0093] The item "Question ID" is information that identifies each question to which an answer is to be received from the user.

[0094] The item "question text" is information indicating the content of the question.

[0095] Specifically, the item "question sentence" includes the following as the content of the question: - Images related to the question (for example, an image showing a situation where a subordinate consulted with a superior) Text and audio showing the content of the question (for example, text showing the content of a consultation from a subordinate) For example, the following questions may be asked: Example question (1) Background of the question: "A female employee has come to you as her supervisor with the following question. Please provide the answer that you think is best for her, her subordinates, the company, etc." Question: "I'd like to ask you something. Our current project is about to launch, and I understand that this is a very difficult time. I also know that you expect me to lead the project as the next manager. However, a family member needs medical treatment, so I would like to take two weeks off work. Is that okay?" Example Question (2) Background of the question: "A female employee has come to you as her supervisor with the following question. Please provide the answer that you think is best for her, her subordinates, the company, etc." Question: "My partner has been transferred, so I'm sorry, but I'm thinking of leaving my job. I'm just starting to get used to the job, so I'm sorry."

[0096] The item "attribute" is information indicating the attribute of the question. Specifically, the item "attribute" includes the following attributes: ·Improve the ratio of female managers ·Improve the rate of men taking childcare leave Unconscious bias of administrators Subordinates' unconscious bias Unconscious bias regarding family roles Unconscious gender bias

[0097] The item "creator" is information about the user who created the question. Specifically, the "Creator" item may include the identification information of the user who created the question, and for questions created by automatic generation, this may be treated as the creator.

[0098] The item "Date of Creation" is information indicating when the question was created.

[0099] Specifically, the item "Date of Creation" includes information on the timing when the question was created, for example, the date and time when the question was registered in the question database 212.

[0100] The item "use start date" is information indicating the timing at which questions will start being presented to users.

[0101] Specifically, the item "Start Date of Use" stores the value set as the start date of use when you want to set the time for questions to be asked at a specific time. For example, you may want to specify the time for questions to be asked in the following cases: - When operational rules and business practices are expected to change from a certain date due to legal amendments or the establishment of guidelines. When you want to focus your employees on checking for specific unconscious biases

[0102] The item "End of Use Date" is information indicating the timing at which questions will no longer be available to users. Specifically, the "End of Use Date" item stores the value set as the end of use date when you want to limit the time for sending questions to a certain date and end the service after that. For example, you may want to specify the time for stopping the sending of questions in the following cases: - When operational rules and business practices are expected to change from a certain date due to legal amendments or the establishment of guidelines, and the question is no longer appropriate after the changes. - When certain unconscious biases are excluded from employee checks

[0103] The item "Related Laws and Regulations" is information indicating laws, regulations, etc. related to the question. Specifically, the item "Relevant Laws and Regulations" may include information identifying guidelines from public agencies, laws, statements from industry associations, and the like.

[0104] The item "Availability" is information indicating whether or not the item is to be presented to the user as a question.

[0105] Specifically, the item "Availability" includes information on whether or not to allow use as a question. You may want to set it as not available if you want to narrow the scope of questions, if the content of the questions no longer conforms to guidelines, or if the number of times the question has been presented to users as a question exceeds a certain number (if it has been repeatedly presented as a question).

[0106] 6 is a diagram showing the data structure of the prompt database 213. The prompt database 213 includes the following items: a "prompt ID," a "prompt body," an "applicable attribute," a "prompt creation date," and a "prompt availability." The server 20 updates the prompt database 213, for example, when a service provider of the server 20 or a user adds a question.

[0107] The item "prompt ID" is information that identifies each prompt.

[0108] The item "prompt body" is information indicating the content of the prompt template.

[0109] Specifically, the item "prompt text" stores a prompt template to be combined with the user's answer to the question and sent to the large-scale language model service server 92. The prompt template includes the following information. Examples of prompt templates will be described later. -See user answers Constraints for evaluating user answers: Identifying the score In line with company policy The user's answer is not a string of gibberish Constraints for generating sentences based on the evaluation of user answers: Writing style (e.g., generate in the "desumasu" style) Whether to include a line break Words that are not included when generating a sentence (for example, "constraints" or "evaluation reasons" included in the prompt are not used when generating a sentence) -Generating model answers and their conditions: Generate in response to social needs Character limit The answer must be different from the one entered by the user. What not to include in a model answer Evaluation criteria: Evaluations should be generated based on multiple evaluation criteria. Specific examples of the evaluation criteria (for example, 1. Are you imposing unconscious assumptions? 2. Do you understand the requests of your subordinates? 3. Are you following the company's policies?) Company Policy ·Specify the format of the evaluation results: The order of the generated text (for example, the score for each evaluation axis, the reason for the evaluation, and the model answer) A concrete prompt template is, for example, as follows: In the prompt template below, variables are defined in the format "${variable name}". Specific examples of variables will be given later. "From the following conversation, evaluate the boss's answer so that it meets the constraints, and write the reasons for your evaluation and a model answer that also meets each of the constraints, and display them according to the format. ## Evaluation constraints If the text is a string of meaningless characters, all evaluation criteria will be assigned the lowest score of 1 point. Each evaluation axis is evaluated on a five-point scale, with 5 being the highest and 4, 3, 2, and 1 being the lowest. Consider whether you are taking ${QuestionerGroup} seriously ## Restrictions on evaluation reasons Start with "You" To be produced in the polite form Generated without line breaks Do not include the words "constraints" or "reason for evaluation" above. ## Model solution constraints Generate content that encourages people to respond to social demands as much as possible. Generate up to 200 characters Generate so that each evaluation axis is worth 5 points Generate answers that are different from your boss's Generated without line breaks Do not include the terms "constraints" or "model solution" above. ## Evaluation criteria 1. Are you imposing unconscious beliefs on others? 2. Do you understand your subordinates' needs? 3. Does it comply with company policy? ## Company Policy ${CompanyPolicies} ## Dialogue ${QuestionerGroup}: "${QuestionContent}" Boss: "${Answer}" ## Format Score for evaluation axis 1 * Score for evaluation axis 2 * Score for evaluation axis 3 * Reason for evaluation * Model answer " In the prompt template above, the following variables are defined: (1)${QuestionerGroup} The above variables indicate the attributes of the client. Specifically, the server 20 applies a value indicating the attribute of the questioner, such as "female employee," "male employee," or "senior male employee," to the variable ${QuestionerGroup} to generate a prompt. For example, the server 20 may store in advance a value to be applied to the variable ${QuestionerGroup} for each question, or may store a value to be applied to the variable ${QuestionerGroup} according to the item "attribute" in the question database 212. This allows the server 20 to apply a value to the variable ${QuestionerGroup} according to each question shown in the question database 212 and generate a prompt. (2) ${CompanyPolicies} The above variables indicate the company's policy. The server 20 applies a value indicating a company policy, such as "promotion of women in the company" or "increase in female managerial positions," to the above variable ${CompanyPolicies} to generate a prompt. For example, the server 20 may store in advance a value to be applied to the above variable ${CompanyPolicies} for each question, or may store a value to be applied to the above variable ${CompanyPolicies} according to the item "attribute" in the question database 212. (3)${QuestionContent} The above variables indicate the content of the consultation. Specifically, the server 20 applies the following values to the above variable ${QuestionContent} to generate a prompt. "I'd like to ask for your advice. Our current project is about to begin service, and I understand that it's a very difficult time. I also understand that you are expected to lead the project as the next manager. However, my child has become ill and will require long-term treatment, so I would like to take two weeks off work. Is that okay?" For example, the server 20 applies the value stored in the item "question text" in the question database 212 to the above variable ${QuestionContent} to generate a prompt. (4)${Answer} The variables above indicate the answer. The server 20 applies the answer received from the user (the "Answer" item in the answer history database 216) to the above variable ${Answer} to generate a prompt. For example, the answer received from the user may be something like the following: "Aiming for a managerial position isn't everything. However, there are some things that only managers can do. Why not learn about how managers work and think about your career together?" The server 20 uses a prompt template and adds the user's answer (in the above case, the answer as a supervisor), the content of the question (in the above case, the consultation content of the subordinate indicated in the question), and other necessary information (in the above case, text indicating company policy) to the prompt template, and creates a prompt to be sent to the large-scale language model service server 92. An example of the results of generating a prompt as described above and output by the large-scale language model service server 92 is as follows: Attributes of the client: "female employee" Question: "I'd like to ask you something... Our current project is about to launch, and I understand that this is a very difficult time. I also know that you expect me to lead the project as the next manager. However, a family member needs medical treatment, so I would like to take two weeks off work. Is that okay?" Response from a user: "That's tough, but I don't want you to take time off. Can you communicate with your husband and make sure he doesn't take time off?" Output from large-scale language model service server 92: "There is a problem with unconscious bias. The boss's response does not take into consideration the situation of female employees and shows a negative attitude toward taking vacation time. Also, the remark that makes it up to the husband to deal with the situation may indicate a stereotype regarding household roles."

[0110] The item "applicable attribute" is information indicating the attribute of the question to which the prompt is applied. Specifically, the item "applicable attribute" may correspond to the item "attribute" in the question database 212. This makes it possible to specify a template prompt to be applied to a user's answer to a question presented to the user, and to specify a prompt that matches the content of the question, making it easier to have the large-scale language model service server 92 generate an appropriate evaluation result.

[0111] The item "prompt creation date" is information indicating when the prompt template was created.

[0112] Specifically, the item "prompt creation date" includes information on the date and time when the prompt template was created, for example, the date and time when the question was registered in the prompt database 213.

[0113] The item "prompt usability" is information indicating whether or not the item is to be used as a prompt template.

[0114] Specifically, the item "prompt availability" includes information on whether or not the prompt template is available for use. For example, the large-scale language model service server 92 may update a prompt template to generate better evaluation results and stop using the previous prompt template.

[0115] 7 is a diagram showing the data structure of the answer history database 216. The answer history database 216 includes an item "answer ID," an item "user ID," an item "question ID," an item "answer content," an item "prompt," an item "LLM output result," an item "first evaluation axis," an item "first score," an item "advice," an item "model answer," an item "answer date and time," and an item "suggested content."

[0116] The item "Answer ID" is information for identifying each of the answer histories that associate answers received from a user when a question is presented to the user with the evaluation results of the service server 92 for the large-scale language model.

[0117] The item "user ID" is information that identifies each user.

[0118] Specifically, the item "user ID" may include information that identifies each user listed in the user database 211.

[0119] The item "Question ID" is information that identifies each question.

[0120] Specifically, the item “Question ID” may include information that identifies each question shown in the question database 212.

[0121] The item "answer content" is information indicating the content of the answer entered by the user in response to the question.

[0122] Specifically, the item "answer content" includes the following information as the answer entered by the user: - Free text entered by the user User-input speech (which may include accepting the results of automatic transcription as free text) - Images specified by the user (for example, still images or videos taken by the user on a device, capturing the user's facial expressions, voice, etc., may also be included, and information identifying the subject by taking these still images, etc. (such as the name of the object), and the subject's emotions (such as anger or joy) obtained by analyzing the subject's facial expressions, etc.)

[0123] The item "prompt" is information for identifying the prompt template applied when sending to the service server 92 of the large-scale language model. Specifically, the item "prompt" may include information identifying the template for each prompt as represented in prompt database 213.

[0124] The item "LLM output result" is information indicating the content output by the service server 92 of the large-scale language model by sending a prompt to the service server 92 of the large-scale language model.

[0125] The item "first evaluation axis" is information indicating an evaluation axis among the evaluation results to be output by the service server 92 of the large-scale language model.

[0126] Specifically, the item "first evaluation axis" indicates information for identifying one evaluation axis among multiple evaluation axes. The server 20 analyzes the output result of the large-scale language model service server 92 and identifies the part corresponding to the "first evaluation axis."

[0127] In the example shown in the figure, one evaluation axis out of multiple evaluation axes is shown, and information identifying other evaluation axes such as a "second evaluation axis" and a "third evaluation axis" is also stored in the response history database 216.

[0128] The item "first score" is information indicating the score corresponding to the evaluation axis among the evaluation results output by the service server 92 of the large-scale language model.

[0129] Specifically, in the illustrated example, the item "first score" indicates the score corresponding to the "first evaluation axis" among multiple evaluation axes. The server 20 analyzes the output result of the large-scale language model service server 92, identifies the portion corresponding to the "first evaluation axis," and identifies the score of the "first evaluation axis" (for example, identifies the score generated alongside the item name of the evaluation axis in the output result).

[0130] In addition, the answer history database 216 stores scores corresponding to other evaluation axes such as the “second evaluation axis” and the “third evaluation axis.”

[0131] The item "advice" is information indicating the advice portion for the user from the evaluation results output by the large-scale language model service server 92.

[0132] Specifically, in the illustrated example, the server 20 analyzes the output results of the large-scale language model service server 92 to identify the part of the "advice" item that corresponds to advice to the user (for example, by detecting the order of the output results, or whether the output results contain wording indicating the analysis results, such as "~~ has been achieved").

[0133] The item "model answer" is information indicating the part of the model answer that is to be generated by the service server 92 of the large-scale language model out of the evaluation results that are to be output by the service server 92.

[0134] Specifically, in the illustrated example, the item "Model Answer" is determined by the server 20 analyzing the output results of the large-scale language model service server 92, and identifying the part of the model answer for the user from, for example, the order of the output results and the content of the wording of the output results (for example, content that provides examples such as "The following are some possible responses").

[0135] The item "answer date and time" is information indicating the time when the user answered the question.

[0136] The item "Proposed Content" is information indicating training content proposed to the user.

[0137] Specifically, the item "Proposed Content" includes the results of the server 20 searching for training content on the training service server 94 based on the content shown in the item "LLM Output Results", the scores for each evaluation axis, etc. (for example, identifying training content corresponding to an evaluation axis with a low score).

[0138] 8 is a diagram showing the data structure of the training content attendance history database 217. The training content attendance history database 217 includes an item "attendance ID," an item "user ID," an item "content ID," an item "attendance date and time," and an item "answer history."

[0139] The item "attendance ID" is information that identifies each history of training content attended by a user.

[0140] The item "user ID" is information for identifying a user.

[0141] The item "content ID" is information for identifying the training content that the user has attended.

[0142] Specifically, the item "content ID" may be updated by receiving information on the training content attended by the user indicated by the user ID from the training service server 94. Although not shown, the training service server 94 may classify each training content according to its content and assign an attribute such as a tag (for example, "gender-specific unconscious bias").

[0143] The item "attendance date and time" is information indicating the time when the user attended the training content.

[0144] Specifically, the item "attendance date and time" indicates the date and time when the user attended the training content.

[0145] The item "answer history" is information that identifies each answer that triggered the user to take training content.

[0146] Specifically, the item "answer history" may include information identifying each answer shown in the answer history database 216. When a user answers a question and the evaluation results for the answer are presented, the proposed training content may be included. When the user specifies the training content on the screen that proposes the training content, the server 20 can associate the answer ID of the answer given by the user with information received from the training service server 94 about the training content taken by the user and store the association information in the training content attendance history database 217 or the like.

[0147] <3 operations> FIG. 9 is a diagram showing the flow of a process for accepting a user's response to a situation presented to the user and presenting an evaluation result for encouraging the user to change his or her behavior.

[0148] In step S921, the server 20 presents the user with a screen that presents a situation related to an unconscious belief and receives an answer as to what action the user will take in response to the situation.

[0149] When presenting a question to a user, the server 20 may refer to the answer history database 216 to obtain the user's history of past answers, and may identify the question to present from among the questions stored in the question database 212 based on the attributes of questions previously presented to the user, past evaluation results for the questions, etc.

[0150] For example, the server 20 may identify a question that is different from questions previously presented to the user.

[0151] The server 20 may specify attributes of questions to be presented to the user depending on the evaluation axes and the scores of the user's past answers (for example, specify attributes corresponding to the evaluation axes that received low ratings).

[0152] The server 20 may refer to the training content attendance history database 217 to identify questions to be presented to the user based on the user's training attendance history (for example, lowering the priority of questions related to attributes where the user has attended a certain number of trainings, and giving priority to questions related to attributes where the user has not yet attended many trainings).

[0153] In step S911, the terminal 10 accepts a free text answer from the user.

[0154] The terminal 10 transmits the answer received from the user to the server 20.

[0155] In step S923, the server 20 generates a prompt for input to the large-scale language model service server 92.

[0156] The server 20 refers to the prompt database 213 and generates a prompt to be sent to the large-scale language model service server 92 from the contents of the user's input in step S911 and a prompt template shown in the prompt database 213.

[0157] In step S925, the server 20 sends a prompt to the large-scale language model service server 92 and receives an output result from the large-scale language model service server 92.

[0158] The large-scale language model service server 92 generates an answer by inputting the prompt received from the server 20 into the large-scale language model, and transmits the answer to the server 20. The server 20 updates the answer history database 216 based on the result output by the large-scale language model service server 92.

[0159] In step S927, the server 20 generates a screen showing the evaluation results of the user's answer based on the output of the large-scale language model service server 92, and presents the screen to the user.

[0160] In step S913, the terminal 10 displays the scores for each of the multiple evaluation axes, advice, model answers, and recommended training content as evaluation results of the user's answers.

[0161] In step S915, the terminal 10 accepts the designation of the training content and displays the training content so that it can be viewed.

[0162] In step S929, the server 20 stores information about the training content designated by the user.

[0163] FIG. 10 is a diagram showing the flow of processing for presenting evaluation results in response to an inquiry from another service.

[0164] In step S1021, the server 20 presents a screen for accepting the designation of another service.

[0165] In step S1011, the terminal 10 accepts the specification of another service for which unconscious beliefs are to be evaluated, and the specification of content (posts, etc.) generated by the user in the other service.

[0166] In step S1023, the server 20 generates a prompt for input to the large-scale language model service server 92 according to the service.

[0167] The server 20 may identify the "applicable attributes" item in the prompt database 213 and select a prompt template depending on the type of service or the user information (such as the user's job title) shown in the user database 211.

[0168] The server 20 may accept from the user specifications of the attributes of the prompt template and the content of the prompt template, and use the specified prompt template to generate a prompt based on the content specified by the user in step S1011.

[0169] In step S1025, the server 20 transmits the prompt to the large-scale language model service server 92, receives the output result from the service server 92, and stores it in the answer history database 216 in association with information on other services.

[0170] Although not shown, the server 20 may store, in the response history database 216, information identifying other services that have been evaluated.

[0171] In step S1027, the server 20 generates a screen showing the evaluation results based on the output of the large-scale language model service server 92, and presents it to the user together with information about the other services that were the subject of the evaluation.

[0172] In step S1013, the terminal 10 displays the evaluation results, including information about the other services that were the subject of evaluation, scores for each of the multiple evaluation axes, advice, model answers, and recommended training content.

[0173] In step S915, the terminal 10 accepts the designation of the training content from the user and displays the designated training content in a viewable manner.

[0174] In step S929, the server 20 refers to the training content attendance history database 217 and stores information about the training content designated by the user.

[0175] <4 Screen example> FIG. 11 shows an example of a screen for presenting a question to the user and receiving an answer from the user.

[0176] A screen 1100 is a screen for checking the user's unconscious assumptions.

[0177] As shown in the figure, screen 1100 presents the user with a situation for checking the degree of unconscious beliefs and accepts a response from the user, which corresponds to the processing in step S921 in FIG.

[0178] User information 1110 is a screen for displaying information about a user who is to receive a check for a service provided by the server 20 .

[0179] In the illustrated example, the user information 1110 displays the user's name and employee number. When using a service, the server 20 accepts a login operation from the user using the user's name and employee number, and refers to the user database 211 to accept the user's login.

[0180] The question display area 1120 is an area for displaying questions to be presented to the user.

[0181] The question display area 1120 includes a question background 1121, which will be described later, and a question content 1122. The server 20 refers to the item “question text” in the question database 212, and displays the question in the question background 1121 and the question content 1122.

[0182] The question background 1121 is an area for presenting the background situation of the question to the user.

[0183] The question background 1121, as in the illustrated example, displays the following information to make it easier for the user to answer. Characters in the question Relationships between characters The purpose of communication between the characters ·Policy for getting users to answer questions For example, the server 20 may hold in advance information on the "background situation" of each question in the question database 212, and present the "background situation" to the user in the question background 1121. For example, the question database 212 may hold information on the attributes of the person seeking advice, as explained in the prompt template above, and may select an explanation of the "background situation" appropriate for the question from among a plurality of pieces of "background situation" information prepared in advance, depending on the attribute of the person seeking advice (for example, "female employee"). The question content 1122 is an area for displaying the content of the question to which a specific answer is to be received from the user.

[0184] As shown in the example, the question content 1122 prompts the user to answer by displaying the following specific communication content: -Content of consultations made to users For example, the contents of the consultation may include the following: Health consultation: Advice about the health of the person being consulted or someone related to them (family, pets, etc.). ·Consultation for the future: For example, concerns about the person you are consulting about their future career. ·Interpersonal relationship counseling: Advice on relationships with colleagues, family, etc. Financial advice For example, a consultation about anxiety caused by a sudden expense that the person you are consulting with will incur.

[0185] The answer input area 1130 is an area for receiving the user's answer to the question.

[0186] The answer input section 1131 is an area for accepting an answer from the user.

[0187] The answer input unit 1131 does not need to display answer options as in the illustrated example, but accepts the user's answer in free text.

[0188] In addition, the server 20 may accept answers from the user in the form of voice, images, etc. For example, an operating member for accepting recording operations may be provided, and the user may operate it on a smartphone, etc., to accept voice input as an answer.

[0189] In the illustrated example, the user's answer is made easier by displaying points to note when accepting an answer from the user near the answer input area 1130. Points to note include constraints related to prompt creation (in this example, a character count constraint) such as "at least 40 characters and no more than 200 characters" as well as constraints related to information management such as "please do not enter personal or confidential information."

[0190] The response sending unit 1132 is an operation member for accepting an operation from the user to finalize the content of the response and proceed to evaluation by the server 20 and the large-scale language model service server 92.

[0191] In the example shown in the figure, the response sending unit 1132 does not have any special display mode, but in order to make the response sending unit 1132 easier to see and to make it easier to respond, the response sending unit 1132 may be displayed in a way that makes the background color stand out, for example.

[0192] Furthermore, when the user operates the answer sending unit 1132 to confirm the answer, the display mode of the answer sending unit 1132 may be changed to display in a mode indicating that the user has already answered, as shown in FIG. 12.

[0193] FIG. 12 shows an example of a screen for displaying the results of evaluating the user's input content.

[0194] The evaluation result display area 1140 is an area for displaying the evaluation result for the user's answer.

[0195] The evaluation score 1141 is an area for displaying a score as a result of evaluating the user's answer.

[0196] In the illustrated example, the evaluation score 1141 does not indicate any particular evaluation axis for evaluating the answer, but displays one overall score.

[0197] Advice 1142 is an area for displaying advice on the answer as a result of evaluating the user's answer.

[0198] Advice 1142 displays the result of the server 20 identifying the part of advice to the user based on the output result generated by the large-scale language model service server 92 (item "Advice" in the answer history database 216).

[0199] The model answer example 1143 is an area for displaying a model answer to a question generated by the server 20 and the large-scale language model service server 92.

[0200] Model answer example 1143 displays the result of server 20 identifying the model answer part based on the output result generated by large-scale language model service server 92 (item "Model Answer" in answer history database 216).

[0201] The progress operation section 1150 is an area for accepting subsequent operations from the user.

[0202] In the illustrated example, the progress operation unit 1150 presents a plurality of questions to the user, and receives an operation from the user to display questions that follow a question for which an evaluation result has been given.

[0203] FIG. 13 shows an example of a screen for presenting evaluation results based on multiple evaluation axes and training content.

[0204] The number of answers 1145 is an area for displaying the number of questions that the user has answered.

[0205] The score for each evaluation axis 1146 is an area for displaying the score for each of the multiple evaluation axes as a result of evaluating the user's answer.

[0206] In the illustrated example, the score 1146 for each evaluation axis displays the results of evaluation on a five-point scale for each of the multiple evaluation axes (the large-scale language model service server 92 is caused to generate the evaluation results so as to evaluate on a five-point scale).

[0207] The general advice 1147 is an area for displaying advice on the answer as a result of evaluating the user's answer.

[0208] The general advice 1147 may display advice included in the evaluation results for one of the questions, or may display advice generated as evaluation results for multiple questions in a list or summary form.

[0209] The training content display area 1160 is an area for displaying training content that is recommended for the user to take.

[0210] The answer viewing section 1161 is an area for accepting an operation from the user to display the history of answers the user has given to questions.

[0211] In the illustrated example, the answer viewing unit 1161 transmits a request to the server 20 to acquire the user's past answers and evaluation results shown in the answer history database 216 in response to receiving an operation from the user.

[0212] The training suggestion section 1162 is an area for displaying recommended training content that is identified according to at least one of the content of the user's answer to the question and the evaluation result of the answer.

[0213] As shown in the figure, the training suggestion unit 1162 displays recommended training content corresponding to each evaluation axis for each answer so that the user can select it. In response to an operation by the user to specify the training content, the server 20 inquires of the training service server 94, allows the user to view the training content, and updates the training content attendance history database 217.

[0214] The recommended content viewing unit 1163 is an operation member for accepting an operation to start viewing training content recommended to the user.

[0215] In the illustrated example, the recommended content viewing unit 1163 receives a user operation, and the server 20 queries the training service server 94, and displays the training content recommended based on the user's answer history, etc., so that the training content can be viewed.

[0216] The course history viewing section 1164 is an area for accepting an operation to view the history of the training courses that the user has taken.

[0217] In the illustrated example, the course history viewing unit 1164 causes the server 20 to display the user's course history shown in the training content course history database 217 in response to a user operation.

[0218] FIG. 14 shows an example of a screen for evaluating a user based on data from other services and presenting the results.

[0219] The other service designation reception area 1170 is an area for receiving an operation to check content posted by the user in other services.

[0220] The other service calling unit 1171 is an operation member for calling another service to be checked and accepting the designation of the content to be checked (such as text posted by a user).

[0221] The other service calling unit 1171 corresponds to the processing of step S1011 in FIG.

[0222] The check target display section 1172 is an area for displaying the details of the content that the user has designated as the check target.

[0223] The check item specification section 1173 is an area for receiving, from the user, specification of the evaluation axis to be checked.

[0224] In the illustrated example, the check item specification unit 1173 accepts user specification of whether each evaluation axis should be evaluated. In response to the user's specification, the server 20 refers to the prompt database 213 and determines which prompt template to use.

[0225] In addition, it is also possible to not accept the user's specification of evaluation axes, and to evaluate using predetermined evaluation axes (without allowing the user to edit the evaluation axes) (using a prompt template corresponding to the evaluation).

[0226] The check start section 1174 is an operation member for accepting an operation to start evaluation by the server 20 and the large-scale language model service server 92 of the content to be checked, which is specified in the check target display section 1172 .

[0227] In the illustrated example, the check start unit 1174 causes the server 20 to perform the processes of steps S1023, S1025, etc. in FIG. 10 in response to a user operation.

[0228] The check history display area 1180 is an area for displaying the history of evaluations made on other services.

[0229] The history details display section 1182 is an area for displaying details such as the evaluation results and the time when the evaluation was made for each other service.

[0230] In the illustrated example, the history details display unit 1182 accepts a user's operation on the "Confirm" button shown under "Details," and in response to the operation, the server 20 refers to the answer history database 216 and provides the user with details such as the question text and the content of the user's answer.

[0231] <Modification> In addition to the aspects described in the above embodiment, the following may be adopted: The following will be described along the chronological order of experiences from when the user starts using the service.

[0232] (1) How to log in to the service In the above embodiment, an example was given in which when a user of an organization such as a business company logs in to a service provided by server 20, an employee number and name are entered as information to identify each user, and the information is compared with user database 211.

[0233] In addition, server 20 may use the login information received from the user to communicate with server 96 of the human resources and labor service used by the organization to which the user belongs, thereby verifying the user and enabling the user to log in to the services of server 20.

[0234] In addition, it may be possible to log in using account information (e.g., an email address) used to log in to other services. The other services may be a service that includes a messaging tool, a conference tool, and a storage service, or it may be possible to log in using an account for a service such as attendance management or labor management. By linking these other services with the services provided by server 20, it is possible to understand from the other services how each user is checking their unconscious assumptions using the services provided by server 20 (e.g., information stored in answer history database 216), making it even easier for a human resources person, for example, to understand employee information.

[0235] (2) How to add candidate questions to be presented to users In the above embodiment, an example has been described in which a business providing the service of the server 20 or a user adds questions to the questions managed in the question database 212.

[0236] Alternatively, the server 20 may generate questions without the user creating the questions. For example, the server 20 may send a prompt to the large-scale language model service server 92 to generate a question, and then register the generated question in the question database 212.

[0237] The server 20 may refer to the question database 212 and cause the large-scale language model service server 92 to generate questions depending on the number of questions stored and the number of questions for each attribute. For example, when the number of questions is small, questions may be generated.

[0238] The server 20 may refer to the question database 212 and the answer history database 216 to cause the large-scale language model service server 92 to generate questions based on the history of question use.

[0239] For example, if the number of times a question about a particular attribute has been presented to a user exceeds a predetermined value (e.g., if it is assumed that the likelihood of presenting the same question to the user is increasing), a question about that attribute can be added by sending a prompt to the large-scale language model service server 92 to generate a question about that attribute.

[0240] Furthermore, the server 20 may cause the large-scale language model service server 92 to generate questions, for example, according to the evaluation results of the user's answers. For example, for evaluation axes that have been poorly rated by a certain number of employees (and may also target evaluation axes that have consistently been poorly rated), a prompt may be generated to create questions suitable for evaluating the evaluation axis so that training content can be continuously taken and checks can be performed using the service of the server 20, and the generated prompt may be sent to the large-scale language model service server 92, thereby adding questions.

[0241] Furthermore, the server 20 may receive user feedback (such as a thumbs-up) on the evaluation result of the user's answer, and may cause the large-scale language model service server 92 to generate a question in response to the feedback. For example, if the user's feedback results in a low evaluation, the content of the question or the content of the prompt may be inappropriate, and therefore the large-scale language model service server 92 may be caused to generate a different question based on the attributes of the question.

[0242] (3) How to extract users to whom questions are to be presented In the above embodiment, an example has been described in which a question is presented to a user who logs in to a service provided by the server 20.

[0243] In addition, for example, a human resources person may want to extract employees who will be checked for unconscious assumptions using the services of the server 20. The server 20 may extract employees by, for example, referring to the user database 211 and referring to the date of joining the company, the number of working days, and the type of work. For example, the server 20 may check employees who joined the company within the last few months as part of in-house training.

[0244] (4) How to identify questions to be presented to users In the above description of the embodiment, the following example has been described for specifying the question to be presented to the user.

[0245] (i) Identifying a question to be presented to a user based on the history of attributes of questions previously presented to the user. For example, identifying a question different from those previously presented to the user. (ii) Identifying questions to be presented to the user based on the results of evaluating the user's answers to questions presented to the user. For example, depending on the evaluation scores of past answers, identifying the attributes of the questions (e.g., attributes corresponding to the evaluation axis with a low score) and presenting new questions with the identified attributes. (iii) Assuming that the training content is classified, the questions to be presented to the user are determined based on the user's training attendance history. For example, questions related to the attributes of the user having attended a certain number of training sessions may be given lower priority, and questions related to the attributes of the user having attended few training sessions may be given priority. Additionally, the server 20 may refer to the user database 211 and identify questions to be presented to the user from the question database 212 according to the user's position. For example, if the user is a member of management, questions aimed at management (such as inquiries from subordinates) may be determined as questions to be presented to the user.

[0246] Furthermore, the server 20 may refer to the user database 211, and when a superior or subordinate is set for the user, identify questions to be presented to the user from the question database 212 according to the setting. For example, when a superior is set, questions to be answered from the subordinate's perspective may be determined as questions to be presented to the user.

[0247] (5) How to receive responses from users In the above description of the embodiment, the following example has been described as a method for accepting an answer from a user. (i) Free text entered by the user using a keyboard, etc. (ii) User-input speech (which may include accepting the results of automatic transcription as free text) (iii) Images designated by the user (for example, still images or videos taken by the user on a terminal, which may include the user's facial expressions, voice, etc., and may include information identifying the subject by taking these still images, etc. (such as the name of the object), and the subject's emotions (such as anger or joy) obtained by analyzing the subject's facial expressions, etc.)

[0248] In addition, by sensing the user's body movements (such as estimating the user's posture from images captured by a camera or measuring the user's body as point cloud data), information estimated from the user's body movements may be accepted as the user's answer. For example, the user's emotions may be estimated from the user's body movements, and the estimated results may be accepted as the user's answer.

[0249] (6) How to manage prompt templates In the above embodiment, an example has been described in which prompt templates managed in prompt database 213 are added by a business that provides the service of server 20 or by a user.

[0250] Additionally, the server 20 may refer to the answer history database 216 for prompt templates stored in the prompt database 213 and determine whether or not to use the prompt template depending on its frequency of use. For example, a prompt that is not used often may not be considered for use.

[0251] Furthermore, if the server 20 is configured to receive user feedback regarding the evaluation results of the answers, the server 20 may determine whether or not to use a prompt template depending on the content of the feedback. For example, if the user provides feedback indicating that the evaluation results are undesirable, the applied prompt may make it difficult for the large-scale language model service server 92 to generate an appropriate evaluation. The server 20 may refer to the answer history database 216, determine not to use a prompt template used in an answer for which the user has provided negative feedback, and update the prompt database 213.

[0252] (7) How to determine the prompt template to apply to a question In the above description of the embodiment, the following example has been described for determining the prompt to be applied to the question.

[0253] (i) Attributes are set for questions (question database 212) and attributes are also set for prompt templates (prompt database 213), so that the prompt template to be applied to the user's answer is identified. For example, if the attributes of the question and the attributes set in the prompt template match or are classified as similar attributes, the prompt template is used to generate the prompt. (ii) Determine the prompt template according to the information of the user who answers the question. For example, specify the prompt template according to the user's job title, etc. (iii) Identifying prompt templates for other services when rating content posted by users on those services. (iv) Accept the prompt template specification (prompt template attributes and content) from the user. (v) Accepting from the user the specification of evaluation criteria for evaluating the answer, and identifying a prompt template according to the specified evaluation criteria.

[0254] In addition, the server 20 may store in advance each question in association with a prompt to be applied.

[0255] Furthermore, when the server 20 receives feedback from the user regarding the evaluation result, the server 20 may determine the prompt template to be applied to the question according to the content of the feedback. For example, when the user provides feedback indicating that the evaluation result of the answer to the question is favorable, the server 20 may determine that the applied prompt template is appropriate, and may set the prompt template to be available in the prompt database 213, while changing the weighting for applying the prompt template to the answer to the question (making it more likely to be applied).

[0256] (8) Method for presenting evaluation results to users In the above embodiment, an example has been described in which a user logs in to a service provided by the server 20 and displays the evaluation results on the service.

[0257] In addition, the content of a user's input in another service (which may be the content posted by the user in the other service, or the other service may accept the input of a user's answer to be sent to server 20) may be evaluated in the service provided by server 20 on the screen of the other service, and the evaluation results may be displayed on the screen of the other service.

[0258] (9) A method for identifying training content to be suggested to a user In the above description of the embodiment, the following example has been described for identifying training content to be proposed to the user. (i) Identifying training content based on the user's response to the question. For example, referring to the wording contained in the response, the wording is used as a keyword for searching for training content on the training service server 94. (ii) Identifying training content according to the evaluation results of the user's answers to the questions. For example, referring to the wording of advice, scores, etc. included in the evaluation results, the content is used as a keyword for searching for training content on the training service server 94.

[0259] In addition, the server 20 may refer to the user database 211 to identify training content to be proposed to the user according to the user's position and whether the user is a superior or subordinate. For example, if the user is a member of management, training content for management will be proposed.

[0260] (10) A method for outputting the evaluation results of multiple users belonging to an organization as the state of the organization In the above embodiment, an example has been described in which one user logs in to a service provided by the server 20 and receives a presentation of an evaluation result for an answer.

[0261] For example, human resources personnel and management of an organization may want to refer to the evaluation results of multiple employees belonging to the organization to gauge the state of the organization, for example, whether diversity can be achieved and business can be promoted. The server 20 may extract the evaluation results of multiple employees by department, for example, and display them as a list, or may identify and display an overall score from the evaluation results of these multiple employees.

[0262] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0263] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes programs stored in memory.

[0264] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0265] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0266] <Additional Notes> The matters explained in the above embodiment will be supplemented below.

[0267] (Appendix 1) A program for operating a computer (20, 10), causing a processor of the computer to execute the steps of: presenting a situation related to unconscious beliefs to a user as a question, and presenting a screen for accepting input of an answer as to what action the user will take in response to the situation (2044, S921, 1120); accepting input of the answer in free text from the user (2041, S911, 1130); analyzing the free text answer accepted from the user and evaluating the degree of the user's unconscious beliefs (2045, 2046, S923, S925); and presenting the evaluation result to the user (2046, S927, S913, 1140).

[0268] (Appendix 2) 11. The program according to claim 1, wherein in the step of presenting the screen, the screen is presented without presenting options for a candidate answer to the situation (1120, 1130).

[0269] (Appendix 3) In the evaluation step, the degree of unconscious belief is evaluated by generating a prompt to be input into a large-scale language model (92) based on the free-text response (S923), and inputting the generated prompt into the large-scale language model to evaluate the output result output from the large-scale language model (S925).In the result presentation step, the evaluated result is presented based on the output result of the large-scale language model (S927, S913, 1140).

[0270] (Appendix 4) The program described in Appendix 3, wherein in the evaluation step, a prompt (213, S923) that defines conditions for evaluating the user's answer including the reason for the evaluation is input to the large-scale language model, and an output result including the reason for the evaluation is received from the large-scale language model (216, 1140).

[0271] (Appendix 5) In the evaluation step, a prompt (213, S923) that defines conditions for evaluating the user's answer including an evaluation score is input into the large-scale language model, and the conditions for the evaluation score define a condition that if the user's answer is a meaningless sentence such as a string of incomprehensible characters, the answer will be given the lowest score or a score that is not eligible for evaluation, and output results including the evaluation score are received from the large-scale language model (S925, 1141, 1146).

[0272] (Appendix 6) A program described in any of Appendices 3 to 5, wherein in the evaluation step, a prompt (213, S925) that sets conditions for evaluating the user's answer including a model answer is input to a large-scale language model, output results including the model answer are received from the large-scale language model, and in the result presentation step, evaluated results including the model answer are presented based on the output results of the large-scale language model (S927, S913, 1143).

[0273] (Appendix 7) A program described in any of Appendices 3 to 6, wherein in the evaluation step, a prompt (213, S925) that defines conditions for evaluating the user's answer along each of multiple evaluation axes is input to a large-scale language model, and output results evaluated along each of the multiple evaluation axes are received from the large-scale language model, and in the presentation step, the evaluation results along each of the multiple evaluation axes are presented based on the output results of the large-scale language model (1146).

[0274] (Appendix 8) In the evaluation step, a prompt is input into the large-scale language model with conditions set to evaluate the employee based on at least two of the following multiple evaluation criteria (S923, S925, 1146): whether the employee imposes unconscious assumptions, whether the employee understands the employee's requests, whether the employee understands company policies, and whether the employee understands diversity. This is the program described in Appendix 7.

[0275] (Appendix 9) The program described in any of Appendices 1 to 8, further causing the processor to execute steps of identifying training content and providing the identified training content to the user depending on the results of the evaluation in the evaluating step (S915, S929, 1160).

[0276] (Appendix 10) In the presenting step, a plurality of types of situations are presented to the user (1150), and in the evaluating step, the types of situations are associated with the results of evaluating the user's answers and stored in a memory unit (216), and training content is provided to the user, and the training content is identified according to the results of the user's evaluation for each type of situation (S915), the program described in Appendix 9.

[0277] (Appendix 11) The program described in Appendix 1 further causes the processor to execute the steps of accepting submissions of situations to be presented to users, and generating questions to be answered based on the situation questions stored in the memory unit.

[0278] (Appendix 12) The program further causes the processor to receive content such as text entered by a user of another system from another system (S1021, S1011), evaluate the received content as the subject of analysis in an evaluation step (S1023, S1025), and present the evaluation results to the user (S1027, S1013), a program as described in Appendix 1.

[0279] (Appendix 13) A method for operating a computer, the method comprising the steps of: presenting a situation related to unconscious beliefs to a user as a question, and presenting a screen for accepting input of an answer as to what action the user will take in response to the situation; accepting input of the answer in free text from the user; analyzing the free text answer accepted from the user to evaluate the degree of the user's unconscious beliefs; and presenting the evaluated results to the user.

[0280] (Appendix 14) An information processing device, wherein a control unit of the information processing device executes the steps of: presenting a situation related to unconscious beliefs to a user as a question, and presenting a screen for accepting input of an answer as to what action the user will take in response to the situation; accepting input of the answer in free text from the user; analyzing the free text answer accepted from the user to evaluate the degree of the user's unconscious beliefs; and presenting the evaluated results to the user.

Claims

1. A program for operating a computer, the program comprising: a step of presenting a screen to a user to ask a question about a situation related to an unconscious belief and to receive an answer as to what action the user will take in response to the situation; accepting an input of the answer from the user in free text; analyzing the free text responses received from the user to assess the extent of the user's unconscious beliefs; and presenting the evaluated result to the user.

2. 2. The program according to claim 1, wherein in the step of presenting the screen, the screen is presented without presenting options of answer candidates for the situation.

3. In the evaluating step, the degree of the unconscious belief is evaluated by generating prompts for input to a large-scale language model based on the free-text responses; inputting the generated prompt into the large-scale language model and outputting an output result from the large-scale language model as the evaluation; The program according to claim 1 , wherein in the step of presenting the results, the evaluated results are presented based on the output results of the large-scale language model.

4. In the evaluating step, inputting the prompt, which defines conditions for evaluating the user's answer including the reason for the evaluation, into the large-scale language model; The program according to claim 3 , further comprising receiving the output result from the large-scale language model, the output result including the evaluation reason.

5. In the evaluating step, inputting the prompt, conditioned on rating the user's response including a rating score, into the large-scale language model; The condition for the evaluation score specifies that if the user's answer is a meaningless sentence such as a string of incomprehensible characters, the answer will be given the lowest score or a score that is not eligible for evaluation, The program of claim 4 , further comprising receiving the output results from the large-scale language model, the output results including a score of the evaluation.

6. In the evaluating step, inputting the prompt, which defines conditions for evaluating the user's response, including exemplary responses, into the large-scale language model; receiving the output results from the large-scale language model, the output results including the model answers; 4. The program according to claim 3, wherein in the step of presenting the results, the evaluated results including the model answers are presented based on the output results of the large-scale language model.

7. In the evaluating step, inputting the prompt, which defines conditions for evaluating the user's answer along each of a plurality of evaluation axes, into the large-scale language model; receiving the output results evaluated with respect to each of the plurality of evaluation axes from the large-scale language model; The program according to claim 3 , wherein the step of presenting comprises presenting a result of evaluation along each of the plurality of evaluation axes based on the output result of the large-scale language model.

8. In the evaluating step, Examples of the multiple evaluation axes include: Are you imposing unconscious beliefs? Do you understand the needs of your employees? Do you understand the company's policies? Do you understand diversity? 8. The program of claim 7, wherein the prompt is input to the large-scale language model with conditions set to evaluate the prompt including at least two of the following:

9. The program further causes the processor to The program according to claim 1 , further comprising a step of identifying training content according to a result of the evaluation in the step of evaluating, and providing the identified training content to the user.

10. In the step of presenting the screen, a plurality of types of situations are presented to the user; In the step of evaluating, the type of the situation and the result of evaluating the answer of the user are stored in a storage unit in association with each other, The program according to claim 9 , wherein the training content to be provided to the user is identified according to a result of the user's evaluation of each of the situation types.

11. The program further causes the processor to receiving a posting of a situation to be presented to the user; The program according to claim 1 , further comprising: a step of generating a question to be answered based on the situation questions stored in a storage unit.

12. The program further causes the processor to Accepting content such as text entered by a user of another system from the other system; The received content is evaluated in the evaluating step as an analysis target; The program according to claim 1 , wherein the evaluated results are presented to the user.

13. 1. A method for operating a computer, the method comprising: a step of presenting a screen to a user to ask a question about a situation related to an unconscious belief and to receive an answer as to what action the user will take in response to the situation; accepting an input of the answer from the user in free text; analyzing the free text responses received from the user to assess the extent of the user's unconscious beliefs; presenting the evaluated results to the user.

14. An information processing device, wherein a control unit of the information processing device a step of presenting a screen to a user to ask a question about a situation related to an unconscious belief and to receive an answer as to what action the user will take in response to the situation; accepting an input of the answer from the user in free text; analyzing the free text responses received from the user to assess the extent of the user's unconscious beliefs; and presenting the evaluated result to the user.

Images