Device and method

Abstract

The purpose of the present invention is to perform detailed response according to a situation in which a user is placed.　In a mental support server 100 according to the present disclosure, a reception unit 101 receives, from a user terminal 400, a generation request for counseling content for a distribution user of distribution content such as a moving image. When receiving the generation request for the counseling content, an acquisition unit 102 acquires information pertaining to emotion estimation of the distribution user of the user terminal 400 and information pertaining to the distribution content. A control unit 103 determines a generation policy of the counseling content on the basis of the information pertaining to the emotion estimation of the distribution user and the information pertaining to the distribution content. A generation unit 104 generates a prompt for instructing generation of the counseling content on the basis of the generation policy of the counseling content.

Description

Apparatus and method

[0001] The present invention relates to an apparatus and method that utilizes a Large Language Model (LLM).

[0002] Patent Literature 1 describes providing a role-playing style dialogue with a persona by an AI chatbot. More specifically, Patent Literature 1 describes that the persona can respond to everyday negative emotions or thoughts that a user faces, and that the persona has characteristics and guidelines for action that include things that make the user happy just by being alive, thereby increasing the user's sense of meaning in life and self-affirmation.

[0003] Japanese Patent Application Laid-Open No. 2023-171706

[0004] However, the invention described in Patent Document 1 has a problem in that it is not possible to respond in detail to the user's voice or to the reaction of the situation the user is in. For example, Patent Document 1 has difficulty responding to changes in the state that the user is not aware of, such as the sound of a video.

[0005] Therefore, an object of the present invention is to provide a device and method that can provide detailed mental support according to the situation in which the user is placed.

[0006] The device of the present invention includes a reception unit that receives request information for generating counseling content for a distribution user of the distribution content, an acquisition unit that acquires information regarding the distribution user's emotion estimation and information regarding the distribution content upon receiving the request information for generating counseling content, a control unit that determines a generation policy for the counseling content based on the information regarding the distribution user's emotion estimation and the information regarding the distribution content, and a generation unit that generates a prompt that instructs the generation of the counseling content based on the generation policy for the counseling content.

[0007] According to the present invention, it is possible to provide detailed mental support according to the situation in which the user is placed.

[0008] FIG. 1 is a diagram showing a system configuration including a mental support server 100 of the present disclosure, and a diagram showing the distribution of distribution content and comment posting. FIG. 2 is a diagram showing a system configuration and an overview of operation including the mental support server 100 of the present disclosure, and a diagram showing a request for generating counseling content. FIG. 3 is a diagram showing the functional configuration of a distribution server 200. FIG. 4 is a diagram showing the functional configuration of the mental support server 100 of the present disclosure. FIG. 5 is a diagram showing the transition of information related to emotion estimation (emotion score). FIG. 6 is a diagram showing a specific example of a content understanding information DB 102b. FIG. 7 is a diagram showing a specific example of a knowledge DB 103a. FIG. 8 is a flowchart showing the operation of the mental support server 100. FIG. 9 is a diagram showing an example of a prompt generated by the generation unit 104 of the present disclosure. FIG. 10 is a diagram showing results obtained from the LLM 200. FIG. 11 is a diagram showing the configuration of a user terminal 400 on which the mental support server 100 or the LLM 300 is installed. FIG. 12 is a diagram illustrating an example of the hardware configuration of the mental support server 100, the distribution server 200, and the user terminal 400 according to an embodiment of the present disclosure.

[0009] The present disclosure will be described with reference to the accompanying drawings. Whenever possible, the same parts are designated by the same reference numerals and redundant description will be omitted.

[0010] 1 and 2 are diagrams showing the configuration and operation overview of a system including a mental support server 100 of the present disclosure. This system includes the mental support server 100, a distribution server 200, and an LLM (Large Language Model) 300.

[0011] As shown in Fig. 1, distribution server 200 distributes content (e.g., video) to user terminal 410 in response to a content distribution request from user terminal 400. The user of user terminal 410 posts a comment (text) on the distributed content to distribution server 200. Distribution server 200 stores the comment in association with the posting time. The comment may be voice, a symbol, a stamp (image), or the like.

[0012] In general SNS (Social Network Services) including video distribution, users can post comments on distributed content. If the content is being distributed live (in real time), the user of the user terminal 400 can change the content of the distributed content while viewing the comments. For example, if the content is a user giving a lecture, the user can change what they are saying while viewing the comments.

[0013] The posted comments may be attacks or criticisms against the broadcast user of user terminal 400. Such attacks and criticisms can cause a mental breakdown for the user who is the broadcaster of the broadcast content.

[0014] In the present disclosure, the mental support server 100 provides mental support to the distributor user who distributed the distribution content, based on the distributed video (content), posted comments, and poster.

[0015] 2, specifically, after the distribution is completed, the user of the user terminal 400 makes a request for the generation of counseling content (including a user ID and a content ID, etc.) to the mental support server 100. At this time, the user may fill out a questionnaire (to understand the mental state) about how the user actually felt and send it to the mental support server 100. The user ID and content ID do not need to be IDs, but may be any information that can identify the distribution user and the content.

[0016] Based on this generation request, the mental support server 100 acquires the posted comment, the posted time, and the distribution content from the distribution server 200.

[0017] Then, the mental support server 100 generates a prompt for requesting the generation of counseling content based on this information, requests the LLM 300 to generate counseling content, and transmits the counseling content obtained thereto to the user terminal 400.

[0018] This allows appropriate counseling content to be sent to the distribution user of user terminal 400. In the above explanation, for convenience, the video distribution request and the counseling content generation request are made from user terminal 400, but these do not have to be made from the same terminal and may be made from separate terminals.

[0019] In this disclosure, the description is based on the premise that a request for generating counseling content is made after the distribution of the distribution content, but this is not limited to this. A request for generating counseling content may be made while the distribution content is being distributed. This allows counseling to be received in real time.

[0020] 3 is a diagram showing the functional configuration of distribution server 200. As shown in the figure, distribution server 200 is configured to include a communication unit 201, a content storage unit 202, and a comment storage unit 203. Communication unit 201 distributes distribution content (e.g., videos) stored in content storage unit 202 to user terminal 410. Communication unit 201 also stores comments posted from user terminal 410 in comment storage unit 203 together with the posting time.

[0021] The communication unit 201 sends comments and distribution content in response to a request from the mental support server 100 .

[0022] 4 is a diagram showing the functional configuration of the mental support server 100 of the present disclosure. As shown in the figure, the mental support server 100 is configured to include a reception unit 101, an acquisition unit 102, a control unit 103, a generation unit 104, a content acquisition unit 105, an emotion estimation model 102a, a content understanding information DB 102b, and a knowledge DB 103a. Note that the estimation model and various DBs may be located in other devices, and the mental support server 100 may cooperate with the other devices.

[0023] The reception unit 101 is a part that receives a request to generate counseling content for a distribution user of the distribution content from the user terminal 400. This counseling content is content for providing mental support to the distribution user, and is, for example, information that provides comfort when a critical comment is posted. The reception unit 101 may also receive mental state understanding information.

[0024] The acquisition unit 102 is a part that acquires information related to the emotion estimation of the distribution user and information related to the distribution content when it receives a request to generate counseling content. The information related to the emotion estimation of the distribution user includes information output by the emotion estimation model 102a as well as optional mental state understanding information. The mental state understanding information is subjective information that indicates the emotions of the distribution user based on a questionnaire. Furthermore, the information related to the distribution content is information related to understanding the content of comments posted by the posting user (viewing user).

[0025] The emotion estimation model 102a is a machine learning model that receives audio information from the distribution content and outputs information (scores) indicating the degree of each emotion item (e.g., joy, excitement, anxiety, anger, and sadness). The emotion estimation model 102a is trained using a scoring method that uses a learning model (a deep learning model (e.g., LSTM or CNN)) that learns the relationship between the distribution user's audio information for training and the emotion information (emotion labels) of each emotion item. The emotion estimation model 102a trained in this way receives input of feature quantities of the audio information (fundamental frequency (pitch), volume (intensity), speaking rate (tempo), voice tone (timbre), formant, ratio of voiceless sounds to voiced sounds, and melody pattern (intonation)) and calculates an emotion score for each emotion item. The acquisition unit 102 acquires the information (scores) indicating the degree of each emotion output from the emotion estimation model 102a.

[0026] Furthermore, to output information indicating the degree of positivity / negativity using these emotion scores, each emotion score is weighted to calculate an overall positive / negative score. For example, joy and excitement contribute to a positive score, while anxiety, anger, and sadness contribute to a negative score. This makes it possible to comprehensively evaluate the emotional state of the broadcast user.

[0027] Furthermore, the emotion estimation model 102a may output these emotion items (joy, anger, sadness, happiness, etc.) in association with scores. Similarly, the emotion estimation model 102a may input the broadcast user's facial expression in addition to, or instead of, audio information and output emotion items and scores. In this case, machine learning using facial expressions is required. Furthermore, the emotion estimation model 102a may output information indicating the degree of positivity / negativity in chronological order using these emotion information and scores. The scores and the like may also be displayed in chronological order as a graph. This makes it possible to determine when changes occurred in the broadcast user's emotions.

[0028] FIG. 5 is a diagram showing the transition of information (emotion score) related to emotion estimation. As shown in the figure, the acquisition unit 102 acquires an emotion score indicating the degree of unified positivity / negativity using the emotion estimation model 102a. FIG. 5 shows state transitions based on the emotion score acquired in this way.

[0029] As will be described later, this graph information can be converted into text using a generative AI model. A summary text such as the following can be generated: (For example, at 10:00 on May 2nd, when distribution began, the level was high (about 75% positivity), but dropped significantly around 11:00 (about 35% positivity)). This makes it possible to reduce the number of tokens.

[0030] The acquisition unit 102 also acquires content understanding information about comments posted by users as information about the distribution content. When acquiring the content understanding information, the acquisition unit 102 classifies the comments posted by users as either aggressive Facebook (equivalent to feedback: comments) or friendly Facebook. The acquisition unit 102 may also classify the comments based on information about whether there were more aggressive Facebook comments or more friendly Facebook comments for each time period, the time period, the type of Facebook, or the number of posts. For example, on a certain date and time, around 4:00 PM, the number of friendly Facebook comments was 126 (high), and the number of aggressive Facebook comments was 16 (low). Whether the number is high or low is determined based on a predetermined threshold. The information may also include the "number of unique users" that indicates who is posting. In other words, even if the same user posts multiple times, each post is counted as one.

[0031] 6 is a diagram showing a specific example of the content understanding information DB 102b. As shown in the figure, the content understanding information DB 102b is a storage unit that stores the Facebook posts sorted by the acquisition unit 102. This content understanding information DB 102b stores the date and time period, the number of friendly Facebook posts, the number of users who posted the friendly Facebook posts, the number of aggressive Facebook posts, and the number of users who posted the aggressive Facebook posts.

[0032] The acquisition unit 102 acquires comments posted to the distribution server 200 and information about the posting user after distribution has ended, periodically, or in real time, grasps the content of the comments, and stores the content in the content grasp information DB 102b. The acquisition unit 102 performs a text classification process on the comments stored in the distribution server 200. Specifically, the acquisition unit 102 classifies the comments into specific categories (e.g., positive, negative, neutral). Specifically, the acquisition unit 102 classifies the comments based on the similarity of their content with comments that have been pre-learned using machine learning or the like, or performs rule-based classification of words separately registered in a dictionary or the like. Alternatively, the classification may be performed using algorithms such as support vector machines (SVMs) and deep learning (e.g., BERT, LSTM).

[0033] The acquisition unit 102 may further acquire mental state understanding information of the broadcasting user for understanding the mental state of the broadcasting user. For example, this mental state understanding information is information indicating the broadcasting user's mental state based on the results of a survey conducted after the distribution of the broadcast content has ended (information acquired from the survey results (e.g., (survey example) Q1 Do you ever feel depressed? "1 Never," "2 Not very often," "3 Generally," "4 Sometimes," "5 Yes," etc.).) Alternatively, it may be score information indicating positivity / negativity calculated based on an existing scoring method based on the survey results.

[0034] The control unit 103 is a part that determines a generation policy of counseling content based on information related to emotion estimation of the distribution user and information related to the distribution content. In the present disclosure, the generation policy of counseling content is determined by referring to the knowledge DB 103a.

[0035] FIG. 7 is a diagram illustrating a specific example of knowledge DB 103a. This knowledge DB 103a is a database for determining a counseling content creation policy according to the transition of estimated emotions. As shown in the figure, knowledge DB 103a stores positivity, negativity, friendly Facebook, aggressive Facebook, and a counseling content creation policy in association with each other. The positivity and negativity columns each contain information indicating a trend, such as an upward trend, a downward trend, or maintaining the status quo. The friendly Facebook and aggressive Facebook columns each indicate whether they are high or low. Furthermore, information indicating whether the number of Facebook posts is high or low, as well as the number of posting users, may also be included.

[0036] In this way, the knowledge DB 103a stores at least one of guidelines and example sentences for what kind of greetings should be used in association with the positivity level (high, normal, low). The knowledge DB 103a also associates example sentences to be used as reference when greeting with "counseling creation guidelines." At least one of these creation guidelines and example sentences may be data created by referring to counselor teaching materials, etc.

[0037] The control unit 103 refers to the knowledge DB 103a and determines a policy for generating counseling content.

[0038] This counseling generation policy may include information indicating the requests of the client (distributed user). For example, it may include information such as "I want someone to listen to me," "I want objective advice," or "I want to change my thinking habits." It may also include information indicating the degree of each request. For example, it may include a percentage or degree, such as 30% indicating "I want someone to listen to me" and 70% indicating "I want objective advice." This degree may be based on information obtained from the results of a mental health questionnaire (e.g., "1. Never," "2. Rarely," "3. Normal," "4. Sometimes," "5. Yes"). If the questionnaire answers Q1 "Do you ever feel depressed?" with a five-level rating of "1. Never," "2. Rarely," "3. Normal," "4. Sometimes," or "5. Yes," the degree of counseling for "depression" may be determined. Here, if the rating is "1. Never," a generation policy may be determined in which no counseling for depression is provided.

[0039] The knowledge DB 103a may also include information indicating a counseling generation policy determined based on information (e.g., positivity) output by the emotion estimation model 102a. For example, when the positivity level is higher than usual, the control unit 103 may register example sentences in the knowledge DB 103a for asking the user what went well, whether they have made any improvements, how close they are to achieving their goal, etc., and collect information to be used in the next counseling session. When the positivity level is lower than usual, the knowledge DB 103a may include example sentences or a generation policy for saying encouraging words or saying words that will make the user feel positive.

[0040] Furthermore, if the patient's positivity level is about the same as usual, the knowledge DB 103a may ask questions such as, "You were feeling good today," or "Is there anything that bothered you?", and may decide to collect information that will be useful in the next counseling session as a policy for generating counseling content.

[0041] In addition, the control unit 103 may store in the knowledge DB 103a guidelines on what kind of greetings to use or example greeting sentences in association with the positivity level (high, normal, low), so that this can be determined as a content creation policy.

[0042] The operation of the mental support server 100 configured as above will now be described. Fig. 8 is a flowchart showing the operation of the mental support server 100. After the distribution content is distributed, the distribution user of the user terminal 400 requests the generation of counseling content. The reception unit 101 receives the request to generate counseling content from the user terminal 400 (S101).

[0043] The acquisition unit 102 acquires information related to emotion estimation from the emotion estimation model 102a (S102). That is, the acquisition unit 102 acquires information related to emotions from audio information of live-streamed content. In the present disclosure, the information related to emotions may be emotions themselves (joy, anger, sadness, or pleasure), or information indicating negative / positive emotions based on the emotions.

[0044] The acquiring unit 102 acquires content understanding information of the posted comments of the posting user from the content understanding information DB 102b as information related to the distribution content (S103).

[0045] Then, the control unit 103 determines a policy for generating counseling content based on information related to emotion estimation (such as a score) and information related to the content to be distributed (such as a video) (S104).

[0046] The generating unit 104 generates a prompt based on the generation policy of the counseling content (S105).

[0047] The content acquisition unit 105 sends the generated prompt to the LLM 300 to generate and instruct counseling content, and acquires the counseling content as a response. The content acquisition unit 105 transmits the acquired counseling content to the user terminal 400, causing the user terminal 400 to display the counseling content (S106).

[0048] FIG. 9 is a diagram illustrating an example of a prompt generated by the generation unit 104 of the present disclosure. As shown in the figure, the following sentence is generated as a prompt: Role: Your role is as a counselor. Please provide mental support to the following client after a live video stream. Client's name and occupation: Live video streamer. Client's expectations from counseling (purpose of counseling): To correct negative thinking habits. ... Analysis of input information and voice tone during the live stream revealed that the positivity level was high (approximately 75%) around 10:00 a.m. when the stream began on May 2, but dropped significantly from around 11:00 a.m. (approximately 35%). - Analysis of comments posted by viewers during the live stream revealed that the number of aggressive comments was low around 10:00 a.m. on May 2, but increased around 11:00 a.m. Note that the number of viewers was small at both times. - Survey results obtained from the streamer (client) after the live stream indicated a high level of negativity and depression. Conditions: As a policy, please provide advice to guide the streamer away from negative thinking. Please refer to the following sentence when giving advice: "Although there have been a certain number of offensive posts, they are only being posted by a small number of people, so there is no need to worry." The generation unit 104 generates the above prompt P. In this prompt P, the portion marked with symbol P1 includes the role, the client's name, occupation, and what the client expects from counseling (the purpose of counseling). Of these, the role is a fixed value and is information determined in advance by the broadcasting user. The name, occupation, and what the client expects from counseling are information set in advance by the broadcasting user and are information included in the counseling generation request. The generation unit 104 obtains the above information from the counseling generation request accepted by the acceptance unit 101 and reflects (writes) it in the prompt P.

[0049] Of the input information P2, symbol P21 indicates information based on emotion estimation information and content understanding information, and is information that has been written down from the emotion estimation information and content understanding information. Based on the emotion estimation information, the generation unit 104 generates a sentence such as, "As a result of analyzing the tone of voice during the live broadcast, it was found that the positivity level was high (approximately 75% positivity) around 10:00 when the broadcast started on May 2, but dropped significantly (approximately 35% positivity) from around 11:00." This sentence is written down based on the emotion estimation information as shown in FIG. 4. Techniques for converting graphs into sentences are well known and can be obtained, for example, by data visualization and natural language generation (NLG). Note that generation AI such as LLM may also be used.

[0050] Reference symbol P22 is information based on the content understanding information. Based on the content understanding information, the generation unit 104 generates a sentence such as, "As a result of analyzing the content of comments posted by viewing users during the live broadcast, the number of aggressive posts was low around 10:00 on May 2nd, but the number of aggressive posts was high between 11:00 and 11:00. Note that the number of viewing users was low in both time periods." As with the above, a technique for generating sentences from a table is publicly known, and this is utilized.

[0051] Reference symbol P23 is information based on a questionnaire after the live broadcast (after the content is distributed). This is also a sentence or a sentence obtained from the questionnaire results. Based on the questionnaire, the generation unit 104 generates the sentence, "According to the questionnaire results obtained from the broadcaster (client) after the live broadcast, it is clear that the broadcaster is highly negative and depressed."

[0052] Symbol P3 is information indicating a generation policy for counseling content. The generation unit 104 determines a policy corresponding to positivity, negativity, friendly feedback, and aggressive feedback by referring to the knowledge DB 103a. In the present disclosure, the generation unit 104 reads out "As a policy, please give advice to guide the user to avoid negative thoughts" and reflects (writes) this in the prompt. Note that the generation unit 104 may write example sentences stored in the knowledge DB 103a in the prompt to indicate that the example sentences should be used as a reference when giving advice.

[0053] Next, a description will be given of a counseling content generation policy of the mental support server 100 of the present disclosure. In the mental support server 100, the control unit 103 may change a policy for contacting a broadcasting user based on information regarding the content of the posting user's comments and information regarding emotion estimation. For example, if the positivity level of the information regarding emotion estimation is above a predetermined threshold during a certain time period and the information regarding the content of the posting user's comments during that time period includes a predetermined number of aggressive Facebook posts, a policy of issuing worrying comments may be determined.

[0054] Furthermore, the control unit 103 may decide to adopt a policy of guiding users not to think negatively if, during a certain time period, the positivity of information regarding emotion estimation falls below a predetermined threshold and the number of aggressive Facebook posts in the content understanding information of the posting user's comments during that time period does not reach a predetermined number.

[0055] The control unit 103 may also obtain previous history information (e.g., content understanding information, information regarding emotion estimation, corresponding counseling content generation policy, the response, and the hearing results) and instruct the system to use this information to generate advice text. For example, the control unit 103 may hear and record what efforts were effective when the positivity level was high. For example, a hearing record may be made about reducing the frequency of viewing comments from live viewers. Furthermore, when the positivity level is low, the control unit 103 may output instructions to the prompt to hear whether efforts were made when the positivity level was high.

[0056] The mental support server 100 has a hearing DB (not shown) that stores the contents of the hearing, and determines whether or not a hearing is necessary depending on each tendency of positivity, negativity, friendly Facebook, and aggressive Facebook, and stores the results of hearing from the broadcasting user in the hearing DB in association with each tendency of positivity, negativity, friendly Facebook, and aggressive Facebook. The hearing is conducted by the broadcasting user separately accessing the mental support server 100 and writing.

[0057] The generation unit 104 may refer to the hearing DB and, when a predetermined tendency is found, write the hearing results in the prompt as a reference.

[0058] The generation unit 104 may instruct the system to generate an advice sentence based on the mental state understanding and information regarding emotion estimation. For example, if there is a difference between the mental state understanding (e.g., survey results (subjective)) and the information regarding emotion estimation (objective), the advice sentence may be generated based on this. For example, if the content understanding information of the viewing user's comments indicates a low number of aggressive Facebook posts (e.g., if the number of aggressive Facebook posts exceeds a predetermined threshold), and the mental state understanding (e.g., survey results (subjective)) indicates a low level of negativity and the information regarding emotion estimation (objective) indicates a high level of positivity, the system may output an instruction to the prompt to "mention this point and give advice to think positively." Figure 10 is a diagram showing the results obtained from the LLM 300. In this disclosure, the results are displayed in chatbot format, but this is not necessarily limited to this. Here, the mental support server 100 first sends a message to the broadcaster saying, "Thank you for your hard work during the broadcast. You seemed less energetic than usual during the broadcast. Are you okay?" The broadcaster then replies. The mental support server 100 reads the text from the broadcast user and infers the broadcast user's emotions, etc. from the broadcast content, and generates a prompt to send to the LLM 300. In the present disclosure, a request from the broadcast user saying, "Actually, I saw someone say XX in the comments, and it's been bothering me..." corresponds to a request to generate counseling content, and the mental support server 100 performs the process of generating counseling content based on this. Furthermore, "I saw someone say XX in the comments, and it's been bothering me" corresponds to the counseling purpose of prompt P1. The mental support server 100 reads the text as appropriate and generates a prompt.

[0059] Note that all or some of the LLM 300 may be located in the mental support server 100 or the user terminal 400. Furthermore, the user terminal 400 may have the functionality of the mental support server 100 and function as the mental support server 100. There are types of generative AI models, such as Tsuzumi, in which the generative AI model is located inside the user terminal 400. In this type, the RAG app is also provided on the user terminal 400. However, the information accessed by the RAG (knowledge DB) may be located inside the user terminal 400 or on the network. There are also types, such as ChatGPT, in which the generative AI model is located on the network. In this type, the RAG app is provided on the user terminal 400. However, the information accessed by the RAG (knowledge DB) is located on the network.

[0060] A generative AI model, such as the LLM 300, is a model that generates content in response to a prompt containing input information, based on the instructions, context, question, and output format indicated by the prompt, and returns the content as response information. The prompt can also contain input information, in which case the generative AI model generates response information targeted at the input information. The generative AI model may be, for example, an interactive AI model that includes a large language model (LLM) and a user interface (UI) for interacting with the user, enabling text or voice chat with the user. Examples of such generative AI models include ChatGPT, GPT (registered trademark)-3.5, GPT-4V, PaLM2, and the like. In this embodiment, the mental support server 100 is capable of providing content provision functions using interactive AI models, which are multiple types of models. These interactive AI models may be stored in the mental support server 100, or may be stored in another device connected to the mental support server 100 via a network, and configured to enable information exchange with the user via the mental support server 100. Note that although only one mental support server 100 is shown in the figure, multiple mental support servers 100 may be included.

[0061] In this disclosure, a prompt is information indicating an instruction or question entered by a user in an interactive system such as an interaction with a generative AI model or a command line interface (CLI).

[0062] Here, we will explain the configuration of the user terminal 400 on which the mental support server 100 or LLM 300 is located. Figure 11(a) shows an example configuration when the user terminal 400 has the functions of the mental support server 100. In this case, the user terminal 400 accesses the knowledge DB 103a located internally or externally, uses it to generate a prompt, sends it to the LLM 300, and obtains the result.

[0063] 11(b) shows an example of a configuration in which a user terminal 400 is equipped with a mental support server 100 and an LLM 300. As shown in the figure, the user terminal 400 accesses an internal or external knowledge DB 103a, uses it to generate prompts, outputs them to the built-in LLM 300, and obtains the results.

[0064] The various DBs arranged in the mental support server 100 may be arranged on an external network.

[0065] Next, the effects of the mental support server 100 of the present disclosure will be described. In the mental support server 100 of the present disclosure, the reception unit 101 receives a request to generate counseling content for a distribution user of distribution content such as a video from the user terminal 400. When the acquisition unit 102 receives the request to generate counseling content, it acquires information related to emotion estimation of the distribution user of the user terminal 400 and information related to the distribution content.

[0066] The control unit 103 determines a policy for generating counseling content based on information relating to emotion estimation of the distribution user and information relating to the distribution content.

[0067] The generating unit 104 generates a prompt that instructs the generation of counseling content based on the generation policy of the counseling content.

[0068] The content acquisition unit 105 acquires counseling content by transmitting the generated prompt to the LLM 300 and receiving a response thereto. The content acquisition unit 105 transmits the acquired counseling content to the user terminal 400.

[0069] According to this configuration, counseling can be provided based on information relating to the broadcast user's emotion estimation and information relating to the broadcast content. In other words, the information relating to the broadcast user's emotion estimation and information relating to the broadcast content indicate the situation in which the broadcast user is placed, making it possible to provide counseling that corresponds to that situation.

[0070] The acquisition unit 102 in the mental support server 100 generates information related to emotion estimation based on at least one of images and audio included in the distribution content distributed by the distributor user. As the information related to emotion estimation, the distributor user's emotional tendency is acquired. As this distributor user's emotional tendency, a positive tendency or a negative tendency is acquired.

[0071] A broadcast user's facial expression or voice often indicates emotions, and in the present disclosure, emotions are estimated from the facial expression or voice. Emotions range widely, including joy, anger, sadness, and happiness, but in the present disclosure, appropriate counseling for the broadcast user is made possible by narrowing down the emotions to two directions, positive and negative, and determining their tendencies. Of course, this is not limited to the two directions of positive and negative.

[0072] Furthermore, the acquisition unit 102 may acquire, as information related to emotion estimation, a questionnaire that is subjective information presented from the broadcast user after the broadcast content. The questionnaire reflects the actual emotions felt by the broadcast user, and counseling based on this questionnaire is more appropriate.

[0073] The acquisition unit 102 also generates information about the distribution content based on comments posted by other users on the distribution content. The information about the content indicates whether the comment is an aggressive or friendly Facebook post (content) toward the distribution user. Generally, comments on video distribution, etc., include not only friendly Facebook posts (comments) but also malicious and aggressive Facebook posts (comments). By providing counseling to the distribution user based on the content of these Facebook posts (comments), appropriate counseling becomes possible.

[0074] The generation unit 104 instructs the generation of a prompt that requests a response according to information on emotion estimation (e.g., positive / negative) or information on distribution content (aggressive / friendly FB).

[0075] This configuration enables appropriate counseling in accordance with information relating to emotion estimation of the broadcast user (emotions) and information relating to the broadcast content (content of comments).

[0076] In the present disclosure, the mental support server 100 further includes a knowledge DB 103 a (corresponding to a generation policy storage unit) that associates information related to emotion estimation of a distribution user, information related to distribution content, and a generation policy for counseling content. The control unit 103 determines the generation policy for counseling content by referring to the knowledge DB 103 a.

[0077] This knowledge DB 103a stores at least one of the broadcast user's emotional tendencies based on information about the broadcast user's sensitivities, and the broadcast user's behavioral tendencies based on information about broadcast content.

[0078] For example, the knowledge DB 103a associates a counseling content generation policy with the tendency of increasing or decreasing positivity, the tendency of increasing or decreasing negativity, the number of aggressive Facebook posts (or information indicating whether it is high or low), and the number of friendly Facebook posts (or information indicating whether it is high or low), respectively. The generation policy indicates a policy of comforting the distribution user or a policy of giving advice. The number of Facebook posts may include the number of posters, as there may be a case where the number of comments is high but the number of posters is low.

[0079] By using this, it is possible to decide on an appropriate counseling policy and have the LLM 300 respond accordingly. For example, even if there are many offensive Facebook posts, if the number of posters is small, or if some posters post a large number of posts, in such cases, it is possible to provide counseling that it is not something to be concerned about.

[0080] The device and method of the present disclosure have the following configuration.

[0081] [1] A device comprising: a reception unit that receives generation request information for counseling content for a distribution user of distribution content; an acquisition unit that acquires information related to emotion estimation of the distribution user and information related to the distribution content upon receiving the generation request information for the counseling content; a control unit that determines a generation policy for the counseling content based on the information related to emotion estimation of the distribution user and the information related to the distribution content; and a generation unit that generates a prompt that instructs the generation of the counseling content based on the generation policy for the counseling content.

[0082] [2] The device according to [1], wherein the acquisition unit generates the information related to the emotion estimation based on at least one of an image or a sound included in the distribution content distributed by the distributor user.

[0083] [3] The device according to [1] or [2], wherein the acquisition unit acquires an emotional tendency of the broadcast user as information related to the emotion estimation.

[0084] [4] The device according to any one of [1] to [3], wherein the information relating to the emotion estimation is subjective information presented by the distribution user after the distribution content.

[0085] [5] The device according to any one of [1] to [4], wherein the acquisition unit generates information about the distribution content based on comments posted by other users on the distribution content.

[0086] [6] The device according to [6], wherein the information about the content indicates whether the comment is offensive or unfriendly to the broadcast user.

[0087] [7] The device according to any one of [1] to [6], wherein the generation unit instructs the generation of a prompt requesting a response according to the information on the emotion estimation or the information on the distribution content.

[0088] [8] The device according to any one of [1] to [7], further comprising a generation policy storage unit that associates information regarding the emotion estimation of the distribution user, information regarding the distribution content, and a generation policy of the counseling content, and the control unit determines a generation policy of the counseling content by referring to the generation policy storage unit.

[0089] [9] The device described in [8], wherein the generation policy storage unit stores at least one of the emotional tendency of the broadcast user based on information regarding the broadcast user's sensitivity, and the tendency of behavior toward the broadcast user based on information regarding the broadcast content.

[0090]

[10] A method comprising: a receiving step of receiving generation request information for counseling content for a distribution user of distribution content; an acquisition step of acquiring information regarding emotion estimation of the distribution user and information regarding the distribution content upon receiving the generation request information for the counseling content; a control step of determining a generation policy for the counseling content based on the information regarding emotion estimation of the distribution user and the information regarding the distribution content; and a generation step of generating a prompt that instructs generation of the counseling content based on the generation policy for the counseling content.

[0091] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are connected directly or indirectly (e.g., via wire, wirelessly, etc.) and these multiple devices. The functional block may also be realized by combining the single device or multiple devices with software.

[0092] Functions include, but are not limited to, judgment, determination, assessment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0093] For example, the mental support server 100, distribution server 200, and user terminal 400 according to an embodiment of the present disclosure may function as computers that perform processing of the mental support method of the present disclosure. Fig. 12 is a diagram illustrating an example of the hardware configuration of the mental support server 100, distribution server 200, and user terminal 400 according to an embodiment of the present disclosure. The mental support server 100, distribution server 200, and user terminal 400 described above may be physically configured as computer devices including a processor 1001, memory 1002, storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

[0094] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configurations of the mental support server 100, the distribution server 200, and the user terminal 400 may be configured to include one or more of the devices shown in the figures, or may be configured to exclude some of the devices.

[0095] Each function of the acquisition unit 102, control unit 103, and generation unit 104 is realized by loading specified software (programs) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communication via the communication device 1004, and control at least one of reading and writing data in the memory 1002 and storage 1003.

[0096] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the above-mentioned acquisition unit 102, control unit 103, and generation unit 104 may be realized by the processor 1001.

[0097] The processor 1001 also reads programs (program code), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these programs. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. For example, the acquisition unit 102, the control unit 103, and the generation unit 104 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and similar implementations may be used for other functional blocks. While the above-described various processes have been described as being executed by a single processor 1001, they may also be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may also be transmitted from a network via a telecommunications line.

[0098] The memory 1002 is a computer-readable recording medium and may be configured by, for example, at least one of a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing the mental support method according to one embodiment of the present disclosure.

[0099] Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other appropriate medium including at least one of memory 1002 and storage 1003.

[0100] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc. to realize at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the above-mentioned reception unit 101 and content acquisition unit 105 may be realized by the communication device 1004. The communication device 1004 may be implemented with a transmission unit and a reception unit that are physically or logically separated.

[0101] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

[0102] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0103] Furthermore, the mental support server 100, the distribution server 200, and the user terminal 400 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0104] The notification of information is not limited to the aspects / embodiments described in the present disclosure and may be performed using other methods. For example, the notification of information may be performed by physical layer signaling (e.g., Downlink Control Information (DCI) and Uplink Control Information (UCI)), higher layer signaling (e.g., Radio Resource Control (RRC) signaling, Medium Access Control (MAC) signaling, broadcast information (Master Information Block (MIB) and System Information Block (SIB))), other signals, or a combination thereof. Furthermore, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

[0105] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0106] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0107] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0108] The aspects / embodiments described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0109] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0110] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0111] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0112] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0113] Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Furthermore, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0114] Furthermore, the information, parameters, etc. described in the present disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information. For example, a radio resource may be indicated by an index.

[0115] The names used for the above-described parameters are not intended to be limiting in any way. Furthermore, the mathematical expressions using these parameters may differ from those explicitly disclosed in this disclosure. The various channels (e.g., PUCCH, PDCCH, etc.) and information elements may be identified by any suitable names, and therefore the various names assigned to these various channels and information elements are not intended to be limiting in any way.

[0116] In this disclosure, the terms "Mobile Station (MS)," "user terminal," "User Equipment (UE)," "terminal," and the like may be used interchangeably.

[0117] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terminology.

[0118] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0119] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using one or more wires, cables, and / or printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0120] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0121] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0122] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0123] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0124] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0125] 100...Mental support server, 200...Distribution server, 300...LLM, 400...User terminal, 410...User terminal, 201...Communication unit, 202...Content storage unit, 203...Comment storage unit, 101...Reception unit, 102...Acquisition unit, 103...Control unit, 104...Generation unit, 102a...Emotion estimation model, 102b...Content understanding information DB, 103a...Knowledge DB, 105...Content acquisition unit.

Claims

1. An apparatus comprising: a reception unit that receives request information for generating counseling content for a distribution user of distribution content; an acquisition unit that acquires information regarding an estimation of the distribution user's emotions and information regarding the distribution content upon receiving the request information for generating counseling content; a control unit that determines a generation policy for the counseling content based on the information regarding the estimation of the distribution user's emotions and information regarding the distribution content; and a generation unit that generates a prompt that instructs the generation of the counseling content based on the generation policy for the counseling content.

2. The device according to claim 1, wherein the acquisition unit generates the information relating to the emotion estimation based on at least one of an image or a sound included in the distribution content distributed by the distributor user.

3. The device according to claim 1, wherein the acquisition unit acquires the emotional tendency of the broadcast user as information related to the emotion estimation.

4. The device according to claim 1, wherein the information regarding emotion estimation obtained by the acquisition unit is subjective information presented by the broadcast user after broadcasting the content.

5. The device according to claim 1, wherein the acquisition unit generates information about the distribution content based on comments posted by other users on the distribution content.

6. The device according to claim 5, wherein the information about the content indicates whether the comment is offensive or unfriendly to the broadcast user.

7. The device according to claim 1, wherein the generation unit issues an instruction to generate a prompt requesting a response according to the information regarding the emotion estimation or the information regarding the distributed content.

8. The device described in claim 1, further comprising a generation policy memory unit that associates information regarding the emotion estimation of the distribution user, information regarding the distribution content, and a generation policy for the counseling content, and the control unit determines the generation policy for the counseling content by referring to the generation policy memory unit.

9. The device according to claim 8, wherein the generation policy storage unit stores at least one of the emotional tendencies of the broadcast user based on information regarding the broadcast user's sensitivities, and the behavioral tendencies toward the broadcast user based on information regarding the broadcast content.

10. A method comprising: a receiving step of receiving request information for generating counseling content for a distribution user of distribution content; an acquisition step of acquiring information regarding an estimation of the distribution user's emotions and information regarding the distribution content upon receiving the request information for generating counseling content; a control step of determining a generation policy for the counseling content based on the information regarding the estimation of the distribution user's emotions and information regarding the distribution content; and a generation step of generating a prompt that instructs the generation of the counseling content based on the generation policy for the counseling content.

Images