Story generation device

Abstract

A story generation device of the present invention includes: an input unit; an emotion analysis unit; an event extraction unit; a story generation unit; and an output unit. The input unit receives input of text including two or more events, and a target score percentage that is a score regarding a desired emotion a user wants to experience. The emotion analysis unit analyzes each of the two or more events included in the text and obtains an emotion score that represents the magnitude of the emotion evoked by the event. The event extraction unit extracts a first event from the text such that the emotion score for each emotion in the extracted event approaches a score corresponding to the target score percentage. On the basis of the first event, the story generation unit generates a story in which the first event that evokes the same emotion does not occur consecutively more than a prescribed number of times. The output unit outputs the story.

Description

Story generation device

[0001] The embodiment relates to a story generation device.

[0002] For patients with anxiety disorders and depression, controlling negative emotions or changing emotional responses is difficult. Therefore, virtual reality (VR) systems using biofeedback for emotion control training have been proposed to help them adapt to their emotions. One such VR system for emotion control training is one in which the images presented to the user, such as a maze, are dynamically changed according to the user's emotions, thereby enabling training in emotion control.

[0003] S. Bermudez i Badia et al., "Toward Emotionally Adaptive Virtual Reality for Mental Health Applications," in IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 5, pp. 1877-1887, Sept. 2019

[0004] Simply presenting emotionally evocative events to a user tends to have only a temporary effect. In contrast, when there is a narrative element between events, each event gives meaning to the others, leading to the user empathizing with or relating to the characters in the story. This new perspective gained through empathy or re-evaluation causes the user to re-evaluate their own emotional response. The self-transformative effect resulting from this re-evaluation is known to be persistent.

[0005] The embodiment provides a narrative generation device for presenting narrative events that are likely to evoke emotions in the user.

[0006] One embodiment of a story generation device comprises an input unit, an emotion analysis unit, an event extraction unit, a story generation unit, and an output unit. The input unit receives input of text containing two or more events and a target score percentage, which is a score for the desired emotion that the user wants to experience. The emotion analysis unit analyzes each of the two or more events contained in the text and obtains an emotion score representing the magnitude of the emotion evoked by the event. The event extraction unit extracts a first event from the text such that the emotion score for each emotion in the extracted event approaches a score corresponding to the target score percentage. The story generation unit generates a story based on the first event, in which the first events that evoke the same emotion do not occur consecutively more than a predetermined number of times. The output unit outputs the story.

[0007] According to one embodiment, a narrative generation device is provided for presenting narrative events that are likely to evoke emotions in the user.

[0008] Figure 1 is a block diagram showing the configuration of a story generation device according to the first embodiment. Figure 2 is a flowchart showing the operation of the story generation device according to the first embodiment. Figure 3 is an image diagram of the emotion analysis unit. Figure 4 is an image diagram showing a specific example of paragraph insertion. Figure 5 is a flowchart showing the operation of the story generation device according to the second embodiment. Figure 6 is an image diagram showing a specific example of paragraph rearrangement. Figure 7 is a block diagram showing the configuration of the output unit in the story generation device according to the third embodiment. Figure 8 is a block diagram showing the configuration of the output unit in the story generation device according to the fourth embodiment. Figure 9 is a diagram showing an example of the VR input unit. Figure 10 is a flowchart showing the operation of the output unit in the fourth embodiment. Figure 11 is a diagram showing an example of a virtual space played back on an HMD. Figure 12 is a block diagram showing an example of the hardware configuration of a computer equipped with the story generation device according to the embodiment.

[0009] The embodiments will be described below with reference to the drawings.

[0010] (First Embodiment) First, the first embodiment will be described. Figure 1 is a block diagram showing the configuration of a story generation device according to the first embodiment. The story generation device 1 includes an input unit 11, an emotion analysis unit 12, an event extraction unit 13, a story generation unit 14, and an output unit 15.

[0011] The input unit 11 accepts text and target score percentage input.

[0012] The text in the embodiment includes text of events arranged in chronological order. The events may be separated into paragraphs. The events may be those that easily evoke the user's emotions, such as, "I fell into the mud on my way to school. People around me saw me, and I was embarrassed," or "I made a mistake at my part-time job and got scolded." The events are not necessarily limited to everyday events. Any event that can evoke the user's emotions may include extraordinary events. The input unit 11 may, for example, display a UI (User Interface) on the display that presents several texts to the user and accept the text selected by the user as input.

[0013] The target score percentage is the percentage of the target emotional score for the emotions the user wants to experience in the story generated by the story generation device 1. The emotional score is a score that represents the magnitude of the emotions evoked by an event. The target score percentage can take values ​​from 0 to 1. Hereinafter, the target score percentage is s ａ = {s １ ,s ２ , ..., s Ｎ It is sometimes written as}. 'a' is a subscript corresponding to emotions that the user may experience, such as "sadness," "anger," and "joy." ａ = 1 indicates that the magnitude of the emotion a that can be experienced is at its maximum. On the other hand, s ａ= 0 indicates not experiencing emotion a. For example, the input unit 11 displays a UI including input fields for a target score ratio for each emotion on a display, and can accept numerical values input into the input fields as input of emotion scores. The numerical value input into the input field is not limited to a value between 0 and 1, and may be a percentage value of the target score ratio, that is, a value from 0% to 100%.

[0014] The emotion analysis unit 12 analyzes each event included in the text for which input is accepted by the input unit 11, and calculates an emotion score representing the magnitude of emotion evoked by each event. Then, the emotion analysis unit 12 outputs the emotion mainly evoked by each event and the emotion score of the emotion. The emotion analysis unit 12 extracts text of each paragraph, that is, each event, from the input text, inputs the extracted event text into an emotion analysis model using pre-trained BERT (Bidirectional Encoder Representations from Transformers) for example, and obtains an emotion score for each paragraph, that is, each event. Hereinafter, the emotion score is S ｐａ may be denoted as. p is a subscript representing a paragraph number. That is, the emotion score S ｐａ is represented as a matrix shown in the following formula (1). As shown in formula (1), the emotion score S ｐａ stores the corresponding emotion score S ａ for each paragraph. Therefore, the emotion score S ｐａ has a number of rows that matches the number n of paragraphs constituting the input text, and a number of columns that matches the number N of emotions that a user can experience.

[0015] Here, the emotion analysis unit 12 does not necessarily need to use an emotion analysis model employing BERT to analyze emotions for each event. That is, the emotion analysis method in the emotion analysis unit 12 is not limited to a specific method. For example, the emotion analysis unit 12 may split a paragraph into sentence and word levels and calculate an emotion score using an emotion dictionary based on the method disclosed in Hisatsugu Sugawara et al., "Emotion Extraction from Japanese Text", Proceedings of the 23rd Annual Conference of the Japanese Society for Artificial Intelligence (2009).

[0016] The event extraction unit 13 extracts paragraphs from the text acquired by the input unit 11 based on the sentiment score S for each event obtained by the sentiment analysis unit 12 ｐａ and the target score ratio s acquired by the input unit 11 ａ . Specifically, for each sentiment a in the extracted p-th paragraph, the event extraction unit 13 solves an optimization problem for extracting paragraphs such that the sentiment score S ｐａ for each sentiment a approaches a score corresponding to the target score ratio s ａ for each sentiment a, thereby extracting paragraphs. The objective function for optimization may be, for example, the following formula (2). Here, c ｐａ is a matrix representing whether or not the p-th paragraph is to be extracted for sentiment a, and is represented by the following formula (3). An element of c ｐａ takes a discrete value of 0 or 1. For example, 0 indicates that the p-th paragraph is not an extraction target for the corresponding sentiment a, and 1 indicates that the p-th paragraph is an extraction target for the corresponding sentiment a. The event extraction unit 13 optimizes the value of each element of c ｐａ so as to minimize the value of formula (2). Although not limited thereto, for example, integer linear programming (ILP) can be used for the optimization.

[0017] The story generation unit 14 generates a story based on the paragraphs extracted by the event extraction unit 13. After sorting the paragraphs extracted by the event extraction unit 13, that is, the paragraphs whose c ｐａ element is 1 in chronological order, when paragraphs that primarily evoke the same sentiment are consecutive, the story generation unit 14 generates a story by inserting another paragraph between these paragraphs that primarily evoke the same sentiment. Said another paragraph is a paragraph that is not extracted by the event extraction unit 13, and primarily evokes a sentiment different from that of the consecutive paragraphs, and may be a paragraph with a low sentiment score.

[0018] The output unit 15 outputs the story generated by the story generation unit 14. Story output may include various processes such as displaying story text on a display, displaying video related to the story text on a display, and storing story text data in a storage device.

[0019] Next, the operation of the story generation apparatus 1 according to the first embodiment will be described. FIG. 2 is a flowchart showing the operation of the story generation apparatus 1 according to the first embodiment. Here, for the sake of simplification of description, it is assumed below that emotions evoked by the story in the user are classified into two categories: "positive" and "negative". "Positive" may be emotions felt when the user is in a good mood, such as joy, affection, interest, and peace. On the other hand, "negative" may be emotions felt when the user is in a bad mood, such as anger, sadness, fear, and shame. In the following description, as an example, it is assumed that a=1 corresponds to "positive" and a=2 corresponds to "negative".

[0020] In step S1, the input unit 11 receives input of text and a target score ratio. Here, when emotions are classified into either "positive" or "negative", it is only necessary that values for "positive" and "negative" be input as the target score ratio. Since "positive" and "negative" are paired emotions, the value of the target score ratio for one may be automatically determined based on the input of the target score ratio for the other. For example, the target score ratio for the other may be determined as a value obtained by subtracting the input target score ratio [%] from 100 [%].

[0021] In step S2, the sentiment analysis unit 12 analyzes emotions mainly evoked from respective events included in the text. Then, the sentiment analysis unit 12 obtains the sentiment score s of each paragraph ｐａThe data is output to the event extraction unit 13. Figure 3 is an image of the sentiment analysis unit 12. For example, if the text is divided into paragraphs for each event, the sentiment analysis unit 12 extracts the text t from each paragraph and inputs it into the sentiment analysis model 12a. One example of the sentiment analysis model 12a is a sentiment analysis model that uses a pre-trained BERT, and outputs whether positive or negative emotions are mainly evoked from the input event text, and the sentiment score for each emotion. The sentiment analysis unit 12 repeats this process for each paragraph, thereby determining the emotion that is mainly evoked in each paragraph and the sentiment score S for each emotion in the paragraph. ａ = {S １ , S ２ Emotion score S that stores} ｐａ The output is as follows. Here, in the illustrative diagram in Figure 3, for simplicity, an example is shown in which only the emotion scores for the emotions primarily evoked by the paragraph text are output. In reality, the emotion scores for each emotion specified as the target score percentage may be output.

[0022] Furthermore, some emotion analysis models 12a using BERT output "natural" as an emotion classification, in addition to positive and negative. When using such an emotion analysis model 12a, the emotion analysis unit 12 may delete paragraphs for which the emotion analysis model 12a outputs "natural" without including them as targets for extraction by the event extraction unit 13.

[0023] In step S3, the event extraction unit 13 extracts the emotion score S for each event obtained by the emotion analysis unit 12. ｐａ and the target score percentage s obtained by the input unit 11 ａ Based on this, paragraphs are extracted from the text acquired by the input unit 11. The event extraction unit 13 extracts paragraphs by solving the optimization problem of equation (2), for example, using ILP. The paragraphs to be extracted are, as a result of solving the optimization problem, c ｐａ This is a paragraph where element 1. The event extraction unit 13 is C ｐａ The paragraph number p of the paragraph in which the element is 1, and the emotion score S obtained by the emotion analysis unit 12. ｐａThe output is sent to the story generation unit 14. Note that, as an optimization constraint, the number of paragraphs to be extracted may be set to be between 5 and 10, and weighting may be applied to paragraphs with higher scores. The constraint on the number of paragraphs to be extracted is set according to the length of the story desired by the user, and is intended to avoid a situation where the optimization problem cannot be solved. In the example, the number of paragraphs to be extracted is set to be between 5 and 10, but the number of paragraphs to be extracted is not limited to a specific number.

[0024] In step S4, the story generation unit 14 performs C ｐａ Paragraph number p, where element 1, and emotion score S ｐａ Based on the text input from the input unit 11, a story is generated. Specifically, the story generation unit 14 generates a story. ｐａ The text of paragraphs where element 1 is extracted and sorted in chronological order. When sorted in chronological order, if a predetermined number of paragraphs, for example three or more consecutively, primarily evoke the same emotion, the narrative generation unit 14 may extract and insert paragraphs that are not the extraction target but primarily evoke a different emotion and have a low emotion score between the paragraphs that consecutively evoke the same emotion. If there are paragraphs that evoke different emotions and have low emotion scores between multiple paragraphs, the narrative generation unit 14 may extract and insert only the paragraph with the lowest emotion score among them between the original paragraphs. If there are no paragraphs that primarily evoke different emotions and have a low emotion score between paragraphs that consecutively evoke the same emotion, the narrative generation unit 14 may extract and insert the paragraph closest to the multiple paragraphs that consecutively evoke the same emotion, which evokes a different emotion and has a low emotion score, between the original paragraphs. Figure 4 is an image diagram showing a specific example of paragraph insertion. In Figure 4, when the paragraphs to be extracted are sorted, there is a sequence of paragraphs that evoke a "negative" emotion. In this case, the story generation unit 14 inserts paragraphs with low emotion scores that evoke a different emotion, a "positive" emotion, from among the paragraphs that were not selected for extraction. After this processing, the story generation unit 14 outputs the text of the story, which is composed of the extracted paragraphs, i.e., events, to the output unit 15.

[0025] In step S5, the output unit 15 outputs the story generated by the story generation unit 14. After that, the process shown in Figure 2 is completed. For example, the output unit 15 can perform various processes such as displaying the story text on a display, displaying images related to the story text on a display, or storing the story text data in a storage device.

[0026] As described above, in the first embodiment, an analysis of the emotions evoked by each paragraph, i.e., each event, of the input text is performed. Based on the results of this emotion analysis, paragraphs are extracted from the text in a way that approaches the target score percentage entered by the user.

[0027] Here, even if events that evoke the same emotion are presented to the user consecutively, the effect of evoking that emotion tends to be temporary, and it is not guaranteed that an emotional response will be achieved. Therefore, in this embodiment, when paragraphs that evoke the same emotion are extracted consecutively, paragraphs that evoke a different emotion are inserted between them. This creates a narrative between the events, and it is expected that the user will re-evaluate their own emotional response. In this way, the emotions of the user experiencing the story can be balanced.

[0028] Furthermore, in the first embodiment, paragraphs inserted between paragraphs are not targeted for extraction. In other words, they are different from the paragraphs used to obtain the target score percentage for emotion. In this embodiment, paragraphs with low emotion scores among the paragraphs not targeted for extraction are selected for extraction, so it is expected that they will have less impact on the user's emotions. On the other hand, the effect of creating a narrative is expected.

[0029] (Second Embodiment) Next, a second embodiment will be described. Here, explanations of parts that overlap with the second embodiment will be simplified or omitted as appropriate. For example, the block diagram showing the configuration of the story generation device according to the second embodiment may be the same as that of Figure 1. Therefore, its explanation will be omitted. The second embodiment differs from the first embodiment in the method of generating the story after the extraction of paragraphs, i.e., events.

[0030] Figure 5 is a flowchart showing the operation of the story generation device 1 according to the second embodiment. Here, in the following description, the emotions evoked by the story by the user will be classified into two categories: "positive" and "negative". Also, the processes in steps S1-S3 and S5 of Figure 5, which are denoted by the same reference numerals as in Figure 2, may be the same as the processes of the first embodiment described in Figure 2. Therefore, the explanation of the processes in steps S1-S3 and S5 will be omitted.

[0031] In step S14 after paragraph extraction, the story generation unit 14 performs C ｐａ Paragraph number p, where element 1, and emotion score S ｐａ Based on the text input from the input unit 11, a story is generated. In the second embodiment, the story generation unit 14 is C ｐａ The text of paragraphs where element 1 is extracted and the extracted texts are arranged. The order of the texts does not necessarily have to be chronological. When the texts are arranged, if a predetermined number of paragraphs, for example three or more consecutively, primarily evoke the same emotion, the story generation unit 14 replaces one of the paragraphs that evokes the same emotion with the paragraph closest to it that evokes a different emotion. Figure 6 is an image diagram showing a specific example of paragraph replacement. In Figure 6, as in Figure 4, there are consecutive paragraphs that evoke the "negative" emotion. In this case, the story generation unit 14 replaces one paragraph that evokes the "negative" emotion with a paragraph that evokes a different emotion, "positive," from the paragraphs to be extracted. After this processing, the story generation unit 14 outputs the text of the story, which is composed of the extracted paragraphs, i.e., events, to the output unit 15.

[0032] As explained above, in the second embodiment, to prevent consecutive paragraphs that primarily evoke the same emotion, one paragraph in a sequence is replaced with a paragraph from the extracted paragraphs that evokes a different emotion. This creates a narrative between events, and is expected to lead to a re-evaluation of the user's own emotional response. In this way, the user's emotions as they experience the story can be balanced. Furthermore, in the second embodiment, the paragraphs inserted between paragraphs are the extracted paragraphs. Therefore, it is easier to obtain an experience of the target score percentage entered by the user.

[0033] (Third Embodiment) Next, a third embodiment will be described. Figure 7 is a block diagram showing the configuration of the output unit in the story generation device according to the third embodiment. The configuration other than the output unit 15 in the third embodiment may be those shown in Figure 1.

[0034] In the third embodiment, the output unit 15 includes a video correction unit 151 and a playback unit 152. In the third embodiment, the output unit 15 receives the story text generated by the story generation unit 14. The method for generating the story text may be the method described in the first embodiment or the method described in the second embodiment.

[0035] The video editing unit 151 takes video and story text as input. Here, the video input to the video editing unit 151 consists of multiple video scenes. Each video scene is assigned an ID. The video may be input by the user via the input unit 11, or it may be pre-stored in a storage device (not shown) of the video editing unit 151. The video may be input at the time the user experiences the video of the story, or it may be at a time before the user experiences the video of the story. When the video is input at the time the user experiences the video of the story, the output unit 15 processes it in real time.

[0036] In the third embodiment, for example, each paragraph of the text received by the input unit 11 is associated with an ID for identifying a video scene. When the video editing unit 151 receives the story text, it also receives an ID for identifying a video scene associated with each paragraph that makes up the story. The video editing unit 151 divides the video into video scene units based on the playback time of each video scene. The video editing unit 151 then extracts video scenes from the divided video scenes that have an ID corresponding to the video scene ID associated with the story text, rearranges the extracted video scenes in chronological order, and concatenates them to generate the story video.

[0037] The playback unit 152 plays back the video of the story obtained by the video correction unit 151. The playback destination for the video of the story can be various displays such as a monitor or a head-mounted display (HMD).

[0038] As described above, in the third embodiment, based on the narrative text, video scenes that evoke the user's emotions are extracted from the video to generate a narrative video. When such a narrative video is presented to the user, the user can experience the story as a video.

[0039] (Fourth Embodiment) Next, a fourth embodiment will be described. Figure 8 is a block diagram showing the configuration of the output unit in the story generation device according to the fourth embodiment. The configuration other than the output unit 15 in the fourth embodiment may be those shown in Figure 1.

[0040] In the fourth embodiment, the output unit 15 includes a VR image correction unit 153 and a VR playback unit 154. Similar to the output unit 15 in the third embodiment, the output unit 15 in the fourth embodiment receives the story text generated by the story generation unit 14. The method for generating the story text may be the method described in the first embodiment or the method described in the second embodiment.

[0041] The VR video editing unit 153 takes the VR video, the story text, and the input from the VR input unit 16 as input.

[0042] The VR video input to the VR video correction unit 153 consists of multiple VR video scenes. Each VR video scene is assigned an ID. Each VR video scene is configured to allow the user to experience a single event. The VR video may be input by the user via the input unit 11, or it may be pre-stored in a storage device (not shown) of the video correction unit 151. The VR video may be input at the time the user experiences the story's video, or it may be at a time before the user experiences the story's VR video. When the VR video is input at the time the user experiences the story's VR video, the output unit 15 processes it in real time.

[0043] The VR input unit 16 is a variety of input devices for the user to control the characters they are experiencing in the VR video, and it inputs the received input to the VR video correction unit 153. Figure 9 shows an example of the VR input unit 16. The VR input unit 16 may be, for example, a rod-shaped controller that the user holds in their hand and operates. The controller has N (N is a natural number) buttons. The VR input unit 16 receives button signals O in response to the user's button operation on the controller. ｕ = {O １ , O ２ , ..., O Ｎ The} is output as sensor information to the VR image correction unit 153 of the story generation device 1. Button signal O ｕ This is a time-series signal represented by 0 or 1, which becomes 1 while the corresponding button is pressed or at the moment it is pressed. The controller may have M (M is a natural number) sticks in place of or in addition to the buttons. In this case, the VR input unit 16 receives the stick signal K associated with the user's stick operation on the controller as sensor information. ｓ = {K １ , K ２ , ..., K Ｍ The stick signal K is output as sensor information to the VR image correction unit 153 of the story generation device 1. ｓThis is, for example, a time-series signal corresponding to the direction and amount of stick operation, and its value changes according to the direction and amount of stick operation. In addition, the VR input unit 16 may include an HMD 154a as an example of the VR playback unit 154. In this case, the VR input unit 16 sets the head rotation angle θ = {θ ｘ , θ ｙ , θ ｚ The sensor information P = {P ｘ , P ｙ , P ｚ This could be a device that outputs the sensor information to the VR image correction unit 153. Thus, the VR input unit 16 shown in Figure 9 is just one example. Any input device may be used as the VR input unit 16.

[0044] In the fourth embodiment, for example, each paragraph of the text received by the input unit 11 is associated with an ID for identifying a VR video scene. When the VR video modification unit 153 receives the text of the story, it also receives an ID for identifying a VR video scene associated with each paragraph that makes up the story. The VR video modification unit 153 extracts VR video scenes having IDs that correspond to the VR video scene IDs associated with the text of the story, rearranges the extracted VR video scenes in chronological order, and then concatenates them to generate a VR video of the story.

[0045] Furthermore, the VR video editing unit 153 updates the VR video so that the input received by the VR input unit 16 is reflected in the VR video scene. For example, if a button corresponding to an operation in which the character being experienced touches and grasps an object in the VR space is pressed from the VR input unit 16, the VR video editing unit 153 updates the video so that the character being experienced is shown grasping the object. After the experience in the VR video scene is completed, the VR video editing unit 153 updates the VR video scene to be played to the next VR video scene.

[0046] The VR playback unit 154 plays back the VR video of the story obtained by the VR video correction unit 153. The playback destination of the VR video of the story may be various displays such as a head-mounted display (HMD) 154a. In this case, the VR playback unit 154 transmits video information to the HMD 154a for playback of the VR video on the HMD 154a. The VR playback unit 154 may also play back sound in conjunction with the playback of the VR video. The playback destination of the sound may be, for example, the speaker 154b shown in Figure 9. In this case, the VR playback unit 154 transmits acoustic information to the speaker 154b for playback of sound on the speaker 154b. The playback destination of the sound is not limited to the speaker 154b; it may also be earphones or the like.

[0047] Figure 10 is a flowchart showing the operation of the output unit 15 in the fourth embodiment. Prior to the processing shown in Figure 10, the VR video modification unit 153 generates a VR video of the story based on the story text received from the story generation unit 14. In step S101 after the generation of the VR video of the story, the VR video modification unit 153 selects the first VR video scene from the VR video of the story.

[0048] In step S102, the VR video editing unit 153 outputs the selected VR video scene to the VR playback unit 154. The VR playback unit 154 plays a virtual space on the HMD 154a, for example, based on the input VR video scene. Figure 11 shows an example of a virtual space played on the HMD 154a. The virtual space may contain an avatar A representing the user as a character in the experience, and various objects O1 and O2 that avatar A can interact with. The user can interact with objects O1 and O2 through avatar A. Objects O1 and O2 may also operate in real time even without interaction from avatar A. The VR video editing unit 153 advances the VR video according to a pre-prepared story. Figure 10 shows an example where a video of avatar A existing in the virtual space is displayed, but on the other hand, the virtual space may be a video of the space as seen from the viewpoint of avatar A. Furthermore, as mentioned above, the VR playback unit 154 can reproduce sound from the speaker 154b. In this case, the sound to be reproduced may be determined considering the location of the hearing of the avatar A being experienced, the location of the sound source, the direction of the sound source, the volume, the frequency of the sound, etc.

[0049] In step S103, the VR image correction unit 153 determines whether or not there has been user operation via the VR input unit 16. If it is determined in step S103 that there has been user operation, the process proceeds to step S104. If it is determined in step S103 that there has been no user operation, the process proceeds to step S105.

[0050] In step S104, the VR video editing unit 153 advances the VR video scene in response to user operations. For example, the VR video editing unit 153 moves avatar A in the virtual space or moves objects that are affected by avatar A's interference in response to controller button operations, stick operations, etc. Avatar A's movements include actions such as making avatar A walk or making the avatar hold objects in the virtual space. Furthermore, avatar A's movements may also include actions that are difficult for a real body to perform, such as flying. In addition, objects in the virtual space may perform actions such as rotating or moving with pre-prepared animations when they meet pre-specified conditions in the progression of the story, such as contact with avatar A. Depending on the conditions, sound may also be played from earphones or speakers. Furthermore, depending on the conditions, vibration stimuli may be presented to the user using a vibrator or the like.

[0051] In step S105, the VR video correction unit 153 determines whether the currently playing VR video scene has ended. If it is determined in step S105 that the currently playing VR video scene has not ended, the process returns to step S102. If it is determined in step S105 that the currently playing VR video scene has ended, the process proceeds to step S106.

[0052] In step S106, the VR video correction unit 153 determines whether or not there is another VR video scene for the user to experience. If it is determined in step S106 that there is another VR video scene, the process proceeds to step S107. If it is determined in step S106 that there is no next VR video scene, the process shown in Figure 10 ends.

[0053] In step S107, the VR video correction unit 153 selects the next VR video scene.

[0054] In step S108, the VR video correction unit 153 moves the user to the next VR video scene. After that, the process returns to step S102. Specifically, in step S108, the VR video correction unit 153 displays avatar A at the initial location of the next virtual space.

[0055] As described above, in the fourth embodiment, based on the narrative text, VR video scenes that evoke the user's emotions are extracted from the VR video to generate a narrative VR video. By presenting such a narrative VR video to the user, the user can experience a more immersive story within the VR space.

[0056] (Modification 1) The following describes modifications of the embodiment. The embodiment described above provides an example of a narrative experience. In contrast, the technology of the embodiment can be applied to experiences other than narratives. For example, the text may be a list of tasks that the user needs to perform, such as a to-do list. Normally, tasks are performed in order of priority or shortest deadline, but it is also possible to perform tasks in an order that matches the desired emotion. In such cases, if there are consecutive tasks in the to-do list that evoke the same emotion, it is possible to achieve emotional balance by inserting tasks that evoke different emotions. Furthermore, by controlling the number of tasks extracted, it is possible to avoid selecting some tasks, and applications such as proceeding with work while maintaining emotional balance are also possible.

[0057] (Modification 2) In this embodiment, the story is generated based on the results of the emotional analysis, but it is not necessarily required that the story be generated based on the results of the emotional analysis. For example, stress levels can be used instead of emotions. For example, the actions of a person can be described in detail as text beforehand, and the results of measuring salivary amylase immediately after performing the action can be recorded. Since it is generally known that salivary amylase increases with stress, a stress dictionary can be created that associates action text with stress levels based on the relationship between action and salivary amylase. Based on this stress dictionary, stress levels can be calculated from the input action text. Existing language models such as BERT and GPT can be used as a model that outputs the closeness between the action text recorded in the stress dictionary and the input action text as a numerical value. In this way, stress levels can be pseudo-calculated from the input action text. Paragraphs of action text, i.e., events, can be extracted based on the stress levels calculated from the input action text in this way. If paragraphs with similar stress levels are consecutive, the stress levels can be balanced by inserting paragraphs with different stress levels.

[0058] (Modification 3) In this embodiment, no particular restrictions are placed on emotional fluctuations in paragraph extraction. Therefore, depending on the extracted paragraph, a significant decrease or increase in emotion may occur. In response to this, optimization may be performed with a constraint that the maximum or minimum value of the cumulative value of each emotion score does not exceed a threshold. For example, the event extraction unit 13 may perform optimization according to an objective function that includes a penalty term to keep the emotion score within a threshold. That is, the event extraction unit 13 calculates the emotion score S for each emotion a. ｐａ If the cumulative value exceeds a predetermined threshold, a penalty value obtained by multiplying the absolute value of the cumulative value minus the threshold by a coefficient may be added to the objective function before optimization. The coefficient can be a predetermined value. Alternatively, the coefficient may also be included in the optimization target.

[0059] Figure 12 is a block diagram showing an example of the hardware configuration of a computer equipped with a story generation device according to an embodiment. As shown in Figure 12, the computer includes, for example, a processor 201, ROM (read-only memory) 202, RAM (random access memory) 203, a communication module 204, a display 205, an input interface 206, and storage 207. The computer may be, for example, a terminal device such as a smartphone or a PC (personal computer). Alternatively, the computer may be a device built on a cloud environment.

[0060] The processor 201 is a processing circuit that controls the overall operation of the computer. The processor 201 executes a program stored in, for example, the storage 207. The processor 201 can also operate as an input unit 11, an emotion analysis unit 12, an event extraction unit 13, a story generation unit 14, and an output unit 15 by executing a program stored in the storage 207. The processor 201 is, for example, a CPU. The processor 201 may also be an MPU, GPU, ASIC, FPGA, etc. The processor 201 may be a single CPU, or multiple CPUs, etc.

[0061] ROM 202 is a non-volatile semiconductor memory. ROM 202 stores the computer's startup program and various settings. RAM 203 is a volatile memory. RAM 203 is used, for example, as working memory during processing in the processor 201.

[0062] The communication module 204 is a circuit equipped with an interface for the computer to communicate with external devices. The communication module 204 may be a wireless communication module or a wired communication module. The communication module 204 can be used to communicate with output devices such as the HMD 154a and speaker 154b, and input devices such as the VR input unit 16 of a controller.

[0063] The display 205 is an LCD display, an organic EL display, etc. The display 205 can be used to display a UI for inputting text and target score percentages, to display story text, etc.

[0064] The input interface 206 can be various input devices such as a mouse, keyboard, or touch panel. The input interface 206 can be used for input on a UI displayed on the display 205, etc.

[0065] Storage 207 is a non-volatile storage device such as a hard disk or flash memory. Storage 207 stores various programs, such as the computer's operating system (OS) and programs for operating the computer as a story generation device 1. Storage 207 may also store the sentiment analysis model 12a for sentiment analysis. On the other hand, storage 207 does not have to store the sentiment analysis model 12a. In this case, the sentiment analysis model 12a can be stored in external storage that the computer can communicate with.

[0066] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention.

[0067] 1...Story generation device 11...Input unit 12...Emotion analysis unit 12a...Emotion analysis model 13...Event extraction unit 14...Story generation unit 15...Output unit 16...VR input unit 151...VR video correction unit 152...Playback unit 153...VR video correction unit 154...VR playback unit 154a...Head-mounted display (HMD) 154b...Speaker 201...Processor 202...ROM 203...RAM 204...Communication module 205...Display 206...Input device 207...Storage

Claims

1. A story generation device comprising: an input unit that accepts input of text containing two or more events and a target score ratio which is a score for the desired emotion that the user wants to experience; an emotion analysis unit that analyzes each of the two or more events contained in the text and obtains an emotion score representing the magnitude of the emotion evoked by the event; an event extraction unit that extracts a first event from the text such that the emotion score for each emotion in the extracted event approaches a score corresponding to the target score ratio; a story generation unit that generates a story based on the first event in which the first events that evoke the same emotion do not occur consecutively for a predetermined number of times or more; and an output unit that outputs the story.

2. The story generation device according to claim 1, wherein the story generation unit arranges the first events in chronological order, and then inserts a second event, which is not extracted but evokes a second emotion different from the first emotion and has a low emotion score, at a point where the first events that evoke a first emotion are consecutive.

3. The story generation device according to claim 1, wherein the story generation unit, after arranging the first events, replaces one of the first events in a location where the first events that evoke a first emotion are consecutive with a second event that has been extracted and evokes a second emotion different from the first emotion.

Images