Image generation device, image generation method, and image generation program

Abstract

To realize efficient accumulation of experiences regarding a job through a user's simulated job experience.SOLUTION: A video generating device 30 comprises an acquisition part 31 that acquires action information 311 representing an action of a testee for a first subjective video 331 when the first subjective video 331 of virtual reality for a simulated job experience is presented to the testee, a determination part 32 that determines a future state change of the object included in the first subjective video 331 that is caused by the action of the testee based on the scenario determination criteria 322 that represents the correlation between the action information 311 and the state change information 321 that represents the future state change of the object, and a generation part 33 that generates a second subjective video 332 following the first subjective video 331 based on the state change information 321 and the video generation criteria 333, and presents the generated second subjective video 332 to the testee.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an image generation device, an image generation method, and an image generation program. [Background technology]

[0002] BACKGROUND ART Technology for providing virtual reality is rapidly advancing in various fields, allowing people to simulate experiences of worlds that are difficult for them to experience in real life.

[0003] As a technology related to this technology, Patent Document 1 discloses a virtual reality service providing system that enables presentation in a virtual reality world that is integrated with the real world. This system stores multiple virtual reality objects and information indicating virtual reality services available to the user. This system determines the placement positions of the virtual reality objects in a virtual space based on surrounding situation information that indicates the user's surroundings. This system then generates coded data for images to be displayed on a display panel of a head-mounted display worn by the user, based on user state information that indicates the user's state and the placement positions of the virtual reality objects.

[0004] Furthermore, Patent Document 2 discloses a virtual reality presentation system that takes into consideration the impact of a user's actions in a virtual reality world on the real world. This system detects the user's surroundings and state, and generates an image of a virtual reality object based on the detected user state. The system displays the generated image and outputs audio associated with the displayed virtual reality object. The system then alerts the user by video or audio based on the detected surroundings of the user. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2016-045814 [Patent Document 2] Japanese Patent Application Laid-Open No. 2014-170330 Summary of the Invention [Problem to be solved by the invention]

[0006] People engaged in various occupations typically improve their occupational skills by accumulating work experience. Therefore, while it would be desirable to efficiently accumulate work experience that significantly improves their occupational skills, in reality, opportunities to gain such experience are limited, making it difficult for individuals to accumulate the desired amount of work experience. Furthermore, it is particularly difficult to accumulate the desired amount of work experience in dangerous occupations, such as police work. One possible solution to this problem would be to simulate the work experience using technology related to providing virtual reality, as disclosed in Patent Documents 1 and 2, as mentioned above. However, at present, there is no system that realizes such a simulated work experience, and Patent Documents 1 and 2 do not specifically mention technology that realizes a simulated work experience.

[0007] A primary object of the present invention is to provide an image generation device or the like that enables a user to simulate a job and thereby efficiently accumulate experience related to that job. [Means for solving the problem]

[0008] An image generation device according to one embodiment of the present invention includes an acquisition means for acquiring behavioral information representing the behavior of a participant in response to a first subjective image of virtual reality for simulating a job, when the first subjective image is presented to the participant; a determination means for determining future changes in the state of an object included in the first subjective image that will occur due to the participant's behavior, based on a scenario determination criterion that represents the relationship between the behavioral information and state change information representing the future changes in the state of the object; and a generation means for generating a second subjective image that follows the first subjective image, based on the state change information and image generation criterion, and presenting the generated second subjective image to the participant.

[0009] In another aspect of achieving the above object, an image generation method according to one embodiment of the present invention, when a first subjective image of virtual reality for simulating a job experience is presented to an experiencer by an information processing device, acquires behavioral information representing the behavior of the experiencer in response to the first subjective image, determines future changes in the state of an object included in the first subjective image that will occur due to the experiencer's behavior based on a scenario determination criterion that represents the relationship between the behavioral information and state change information representing the future changes in the state of the object, generates a second subjective image that follows the first subjective image based on the state change information and the image generation criterion, and presents the generated second subjective image to the experiencer.

[0010] In addition, in a further aspect of achieving the above-mentioned object, an image generation program according to one embodiment of the present invention causes a computer to execute the following processes: an acquisition process for acquiring behavioral information representing the behavior of a participant in response to a first subjective image of virtual reality for simulating a job, when the first subjective image is presented to the participant; a determination process for determining future changes in the state of an object included in the first subjective image that will occur due to the participant's behavior, based on a scenario determination criterion that represents the relationship between the behavioral information and state change information representing the future changes in the state of the object; and a generation process for generating a second subjective image that follows the first subjective image, based on the state change information and image generation criterion, and presenting the generated second subjective image to the participant.

[0011] Furthermore, the present invention can also be realized by a computer-readable, non-volatile recording medium on which such an image generation program (computer program) is stored. [Effects of the Invention]

[0012] According to the present invention, it is possible for a user to have a simulated experience of a job, thereby efficiently accumulating experience related to that job. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing the configuration of an image generation device 10 according to a first embodiment of the present invention. [Figure 2] 10 is a diagram illustrating an example of a scenario setting menu screen that the acquisition unit 11 according to the first embodiment of the present invention displays on the screen 400 of the management terminal device 40. FIG. [Figure 3] 1 is a diagram illustrating an example of a tree-structured graph representing a scenario determination criterion 142 according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating an example of data of status change information 143 according to the first embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing an example of the determination unit 12 according to the first embodiment of the present invention sequentially determining subsequent scenes in accordance with an action selected by an experiencer for a scene. [Figure 6A] 1 is a flowchart (1 / 2) showing the operation of the image generation device 10 according to the first embodiment of the present invention. [Figure 6B] 10 is a flowchart (2 / 2) showing the operation of the image generation device 10 according to the first embodiment of the present invention. [Figure 7] FIG. 10 is a block diagram showing the configuration of an image generation device 30 according to a second embodiment of the present invention. [Figure 8] FIG. 9 is a block diagram showing the configuration of an information processing device 900 that can realize an image generation device according to each embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0015] First Embodiment FIG. 1 is a block diagram showing the configuration of an image generating device 10 according to a first embodiment of the present invention. Image generating device 10 is an information processing device that generates educational image content (e.g., animation) for police officers, for example, by using virtual reality technology to enable police officers to simulate the investigation of a crime or accident. The educational image content is assumed to be subjective image from the perspective of a participant (police officer) experiencing the simulated investigation of a crime or accident. In this embodiment, this subjective image may hereinafter be referred to simply as image. The subjective image may also include audio.

[0016] 1, the video generation device 10 is communicably connected to a VR (Virtual Reality) goggles 21, a camera 22, a microphone 23, an input device 24, and a management terminal device 40. The VR goggles 21 are a head-mounted display worn by a participant who simulates the investigation of a crime or accident by viewing the educational video content described above. Instead of the VR goggles 21, the participant may view the video content using a general monitor.

[0017] The camera 22 captures the user's actions (body movements, etc.) at a predetermined scene (timing) in the video content. The microphone 23 collects sounds made by the user at a predetermined scene in the video content. The input device 24 is a device such as a joystick, mouse, or keyboard that the user uses to perform input operations on a menu screen displayed on the screen of the VR goggles 21.

[0018] The management terminal device 40 is an information processing device such as a personal computer that is used when an administrator of the image generation device 10 inputs information into the image generation device 10 or when checking information output from the image generation device 10. The management terminal device 40 displays the information output from the image generation device 10 on a screen 400 that it is provided with. As will be described later, the management terminal device 40 is used, for example, by a police training officer or the like when setting up a scenario for image content.

[0019] 1, the image generation device 10 includes an acquisition unit 11, a determination unit 12, a generation unit 13, and a storage unit 14. The acquisition unit 11, the determination unit 12, the generation unit 13, and the storage unit 14 are examples of an acquisition means, a determination means, a generation means, and a storage means, respectively.

[0020] The storage unit 14 is, for example, a storage device such as a RAM (Random Access Memory) or a hard disk 904, which will be described later with reference to Fig. 8. The storage unit 14 stores behavior information 141, scenario determination criteria 142, state change information 143, and video generation criteria 144. Details of the information stored in the storage unit 14 will be described later.

[0021] First, the operation of the video generation device 10 according to this embodiment, which generates a scenario for educational video content to be presented to the participant through input operations by an educator or the like, will be described.

[0022] The acquisition unit 11 displays on the screen 400 of the management terminal device 40 a scenario setting menu for generating scenarios for video content relating to the investigation of various incidents or accidents.

[0023] FIG. 2 is a diagram illustrating an example of a scenario setting menu screen that the acquisition unit 11 according to this embodiment displays on the screen 400 of the management terminal device 40. The acquisition unit 11 displays, for example, a scenario setting menu screen on the screen 400, which allows input of information for setting scene x (x is an identifier that can identify the scene), which is any scene in the video content to be presented to the viewer. Note that a scene according to this embodiment represents a scene that is a unit that constitutes the video content. The acquisition unit 11 displays, for example, a pull-down menu on the screen, which allows input of information about people, objects, and places that appear. However, it is assumed that the items shown in the pull-down menu have been provided in advance by, for example, an administrator (educator) of the video generation device 10.

[0024] The acquisition unit 11 acquires information representing the scenario settings entered by the trainer through an input operation on the management terminal device 40 using the pull-down menu. In the example shown in FIG. 2, the trainer has selected a suspicious person as a character, a car as a feature, and a public road as a location. Furthermore, the acquisition unit 11 may display icons representing the character, feature, and location selected by the trainer on the screen 400, as shown in FIG. 2, for example.

[0025] The acquisition unit 11 displays on the screen a pull-down menu into which information representing the actions of people and the movements of objects can be input for each person and object appearing in scene x in the video content set as described above. The acquisition unit 11 acquires information representing the movements in scene x that is input in the pull-down menu by an educator through an input operation on the management terminal device 40. In the example shown in Fig. 2, the educator has set a scene in which a car driven by a suspicious person stops on a public road as scene x of the video content.

[0026] The scenario setting menu screen illustrated in FIG. 2 is an example, and the acquisition unit 11 may display a scenario setting menu screen of a different form from that of FIG. 2 on the screen 400 of the management terminal device 40.

[0027] 2 for each scene included in the educational video content on the screen 400, and an input operation by the educator to set the scene is performed for each scene. The acquisition unit 11 generates or updates state change information 143, which will be described later, based on the information input by the input operation.

[0028] The acquisition unit 11 may also display on the screen 400 a menu screen for selecting an educational objective for the educational video content for which a scenario is to be created. The acquisition unit 11 may then narrow down the characters, character movements, object, object movements, and location selectable on the setting menu screen according to the objective input by the educator. In this case, the acquisition unit 11 prevents inappropriate options from being displayed in the pull-down menu, so as to prevent inappropriate scenes, such as "an incident that occurred inside a private home with a car driving" or "a person swimming in an area with no ocean." However, in this case, it is assumed that criteria for displaying appropriate options for the characters, character movements, object, object movements, and location according to the educational objective are stored in the storage unit 14.

[0029] Next, the operation of the video generation device 10 according to this embodiment to present a subjective video, which is educational video content, to the user will be described.

[0030] The acquisition unit 11 first displays a menu on the screen of the VR goggles 21, which allows the user to select video content depicting the investigation of a crime or accident that the user wishes to experience. The acquisition unit 11 acquires information input by the user via, for example, the input device 24 in response to the displayed menu, as identification information for scene 1, which is the first scene of the video content.

[0031] After the user selects scene 1 of the video content as described above, the generation unit 13 shown in Fig. 1 generates a video of scene 1 and displays the generated video on the screen of the VR goggles 21. That is, the generation unit 13 generates a video (animation) of a car driven by a suspicious person traveling on a public road and stopping. The method of generating a video by the generation unit 13 will be described later.

[0032] At a predetermined timing (for example, immediately before the completion of Scene 1) while the image of Scene 1 is being displayed on the screen of the VR goggles 21 by the generation unit 13, the acquisition unit 11 displays information on the screen of the VR goggles 21 that prompts the user to select an action (action 1) to be taken in the scene of Scene 1, superimposed on the image of Scene 1. The acquisition unit 11 may, for example, display options for Action 1 on the screen of the VR goggles 21.

[0033] The user selects at least one of the options displayed on the screen. The user may make this selection through an input operation using the input device 24, by speaking a word representing the option, or by moving their body in accordance with the option.

[0034] The acquisition unit 11 acquires information representing an input operation performed by the experiencer using the input device 24 as behavioral information 141 representing action 1. The acquisition unit 11 also collects voice uttered by the experiencer using the microphone 23, and acquires the result of voice recognition of the collected voice using existing voice recognition technology as behavioral information 141 representing action 1. The acquisition unit 11 also acquires video of the experiencer via the camera 22, and acquires the result of image recognition of the acquired video using existing image recognition technology as behavioral information 141 representing action 1. In this case, the behavioral information 141 also includes the movement of each body part of the experiencer, the direction of gaze, and the like.

[0035] The acquisition unit 11 stores the acquired behavior information 141 representing the action 1 in the storage unit 14.

[0036] The determining unit 12 determines a scene 2 that follows the scene 1 and occurs as a result of the action 1, based on the behavior information 141 that indicates the action 1 acquired by the acquiring unit 11 and the scenario determining criterion 142.

[0037] Fig. 3 is a diagram illustrating a tree-structured graph representing the scenario determination criteria 142 according to this embodiment. In the graph illustrated in Fig. 3, nodes represented by circles represent scenes in the video to be presented to the viewer, and edges represented by arrows connecting nodes represent actions that the viewer will take in each scene.

[0038] The scenario determination criteria 142 are generated or updated by the acquisition unit 11, for example, through input operations by an educator on the management terminal device 40. At this time, the acquisition unit 11 displays a graph representing the scenario determination criteria 142, such as that illustrated in FIG. 3, on the screen 400 of the management terminal device 40 so that the graph can be generated or updated. The acquisition unit 11 acquires information representing, for example, input operations for creating or deleting nodes, input operations for connecting nodes with edges, and input operations for assigning identifiers for identifying actions to each edge, as illustrated in FIG. 3. The identifiers for identifying actions are information that can identify the actions of people, the movements of objects, and locations in the state change information 143 illustrated in FIG. 4, which will be described later. The acquisition unit 11 generates or updates the scenario determination criteria 142 based on the information representing the input operations.

[0039] 3, scene 1 is set to one of a plurality of scenes 1, such as scene 1-1 or scene 1-2, depending on the video content selected by the viewer. Then, depending on action 1 selected by the viewer in scene 1, scene 2 following scene 1 will be one of a plurality of scenes 2, such as scene 2-1-1 or scene 2-1-2. Similarly, depending on action 2 selected by the viewer in scene 2, scene 3 following scene 2 will be one of a plurality of scenes 3, such as scene 3-1-1-1 or scene 3-1-1-2.

[0040] The scenario determination criteria 142 may be expressed in a form other than the tree-structured graph shown in Fig. 3. The scenario determination criteria 142 may be, for example, a table that indicates the relationship between scenes and actions.

[0041] The future changes in the state of objects included in the video to be presented to the viewer for each scene represented by the scenario determination criteria 142 (i.e., the actions of the characters, the movements of the characters, and the location of the scene) are managed by state change information 143.

[0042] 4 is a diagram illustrating data of the status change information 143 according to this embodiment. According to the status change information illustrated in FIG. 4, scene 1-1 represented by the scenario determination criterion 142 is a scene in which a car driven by a suspicious person stops while traveling on a public road. Similarly, scene 2-1-1 represented by the scenario determination criterion 142 is a scene in which a suspicious person discards an item they are carrying, and scene 3-1-1-1 is a scene in which the suspicious person behaves in an unsettled manner.

[0043] As explained with reference to Figures 3 and 4, the scenario determination criteria 142 represent the relationship between behavioral information 141, which is represented as the actions selected by the viewer in each scene, and state change information 143, which represents future changes in the state of objects included in the video (the actions of the characters, the movements of the characters, and the location of the scene).

[0044] Furthermore, the scenario determination criteria 142 may be a learning model that is the result of learning the relationship between the actions (behavioral information 141) taken by an individual in a certain situation and the changes in the situation (state change information 143) that result from those actions, for example, during the investigation of an actual incident or accident in the past.

[0045] 5 is a diagram showing an example of the determination unit 12 according to the present embodiment sequentially determining subsequent scenes according to an action selected by the user for a scene. Based on the initial settings of the scenario selected by the user, the determination unit 12 determines scene 1-1 as scene 1 from among multiple scenes including scene 1-1, scene 1-2, etc. in the scenario determination criteria 142 illustrated in FIG. 3. However, scene 1-1 is a scene in which a car driven by a suspicious person stops while traveling on a public road, as illustrated in FIGS. 4 and 5.

[0046] Next, the determination unit 12 determines scene 2-1-1 as scene 2 in response to the experiencer's selection of the action of instructing the suspicious person to get off (instructing the suspicious person to get off) as action 1 for the above-mentioned scene 1-1. That is, in this case, in the scenario determination criteria 142 illustrated in Fig. 3, the edge connecting scene 1-1 and scene 2-1-1 represents the action of instructing the suspicious person to get off. Note that scene 2-1-1 is a scene in which a suspicious person discards an item he is holding, as illustrated in Figs. 4 and 5.

[0047] Next, the determination unit 12 determines scene 3-1-1-1 as scene 3 in response to the experiencer's selection of an action of gazing at an object that has fallen to the ground as action 2 for the above-mentioned scene 2-1-1. That is, in this case, in the scenario determination criteria 142 illustrated in Fig. 3, the edge connecting scene 2-1-1 and scene 3-1-1-1 represents the action of gazing at an object that has fallen to the ground. Note that scene 3-1-1-1 is a scene in which a suspicious person behaves restlessly, as illustrated in Figs. 4 and 5.

[0048] Furthermore, in response to selecting the action of instructing a suspicious person to present a driver's license (instructing a suspicious person to present a driver's license) as action 3 for the above-mentioned scene 3-1-1-1, the determination unit 12 determines, in the scenario determination criteria 142, the subsequent scene connected to scene 3-1-1-1 by the edge representing the action as scene 4.

[0049] The generation unit 13 generates a subjective video of each scene determined by the determination unit 12 as described above, based on the state change information 143 and the video generation standard 144. In the example shown in Fig. 5, the generation unit 13 first generates a video of scene 1-1 in which a car driven by a suspicious person traveling on a public road stops.

[0050] Here, an example of a method for generating a subjective video by the generation unit 13 will be described. For convenience of explanation, in Fig. 4, the actions of a person, the movements of an object, and the location represented by the status change information 143 are shown in plain text, but the status change information 143 actually contains the person, the actions of the person, the object, the movements of the object, the location, etc. as codes so that information processing can be performed on them. The video generation standard 144 is information that serves as a standard that represents the relationship between these codes and the video related to these codes.

[0051] The generation unit 13 generates an image that will be the background of the scene by comparing the location code represented by the state change information 143 with the image generation standard 144. For example, when the location code represents an open road, the generation unit 13 generates an image (animation) of a typical open road indicated by the image generation standard 144 as the background of the scene.

[0052] The generation unit 13 generates a video representing the actions of a person or the movements of an object by comparing the codes of the person, the actions of the person, the object, and the movements of the object represented by the status change information 143 with the video generation standard 144. However, the status change information 143 has codes representing changes in status (movements) for each part of a person (e.g., hands, feet, head, torso, eyes, mouth, etc.) and each part of an object (e.g., if the object is a car, the entire car, doors, windows, lights, etc.) so that the actions of a person or the movements of an object can be identified. The generation unit 13 then compares the codes representing changes in status for each part of a person or object with the video generation standard 144, thereby generating a video (animation) of a scene representing the actions of a person or the movements of an object using existing video generation technology.

[0053] The generation unit 13 displays the subjective video of the scene generated as described above on the screen of the VR goggles 21.

[0054] Next, the operation (processing) of the image generation device 10 according to this embodiment will be described in detail with reference to the flowcharts of FIGS. 6A and 6B.

[0055] The acquisition unit 11 displays a video content selection menu on the screen of the VR goggles 21 worn by the user (step S101). The acquisition unit 11 accepts an input operation by the user to select video content from the displayed video content selection menu (step S102). The image generation device 10 sets a variable i (i is a natural number) to "1" (step S103).

[0056] The determination unit 12 determines scene i based on the scenario determination criteria 142 from the selection of video content (when i=1) or the experiencer's action i-1 (when i≧2) (step S104). The generation unit 13 generates a subjective video of scene i based on the state change information 143 and video generation criteria 144 related to the determined scene i, and displays the generated subjective video on the screen of the VR goggles 21 (step S105).

[0057] The image generation device 10 checks whether the scenario determination criteria 142 indicate that the scene i is the last scene (step S106). If the scene i is the last scene (Yes in step S107), the entire process ends. If the scene i is not the last scene (No in step S107), the acquisition unit 11 checks whether it is time to acquire the behavior information 141 representing the experiencer's action i (step S108).

[0058] If it is not the time to acquire the behavior information 141 (No in step S109), the process returns to step S108. If it is the time to acquire the behavior information 141 (Yes in step S109), the acquisition unit 11 displays information prompting the experiencer to select an action i to be taken in the scene i on the screen of the VR goggles 21 by superimposing it on the image of the scene i (step S110). The acquisition unit 11 acquires the behavior information 141 representing the experiencer's performance of action i (step S111). The image generation device 10 adds "1" to the variable i, and the process returns to step S104.

[0059] The image generation device 10 according to this embodiment enables a user to efficiently accumulate experience related to a job through a simulated experience of the job. This is because the image generation device 10 sequentially generates and presents subsequent subjective images to the user based on the user's actions in response to the virtual reality subjective images presented to the user for the simulated experience of the job, and on the scenario determination criteria 142 that indicate changes in the scenario resulting from the user's actions.

[0060] The effects achieved by the image generation device 10 according to this embodiment will be described in detail below.

[0061] People engaged in various occupations usually improve their occupational skills by accumulating work experience. Therefore, it is desirable to be able to efficiently accumulate work experience that will significantly improve their occupational skills. However, in reality, opportunities to experience such occupations are limited, making it difficult for individuals to accumulate the desired amount of work experience. Furthermore, it is particularly difficult to accumulate the desired amount of work experience in occupations that involve risk, such as police work. Therefore, in order to solve this problem, it is a challenge to enable people to experience occupations in a simulated manner, for example, by using technology related to providing virtual reality.

[0062] To address this issue, an image generation device 10 according to this embodiment includes an acquisition unit 11, a determination unit 12, and a generation unit 13, and operates as described above with reference to, for example, FIGS. 1 to 6B. Specifically, when a first subjective video of virtual reality for simulating a job is presented to a user, the acquisition unit 11 acquires behavioral information 141 representing the user's behavior in response to the first subjective video. The determination unit 12 determines a future change in the state of an object included in the first subjective video, which will occur due to the user's behavior, based on a scenario determination criterion 142 representing a relationship between the behavioral information 141 and status change information 143 representing the future change in the state of the object. The generation unit 13 then generates a second subjective video following the first subjective video based on the status change information 143 and image generation criterion 144, and presents the generated second subjective video to the user.

[0063] Then, the image generation device 10 uses scenario determination criteria 142 based on the actual work experiences of many people in the past to present the experiencer with a subjective virtual reality image that is equivalent to an actual job. In this way, the image generation device 10 enables the user to efficiently accumulate experience related to the job through a simulated experience of the job.

[0064] Furthermore, the image generation device 10 according to this embodiment may use a learning model that learns the association between the behavior information 141 and the state change information 143 as the scenario determination criterion 142. This allows the image generation device 10 to more accurately realize the user's efficient accumulation of experience related to the job through a simulated experience of the job.

[0065] Furthermore, the image generation device 10 according to this embodiment may have a function of simultaneously presenting subjective images to multiple participants. In this case, the acquisition unit 11 acquires behavioral information 141 relating to each of the multiple participants who are simultaneously presented with subjective images. The determination unit 12 generates state change information 143 from the behavioral information 141 relating to the multiple participants. The generation unit 13 presents, to each of the multiple participants, subjective images including other participants other than the user. By having such a function, the image generation device 10 can enable a user to efficiently accumulate experience relating to a job that requires cooperation between multiple people, for example.

[0066] Furthermore, the subjective video generated by the video generation device 10 according to this embodiment is not limited to a video in which the user is a police officer and experiences investigating a crime or accident. The subjective video generated by the video generation device 10 may be, for example, a video in which the user is a store clerk and experiences serving customers in a store, or a video in which the user is a teacher and experiences teaching students.

[0067] <Second embodiment> FIG. 7 is a block diagram showing the configuration of an image generation device 30 according to the second embodiment of the present invention.

[0068] The image generation device 30 according to this embodiment includes an acquisition unit 31, a determination unit 32, and a generation unit 33. The acquisition unit 31, the determination unit 32, and the generation unit 33 are examples of an acquisition means, a determination means, and a generation means, respectively.

[0069] When a first subjective video 331 of virtual reality for simulating a job is presented to the user, the acquisition unit 31 acquires behavioral information 311 representing the user's behavior in response to the first subjective video 331. The first subjective video 331 is, for example, an image similar to the subjective video generated by the image generation device 10 according to the first embodiment and displayed on the screen of the VR goggles 21. The behavioral information 311 is, for example, information similar to the behavioral information 141 according to the first embodiment. The acquisition unit 31 operates in the same manner as, for example, the acquisition unit 11 according to the first embodiment.

[0070] The determination unit 32 determines a future change in the state of an object included in the first subjective video 331, which will occur due to the action of the user, based on a scenario determination criterion 322 that represents the association between the action information 311 and state change information 321 that represents a future change in the state of the object. The state change information 321 is, for example, information similar to the state change information 143 according to the first embodiment. The scenario determination criterion 322 is, for example, a criterion similar to the scenario determination criterion 142 according to the first embodiment. The determination unit 32 operates in the same manner as, for example, the determination unit 12 according to the first embodiment.

[0071] The generation unit 33 generates a second subjective video 332 that follows the first subjective video 331 based on the state change information 321 and the video generation standard 333, and presents the generated second subjective video 332 to the user. The video generation standard 333 is, for example, a standard similar to the video generation standard 144 according to the first embodiment. Like the first subjective video 331, the second subjective video 332 is, for example, a subjective video similar to the subjective video generated by the video generation device 10 according to the first embodiment and displayed on the screen of the VR goggles 21. The generation unit 33 operates in the same manner as, for example, the generation unit 13 according to the first embodiment.

[0072] The image generation device 30 according to this embodiment allows the user to simulate a job, thereby enabling the user to efficiently accumulate experience related to the job. This is because the image generation device 30 sequentially generates and presents subsequent subjective images to the user based on the user's actions in response to the virtual reality subjective images presented to the user for the simulated experience of the job, and on the scenario determination criteria 322 that indicate changes in the scenario resulting from the user's actions.

[0073] <Hardware configuration example> In each of the above-described embodiments, each unit in the image generation device shown in Figures 1 and 7 can be realized by dedicated HW (Hardware) (electronic circuitry). In Figures 1 and 7, at least the following components can be considered as functional (processing) units (software modules) of a software program that includes instructions executed by a processor. Acquisition units 11 and 31, decision units 12 and 32, generation units 13 and 33, ·Memory control function in the memory unit 14.

[0074] However, the division of the various components shown in these drawings is for the sake of convenience, and various configurations may be envisioned for implementation. An example of the hardware environment in this case will be described with reference to FIG. 8.

[0075] Fig. 8 is a diagram illustrating an example of the configuration of an information processing device 900 (computer) that can realize the image generation device according to each embodiment of the present invention. That is, Fig. 8 shows the configuration of a computer (information processing device) that can realize the image generation device shown in Fig. 1 and Fig. 7, and represents a hardware environment that can realize each function in the above-mentioned embodiments.

[0076] The information processing device 900 shown in FIG. 8 includes the following components. ·CPU(Central_Processing_Unit)901, ·ROM(Read_Only_Memory)902, ·RAM(Random_Access_Memory)903, Hard disk (storage device) 904, a communication interface 905; Bus 906 (communication line), A reader / writer 908 capable of reading and writing data stored in a recording medium 907 such as a CD-ROM (Compact Disc Read Only Memory), · Input / output interface 909 such as a monitor, speaker, keyboard, etc.

[0077] That is, the information processing device 900 including the above components is a general computer in which these components are connected via a bus 906. The information processing device 900 may include multiple CPUs 901, or may include a CPU 901 configured with multiple cores.

[0078] The above-described embodiment may also provide a computer program capable of realizing the following functions for the information processing device 900 shown in FIG. 8. For example, the functions are the above-described configurations in the block diagrams (FIGS. 1 and 7) referred to in the description of the embodiment, or the functions of the flowcharts (FIGS. 6A and 6B). The functions of the image generation device according to this embodiment are then achieved by reading the computer program into the CPU 901 of the hardware, interpreting it, and executing it. The computer program provided within the device may be stored in a readable / writable volatile memory (RAM 903) or a non-volatile storage device such as the ROM 902 or hard disk 904.

[0079] In the above case, a computer program can be supplied to the hardware using a currently common procedure. Examples of such a procedure include installing the program in the device via a recording medium 907 such as a CD-ROM, or downloading the program from an external source via a communication line such as the Internet. In such a case, the computer program supplied to the information processing device according to this embodiment can be considered to be composed of the code that constitutes the program, or the recording medium 907 on which the code is stored.

[0080] The present invention has been described above using the above-described embodiments as exemplary examples. However, the present invention is not limited to the above-described embodiments. In other words, the present invention can be applied in various aspects that can be understood by a person skilled in the art within the scope of the present invention. [Explanation of symbols]

[0081] 10. Image generation device 11 Acquisition Department 12 Decision Section 13 Generation part 14 Storage section 141 Behavioral Information 142 Scenario Decision Criteria 143 Status Change Information 144 Image Generation Standards 21 VR goggles 22 Camera 23. Mike 24 Input Devices 30 Image generation device 31 Acquisition Department 311 Behavioral Information 32 Decision Section 321 Status Change Information 322 Scenario Decision Criteria 33 Generation part 331 First Subjective Video 332 Second Subjective View 333 Image Generation Standards 40 Management terminal 400 screens 900 Information Processing Equipment 901 CPU 902 ROM 903 RAM 904 Hard disk (storage device) 905 Communication Interface 906 Bus 907 Recording Media 908 Reader / Writer 909 Input / Output Interface

Claims

1. an acquisition means for acquiring, when a first subjective image of virtual reality for simulating a job is presented to the experiencer, behavioral information representing the experiencer's behavior in response to the first subjective image; a determining means for determining a future change in state of an object, which is included in the first subjective video and which is caused by the action of the user and which is selected from candidate objects narrowed down according to the input educational objective, based on the action information and a scenario determining criterion which represents a future change in state of the object and indicates a relationship with state change information according to the educational objective; a generation means for generating a second subjective video subsequent to the first subjective video based on the status change information and a video generation standard that indicates a relationship between the status change information and a video, and presenting the generated second subjective video to the user; An image generating device comprising:

2. the acquiring means acquires the behavioral information representing at least one of the voice, the line of sight, and the movement of each body part of the experiencer; The image generating device according to claim 1 .

3. the acquiring means displays options for the experiencer's actions in the first subjective video, and acquires the action information through an input operation by the experiencer to select at least one of the options.

3. The image generating device according to claim 1.

4. The determining means determines the behavior of the person who is the target and the movement of the object who is the target. The image generating device according to any one of claims 1 to 3.

5. the scenario determination criterion is a learning model that learns the association between the behavior information and the state change information; The image generating device according to any one of claims 1 to 4.

6. further comprising a storage means in which the scenario determination criteria and the image generation criteria are stored, The image generating device according to any one of claims 1 to 5.

7. the acquiring means acquires the behavioral information regarding each of the plurality of experiencers who are simultaneously presented with the first subjective video; the determining means generates the state change information from the behavior information regarding the plurality of experiencers; the generating means presents the second subjective video including the other experiencers to each of the plurality of experiencers; 7. An image generating device according to claim 1.

8. The first subjective video and the second subjective video are videos in which the experiencer experiences investigating an incident or an accident when the experiencer is a police officer, or videos in which the experiencer experiences serving customers in a store when the experiencer is a store clerk, or videos in which the experiencer experiences teaching students when the experiencer is a teacher.

8. An image generating device according to any one of claims 1 to 7.

9. By the information processing device, When a first subjective image of virtual reality for simulating a job is presented to a user, behavioral information representing a behavior of the user in response to the first subjective image is acquired; determining a future change in the state of an object selected from candidate objects included in the first subjective video and narrowed down according to the input educational objective, which change is caused by the action of the user, based on the behavior information and a scenario determination criterion that represents a relevance between the behavior information and state change information that represents the future change in the state of the object and corresponds to the educational objective; generating a second subjective video subsequent to the first subjective video based on the state change information and a video generation standard that indicates a relationship between the state change information and a video, and presenting the generated second subjective video to the user; Video generation method.

10. an acquisition process for acquiring behavioral information representing a behavior of a user in response to a first subjective video of virtual reality for simulating a job, when the user is presented with the first subjective video of virtual reality; a determination process for determining a future change in state of an object, which is included in the first subjective video and is selected from object candidates narrowed down according to the input educational objective, and which is caused by the behavior of the user, based on the behavior information and a scenario determination criterion that represents a relevance between the behavior information and state change information according to the educational objective and indicates the future change in state of the object; a generation process of generating a second subjective video subsequent to the first subjective video based on the status change information and a video generation standard that indicates a relationship between the status change information and a video, and presenting the generated second subjective video to the user; An image generation program for running the above on a computer.

Images