Virtual Space Generator

Abstract

To provide a virtual space generation device with unprecedented gameplay. [Solution] A virtual space generation device (10) comprising: a display data storage unit (112, 113) that stores display data for the virtual space and display data for the player character; a virtual space generation unit (22) that generates a virtual space using the display data for the virtual space; a player character operation processing unit (23) that operates the player character in the virtual space in response to external input operations using the display data for the player character; an image display processing unit (27) that generates an image in the virtual space based on image data including a target area input from the outside; a determination unit (28) that determines whether or not a predetermined operation by the player character has been performed on the target area; and a determination result output unit (29) that outputs the determination result from the determination unit.

Description

Technical Field

[0001] The present invention relates to a virtual space generation device.

Background Art

[0002] In recent years, technologies that allow users to perform events and games by operating player characters in a virtual space have been widely used. By using virtual reality (VR) technology that involves equipping the user with a terminal such as a head-mounted display (HMD) and displaying an image of a virtual space captured from the perspective of the player character on its display unit, it is possible to provide the user with a sense of immersion and allow them to enjoy events and games.

[0003] Patent Document 1 describes a system in which a screen object is placed on a stage provided in a virtual space and images of live events, sports, stage performances, etc. captured in the real space are displayed thereon. In this system, the user operates a player character that operates in the virtual space, and a video of the virtual space captured from the perspective of the player character is displayed on the head-mounted display worn by the user.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

[0006] In the system described in Patent Document 1, the user merely passively enjoys the images displayed on the screen object and does not cause any changes by moving the player character, thus lacking in gameplay.

[0007] The problem that this invention aims to solve is to provide a virtual space generation device that has unprecedented game-like features. [Means for solving the problem]

[0008] The virtual space generation device according to the present invention, which was developed to solve the above problems, A display data storage unit that stores display data for a virtual space and display data for a player character operating in that virtual space, A virtual space generation unit that generates a virtual space using the display data of the virtual space stored in the display data storage unit, A player character operation processing unit that uses the player character display data stored in the display data storage unit to operate the player character in the virtual space in response to external input operations, An image display processing unit that generates an image in the virtual space based on image data including a target region, A determination unit that determines whether or not a predetermined action performed on the image by the player character was performed on the target area, A determination result output unit that outputs the determination result from the determination unit. It is equipped with.

[0009] In the virtual space generation device according to the present invention, an image display processing unit generates an image including a target area within the virtual space. When a user performs an input operation to move a player character to this virtual space generation device using an external input device, the player character movement processing unit moves the player character in the virtual space in response to that input operation. The external input device is, for example, a head-mounted display (HMD), and its display unit displays an image of the virtual space as seen from the player character's perspective.

[0010] The image data generated in the virtual space is input from an external source. Information identifying the location of the target region in the image may or may not be included in the image data input from the external source (i.e., the target region included in the image may be undefined at the time the image display processing unit generates the image). In the latter case, the determination unit may include a classifier that uses a pre-trained model that has been trained on training data consisting of pairs of game images and target regions for various types of games, in order to identify the target region. Furthermore, this image data may be pre-prepared, or it may be generated by a large language model (LLM) or the like based on keywords entered by the user. In the latter case, the determination unit may also include a classifier that uses an LLM.

[0011] When a player character controlled by the user performs a predetermined action on an image generated in the virtual space, the determination unit determines whether or not the action was performed on a target area, and the determination result output unit outputs the determination result. For example, if the generated image relates to a shooting game, the predetermined action is a shooting action by the player character, and the determination unit determines whether or not the bullet fired by the player character's action hit a target area (enemy character, airplane, animal, etc.) included in the image. Alternatively, for example, if the image generated in the virtual space relates to a dance game, the predetermined action is a dance action by the player character, and the determination unit determines the degree of agreement between the player character's body movements and the target area (dancer's body) included in the image. This allows the user to enjoy a novel game in which they control a player character in accordance with images generated in the virtual space. [Effects of the Invention]

[0012] By using the virtual space generation device according to the present invention, it is possible to add game-like elements to a virtual space generation device that places screen objects in a virtual space and displays images there. [Brief explanation of the drawing]

[0013] [Figure 1] A diagram showing the main components of a game system, including one embodiment of the virtual space generation device according to the present invention. [Figure 2] This diagram illustrates the state in which a player character performs an action on an image displayed on a screen object in a shooting game using the virtual space generation device of this embodiment. [Figure 3] An example of generating a virtual space based on images of screen objects in the virtual space generation device of this embodiment. [Figure 4] This figure illustrates the judgment process using a judgment panel in a shooting game using the virtual space generation device of this embodiment. [Figure 5] A diagram for explaining a state in which an action is performed on an image displayed on a screen object by a player character in a dance game using the virtual space generation device of the present embodiment. [Figure 6] A diagram for explaining determination using a determination panel in a dance game using the virtual space generation device of the present embodiment.

Embodiments for Carrying Out the Invention

[0014] An embodiment of the virtual space generation device according to the present invention will be described below with reference to the drawings.

[0015] FIG. 1 is a main configuration diagram of a game system 1 including the virtual space generation device 10 of the present embodiment. The game system 1 includes the virtual space generation device 10 and the user terminal 40. Although only one user terminal 40 is shown in FIG. 1, there may be a plurality of user terminals 40. Further, the game system 1 and the user terminal 40 are interconnected via a wireless communication network such as the Internet or a wired communication network.

[0016] The virtual space generation device 10 includes a storage unit 11. The storage unit 11 has a user information storage unit 11, a virtual space display data storage unit 112, a character data storage unit 113, an effect data storage unit 114, an image generator storage unit 115, an image identifier storage unit 116, and an image data storage unit 117. Note that the concept of an image in the present specification includes not only a still image but also a moving image (for example, a moving image composed of still images at 60 frames per second).

[0017] The user information storage unit 111 stores information such as the user ID and password that identify the user using the user terminal 40, the type of player character operated by that user, and the terminal ID that identifies the user terminal 40. The virtual space display data storage unit 112 stores the display data for the virtual space. The character data storage unit 113 stores the display data for player characters and non-player characters that operate in the virtual space. The effect data storage unit 114 stores data for various effects (sound, images, etc.) that occur in the virtual space.

[0018] The image generator storage unit 115 stores an image generator that generates images based on words or sentences input from an external source. The image generator can suitably utilize generative AI such as Large Language Models (LLMs; e.g., Non-Patent Document 1). The image classifier storage unit 116 stores an image classifier that identifies target and non-target regions in images input from an external source or images generated by the image generator, and identifies the background in an image according to predetermined criteria. The image classifier can utilize, for example, a classifier based on object detection algorithms such as YOLO (You Only Look Once; e.g., Non-Patent Documents 2, 3) and SSD (Single Shot Multi-Box Detector; e.g., Non-Patent Document 3), or a trajectory estimation algorithm such as DeepSORT or StrongSORT (e.g., Non-Patent Document 4) that estimates the trajectory of an object identified by the classifier if it is a moving object. Furthermore, if the image generator is a generative AI such as an LLM, the image classifier can similarly utilize a generative AI such as an LLM (e.g., Non-Patent Document 5). In addition, algorithms can be used to identify moving objects in a video (objects whose position changes between temporally consecutive images) as target regions.

[0019] The image data storage unit 117 stores image data that can be used when allowing the user to play the game. The image data stored in the image data storage unit 117 may include both images to which information identifying the location of the target area contained in the image is associated, and images to which such information is not associated. The target area refers to the area that the user is playing the game on, for example, the display area of ​​the target object in a shooting game, or the display area of ​​the model dancer in a dance game. Furthermore, the image data stored in the image data storage unit 117 may include both images to which data stored in the virtual space display data storage unit 112 is associated, and images to which it is not.

[0020] The virtual space generation device 10 includes, as functional blocks, a user authentication unit 21, a virtual space generation unit 22, a player character motion processing unit 23, an image information input receiving unit 24, an image generation unit 25, an image identification unit 26, an image display processing unit 27, a determination unit 28, and an effect generation unit 29. Details of these functional blocks will be described later.

[0021] The virtual space generation device 10 is composed of, for example, a cloud server or a general personal computer, and each of the above-mentioned functional blocks is realized by executing pre-installed dedicated software (virtual space generation program) on the processor.

[0022] The user terminal 40 is equipped with a storage unit 41. The storage unit 41 stores terminal ID information for identifying the user terminal 40, as well as information about the user using the user terminal 40 (such as user ID information).

[0023] The user terminal 40 also includes, as hardware, a posture acquisition unit 42, an audio input unit 43, an audio output unit 44, an input unit 45, and a display unit 46. The user terminal 40 is, for example, a head-mounted display (HMD), and the posture acquisition unit 42 has various sensors and a calculation processing unit that determines the user's position and posture based on the output signals of the sensors. The posture acquisition unit 42, the audio input unit 43, and the audio output unit 44 transmit and receive signals to and from the virtual space generation device 10 at a predetermined frequency (for example, 60 times / second). As a result, the user's position and posture information acquired by the posture acquisition unit 42, and the audio information input to the audio input unit 43 are transmitted to the virtual space generation device 10 and reflected in the position, posture, and voice of the player character in the virtual space. In addition, audio information generated in the virtual space is output from the audio output unit 44. The display unit 46 is positioned in front of the user's eyes when the user wears the user terminal 40 on their head. Furthermore, the user terminal 40 includes a data processing unit 48 as a functional block.

[0024] Next, the operation of the game system 1 of this embodiment will be described.

[0025] When a user accesses the virtual space generation device 10 by performing a predetermined operation on the user terminal 40, the user authentication unit 21 generates a screen for entering a user ID and password and sends this data to the user terminal 40. On the user terminal 40, the data processing unit 48 processes the received data and displays the screen on the display unit 46. When the user enters a user ID and password, the user authentication unit 21 identifies the user based on the entered user ID and password. Here, the configuration is set up so that the user ID and password are entered, but the user ID and password stored in the storage unit 41 of the user terminal 40, or the terminal ID, may be automatically transmitted. In the following description as well, the display data of the screen generated by each part of the virtual space generation device 10 is sent to the user terminal 40, and the display data is processed by the data processing unit 48 and displayed on the display unit 46, but these series of processes will not be described repeatedly and will simply be described as "display the screen on the display unit 46," etc.

[0026] When the user authentication unit 21 identifies the user, the virtual space generation unit 22 reads the virtual space display data predetermined as the initial state from the virtual space display data storage unit 112 and generates the virtual space. The player character operation processing unit 23 identifies the type of player character that the user will operate based on the data stored in the user information storage unit 111. If multiple types of player characters are associated with the user, the user is prompted to select one of them. Then, the player character is made to appear in the virtual space based on the data stored in the character data storage unit 113. In addition to this, a pre-configured non-player character may also be made to appear.

[0027] Here, the user logs in by entering a user ID and password, but the login process may be omitted. Also, although the user information storage unit 111 is used to pre-store information on the type of player character the user will be controlling, the user may be presented with information on multiple types of player characters stored in the character data storage unit 113 without associating them with user information, and the user may be allowed to select the player character to control each time. Alternatively, the user terminal 40's storage unit 41 may store information on the type of player character, and the player character operation processing unit 23 may read that information.

[0028] Next, the image information input receiving unit 24 displays a screen on the display unit 46 that allows the user to select whether to use an image already stored in the image data storage unit 117 or a newly generated image as the image to be used in the game.

[0029] If the user chooses to use an existing image, the image information input receiving unit 24 displays a list of images (game footage) stored in the image data storage unit 117 on the display unit 46 and prompts the user to select one of them.

[0030] When a user selects an image, the image recognition unit 26 checks whether information identifying the location of a target region contained in the image is associated with the data of the selected image. If information identifying the location of the target region is associated, it is read along with the image. On the other hand, if information identifying the location of the target region is not associated, the image recognition unit reads and operates the image recognition unit stored in the image recognition unit storage unit 116.

[0031] Furthermore, the image recognition unit 26 checks whether the data of the selected image is associated with the display data of the virtual space stored in the virtual space display data storage unit 112. If it is associated with the data stored in the virtual space display data storage unit 112, the virtual space generation unit 22 reads that data. If it is not associated with the data stored in the virtual space display data storage unit 112, the image recognizer stored in the image recognizer storage unit 116 is read and operated. Note that the display data of the virtual space associated with the image data is the display data of the virtual space that is displayed in synchronization with the image (game video) displayed on the screen object 5 described later.

[0032] If the user chooses to use a newly generated image on the selection screen described above, the image generation unit 25 reads the image generator from the image generator storage unit 115 and asks the user what kind of image (game footage) to display. In response, the user inputs a word (e.g., shooting game, forest, hunting) or a sentence (e.g., a shooting game where you hunt in a forest) via voice or text from the user terminal 40, and the image generator generates image data based on the input voice or text.

[0033] For example, if the image created here is footage from a dance game, when a user plays the dance game, they can view the exemplary movements and then generate a dance game that allows them to imitate those movements. In this case, the user should input information from the user terminal 40, such as, "A dance game consisting of two phases: an exemplary phase (a phase in which exemplary movements are displayed for the user to review) and a play phase (a phase in which the user imitates the exemplary movements they have reviewed. In this phase, the exemplary movements may or may not be displayed)." In addition, in a dance game, if the physique of the model performing the exemplary movements does not match that of the player character, the result of the movement matching judgment may be poor. Therefore, it is advisable to input information about the gender and physique of the player character along with the above information. By inputting this information, the image generation unit 25 can generate an image of a dance game in which a model with a physique similar to that of the player character performs the exemplary movements. The information input from the user terminal 40 is stored in the storage unit 11 (for example, the image data storage unit 117).

[0034] Once the selection or generation of images to be used in the game is complete, the image display processing unit 27 virtually displays a transparent screen object 5 at a predetermined position in the virtual space 8 (for example, a position directly facing the player character 9 at a predetermined distance L; see Figure 2), and displays an image (starts video playback) based on the selected or generated data. Because the screen object 5 is transparent, when the player character 9 views the screen object 5 from their perspective, they can simultaneously see the virtual space located behind the screen object 5. As described above, the screen object 5 is virtually displayed in the virtual space, and for example, player characters and non-player characters operated by other users can freely pass through the screen object 5. Furthermore, if the image is associated with data stored in the virtual space display data storage unit 112, the virtual space generation unit 22 generates a virtual space based on the virtual space display data associated with the image, in parallel with the display of the image (video playback; the same applies hereinafter).

[0035] If the selected or generated image is not associated with the virtual space display data stored in the virtual space display data storage unit 112, the image identification unit 26 identifies the elements contained in the image using an image classifier in parallel with the display of the image, and generates a virtual space 8 that extends the image displayed on the screen object 5. For example, as shown in Figure 3, if the image has a sky 6 and ground 7 as a background, with the sky 6 containing the sun 61 and clouds 62, and the ground 7 containing houses 71 and roads 72, then a virtual space 8 is generated in the virtual space with the sky 60 above the boundary between the sky 6 and ground 7, and the ground 70 below the boundary. In addition, clouds 621 are placed in the sky 60, houses 711 are placed on the ground 70, and roads 721 extended from the screen object 5 are displayed, similar to those contained in the image.

[0036] Furthermore, if the data of the selected or generated image does not contain information to identify the location of the target region, the image recognition unit 26 identifies the target region 52 included in the image using an image classifier in parallel with the display of the image (see Figure 2).

[0037] When displaying video on screen object 5, the process of identifying the target area 52 may be performed at the same frequency as the video's frame rate (e.g., 60 frames / second) (e.g., 60 times / second). However, depending on the performance of the devices constituting the virtual space generation device 10 and the type of image classifier, such high-speed processing may be difficult.

[0038] Therefore, as an image classifier, it is advisable to use object detection algorithms such as YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector) in combination with trajectory estimation algorithms such as DeepSORT and StrongSORT, which estimate and track the trajectories of objects detected by these object detection algorithms. By adopting this configuration, it becomes possible to perform object detection at a frequency lower than the video frame rate (for example, once per second), and to identify the target region 52 using the position of the object detected every second and the estimated trajectory of that object, making it possible to configure the virtual space generator 10 on a general-purpose personal computer.

[0039] To avoid VR sickness caused by images displayed in a virtual space, a minimum frame rate of 60 frames per second is required. By using these technologies, it is possible to realize games using high frame rate images. In addition, there will be a wider variety of algorithms that can be selected as image classifiers. Alternatively, if all moving objects in the image are targets, a simpler algorithm can be used that detects the moving objects and identifies them as target regions 52.

[0040] When an image is displayed (video is played) on screen object 5, and the user operates player character 9 through user terminal 40, the player character motion processing unit 23 moves player character 9 in virtual space 8 according to the input operation. For example, player character 9 moves in the virtual space in accordance with the user's movement, and when the user changes their posture, player character 9 also changes to the same posture.

[0041] When the player character 9 (or user) performs a first predetermined action, the determination unit 28 is activated. Then, when the player character 9 (or user) performs a second predetermined action, the determination unit 28 recognizes that the action was performed on the image displayed on the screen object 5. Since the specific content of the predetermined action differs depending on the type and content of the game, the determination unit 28 determines the content of the first and second predetermined actions according to the content of the target area 52. Alternatively, if only a specific type of game is to be played, the content of the first and second predetermined actions may be predetermined.

[0042] In the case of a shooting game, the first predetermined action is, for example, the action of the player character 9 aiming a gun, and the second predetermined action is, for example, the action of operating the trigger (the action of firing a bullet from the gun). Alternatively, in the case of a dance game, the first predetermined action is the action of the player character 9 facing the screen object 5, and the second predetermined action is any action that changes the posture from the state of facing the screen object 5. In the case of a dance game consisting of two phases, a demonstration phase and a play phase, the determination unit 28 detects the first predetermined action and the second predetermined action only during the play phase. Alternatively, the user may notify the virtual space generator 10 of the timing when the player character 9 will perform a predetermined action on the above image (the timing when the action will start) by performing a predetermined input operation on the user terminal 40, and the determination unit 28 may then operate for a predetermined time (for example, 10 seconds).

[0043] When the determination unit 28 recognizes that the action of the player character 9 was performed on an image displayed on the screen object 5, it determines whether or not the action was performed on the target area 52. For example, in a shooting game, it determines whether or not the bullet fired from the gun held by the player character 9 passes through the target area 52. In games such as shooting games, where some object is emitted from the player character 9 towards the screen object 5, the effect generation unit 29 emits a ray of light from the muzzle of the gun held by the player character 9 in the direction the barrel extends, indicating the trajectory 91 of the bullet.

[0044] At this time, the determination unit 28 virtually places a transparent determination panel 51 on the back of the screen object 5 (the side opposite to the player character 9), and determines whether the bullet hit the target area 52 based on whether the incident position 512 of the light ray that passed through the screen object 5 on the determination panel 51 overlaps with the target area 511 on the determination panel 51 that corresponds to the target area 52 on the screen object 5 (see Figure 4). This is one example of hit determination by the determination unit 28, and the method of hit determination can be changed as appropriate.

[0045] Furthermore, for example, in the case of a dance game, as shown in Figure 5, the judgment unit 28 can be configured to virtually place a light source 54 behind the player character 9 (opposite side from the screen object 5), and virtually place a transparent judgment panel 51 behind the screen object 5, and score the dance performed by the player character 9 based on the degree of overlap between the silhouette area 513 of the player character 9 projected onto the judgment panel 51 and the target area 511 on the judgment panel 51 that corresponds to the target area 52 on the screen object 5, by irradiating the player character 9 with light from the light source 54 (see Figure 6). Using a light source 54 is not mandatory; any configuration can be adopted as long as it can project the silhouette area 513 of the player character 9 onto the judgment panel 51. For example, a camera can be virtually placed to capture the player character 9 from the side of the screen object 5, and the silhouette captured by the camera can be projected onto the judgment panel 51. As described above, in a dance game, if the physique of the model performing the exemplary movements does not match that of the player character, the judgment result of the degree of matching of movements may be poor. Therefore, if the image data for the dance game is not generated by inputting information about the gender and physique of the player character, it is advisable to perform processing such as matching the physiques (height and body width) of both when displaying the silhouette of player character 9 on the judgment panel 51.

[0046] Here, the dance performed by the player character 9 is scored based on the degree of overlap between the target area 511 on the judgment panel 51 corresponding to the target area 52 on the screen object 5. However, various other methods can be employed. For example, the posture acquisition unit 42 of the user terminal 42 could be one capable of extracting the user's bone data from the detected user posture, and the player character motion processing unit 23 could be configured to move the player character 9 based on the user's bone data. Furthermore, the judgment unit 28 could extract the bone data of a model performing exemplary movements from the target area 52 and score the dance based on the degree of agreement with the user's bone data. However, if the judgment unit 28 extracts the model's bone data from the target area 52 in real time during gameplay and determines the degree of agreement with the user's bone data, the processing load on the judgment unit 28 may become excessive. Therefore, when the judgment unit 28 makes a judgment based on the degree of agreement between the user's and the model's bone data, it is advisable to extract the model's bone data from the image data in advance, after the image data has been selected or generated and before gameplay using that image data begins. Alternatively, in the case of a dance game consisting of two phases, a demonstration phase and a play phase, the bone data of the model may be extracted while the image data of the demonstration phase is being played back.

[0047] Alternatively, the posture acquisition unit 42 may extract information on changes in the position of a controller held in the user's hand or a specific object attached to the user's foot, and the player character motion processing unit 23 may be configured to reflect this position information in the movements (changes in the position of the player character 9's limbs), and further, the judgment unit 28 may extract changes in the position of the model's limbs from the target area 52 and score the dance based on the degree of agreement with the user's changes in the position of their limbs (that is, the dance may be scored based on the degree of agreement between the position of a specific part of the user and the change in the position of that specific part of the model included in the target area 52).

[0048] The judgment result from the judgment unit 28 is displayed as an image (or video) on the screen object 5 by the effect generation unit 29. For example, in the case of a shooting game, as shown in Figure 2, the effect generation unit 29 can be configured to display an effect 53 representing an explosion or blast centered on the point of impact within the target area, or to display an effect of blood splattering from the point of impact. Alternatively, a sound effect indicating that a bullet has hit the target area 52 may be generated as an effect. Or, the hit can be indicated by displaying the word "HIT!" near the point of impact within the target area 52. In the case of a dance game, as shown in Figure 5, the effect generation unit 29 can be configured to display the judgment result from the judgment unit 28 as a score on the screen object 5, or to display words such as "Excellent!", "Good!", or "Bad" depending on the judgment result from the judgment unit 28 at each point in time. The data for these effects is stored in advance in the effect data storage unit 114 according to the expected type of game.

[0049] If the images (game footage) used in the above game were newly generated by the image generator, after the game ends, the data of those images (game footage) is stored in the image data storage unit 117 along with the information of the target region 52 identified by the image classifier from the game footage. In addition, the display data of the virtual space 8 generated by the image classifier is stored in the virtual space display data storage unit 112 in association with the game footage data. Furthermore, even if the game footage used in the game was previously stored in the image data storage unit 117, after the game ends, the information of the target region 52 identified by the image classifier is stored in the image data storage unit 117, and the display data of the virtual space 8 generated by the image classifier is stored in the virtual space display data storage unit 112.

[0050] In the game system 1 of this embodiment, images input from an external source to the virtual space generation device 10 are displayed on the screen object 5, and the game can be enjoyed using these images. While the use of screen objects has been proposed before, it was limited to passively enjoying the images displayed therein. In contrast, by using the virtual space generation device 10 and the game system 1 including the virtual space generation device 10 of this embodiment, it is possible to enjoy a novel game using images displayed on the screen object.

[0051] Furthermore, in the above embodiment, users can input words or sentences via voice or text to generate images (game footage) that suit their preferences and enjoy them.

[0052] The above embodiments are merely examples and can be modified as appropriate in accordance with the spirit of the present invention.

[0053] Although described as a game system 1 in the above embodiment, the virtual space generation device according to the present invention can be used for purposes other than games. For example, a system with the same configuration as in the above embodiment can be suitably used for training such as yoga or exercise. Furthermore, although the game system 1 was configured with the virtual space generation device 10 and the user terminal 40 in the above embodiment, some or all of the storage unit and functional blocks of the virtual space generation device 10 can be replaced with the storage unit of the user terminal 40. It can also be used as a functional block.

[0054] In the above embodiment, an image including the target region was displayed on a screen object, but other methods can also be used. For example, a hologram of the image corresponding to the target region may be generated in virtual space without using a screen object. Also, in the above embodiment, a rectangular screen object 5 was placed and an image including the target region and non-target region was displayed, but the target region alone may be displayed on a screen object having the same outline as the target region. Since the screen object itself is placed virtually, its shape can be changed instantly and arbitrarily. In the above embodiment, an example in which the determination unit 28 uses a determination panel 51 was described, but determination may be performed without using a determination panel 51.

[0055] In the above embodiment, an example was described in which an image generator creates game footage and an image classifier detects the target region included in the game footage. However, it is also possible for an image former to generate an initial image containing the target region, and for the image generator to generate game footage by assigning predetermined movements to that target region. For example, in the case of an animal, a trajectory of charging towards the player character 9 can be predetermined, and in the case of an airplane, a trajectory of turning while heading towards the player character 9 can be predetermined. In this case, the image classifier can easily track the target region included in the game footage using the movement data provided by the image generator. Furthermore, it is also possible to configure the system to launch predetermined attacks from the target region towards the player character according to the characteristics of the target region (airplane, animal, etc.).

[0056] In the above embodiment, an example of one user enjoying the game was described, but multiple users can also use their respective user terminals 40 to enjoy the game simultaneously in a single virtual space.

[0057] [Pattern] It will be obvious to those skilled in the art that the exemplary embodiments described above are specific examples of the following embodiments.

[0058] (Section 1) A virtual space generation device according to one aspect of the present invention is: A display data storage unit that stores display data for a virtual space and display data for a player character operating in that virtual space, A virtual space generation unit that generates a virtual space using the display data of the virtual space stored in the display data storage unit, A player character operation processing unit that uses the player character display data stored in the display data storage unit to operate the player character in the virtual space in response to external input operations, An image display processing unit that generates an image in the virtual space based on image data including a target region, A determination unit that determines whether or not a predetermined action performed on the image by the player character was performed on the target area, A determination result output unit that outputs the determination result from the determination unit. It is equipped with.

[0059] In the virtual space generation device described in paragraph 1, the image display processing unit generates an image including the target area within the virtual space. When a user performs an input operation to move a player character to this virtual space generation device using an external input device, the player character movement processing unit moves the player character in the virtual space in accordance with that input operation. The external input device is, for example, a head-mounted display (HMD), and its display unit displays an image of the virtual space as seen from the player character's perspective.

[0060] The image data generated in the virtual space is input from an external source. Information identifying the location of the target region in the image may or may not be included in the image data input from the external source (i.e., the target region included in the image may be undefined at the time the image display processing unit generates the image). In the latter case, the determination unit may include a classifier that uses a pre-trained model that has been machine-learned to identify the target region included in the game images of various types of games. Furthermore, this image data may be pre-prepared, or it may be generated by a large language model (LLM) or the like based on keywords entered by the user. In the latter case, the determination unit may also include a classifier that uses an LLM.

[0061] When a player character controlled by the user performs a predetermined action on an image generated in the virtual space, the determination unit determines whether the action was performed on a target area, and the determination result output unit outputs the determination result. For example, if the generated image relates to a shooting game, the predetermined action is a shooting action by the player character, and the determination unit determines whether the bullet fired by the player character's action hit a target area (enemy character, airplane, animal, etc.) included in the image. Alternatively, for example, if the image generated in the virtual space relates to a dance game, the predetermined action is a dance action by the player character, and the determination unit determines the degree of agreement between the player character's body movements and the target area (dancer's body) included in the image. This allows the user to enjoy a novel game in which they control a player character on an image generated in the virtual space.

[0062] (Section 2) The virtual space generation device relating to paragraph 2 is, in the virtual space generation device relating to paragraph 1, The image display processing unit places a screen object in the virtual space and displays the image on the screen object. It is.

[0063] (Section 3) The virtual space generation device relating to paragraph 3 is, in the virtual space generation device relating to paragraph 2, The aforementioned screen object is a transparent object, The determination unit places a determination panel behind the screen object and performs the determination based on the range of actions performed by the player character on the image on the determination panel and the overlap between the target area. It is.

[0064] In the virtual space generation device described in paragraph 1, one method of generating images in the virtual space is to display the images on screen objects placed in the virtual space, as described in paragraph 2. Furthermore, in the virtual space generation device described in paragraph 2, as described in paragraph 3, it is possible to determine whether or not the player character's actions were performed on a target area using a judgment panel placed behind the screen object.

[0065] (Section 4) The virtual space generation device relating to paragraph 4 is a virtual space generation device relating to any of paragraphs 1 to 3, further comprising: An image generator storage unit that stores an image generator that generates images based on words or sentences, An image information input receiving unit that accepts input of words or sentences related to the aforementioned image. Equipped with, The image display processing unit generates image data by inputting the words or sentences entered into the image information input receiving unit into the image generator. It is.

[0066] In the virtual space generation device described in paragraph 4, users can generate images of their choice according to their own preferences and enjoy games using those images.

[0067] (Section 5) The virtual space generation device relating to paragraph 5 is, in the virtual space generation device relating to paragraph 4, The aforementioned image generator includes a large-scale language model.

[0068] As the image generator in the virtual space generation device according to paragraph 4, the large language models (LLMs) described in paragraph 5 can be suitably used.

[0069] (Section 6) The virtual space generation device relating to paragraph 6 is a virtual space generation device relating to any of paragraphs 1 to 5, further comprising: Image classifier storage unit that stores an image classifier that identifies objects in an image, which is composed of a trained model created by machine learning using training data consisting of a game image and a target region contained in the game image. Equipped with, The determination unit identifies the target region included in the image using the image classifier. It is.

[0070] The virtual space generation device described in paragraph 6 allows users to enjoy games using any image, even if the target area is not specified in advance.

[0071] (Section 7) The virtual space generation device relating to paragraph 7 is, in the virtual space generation device relating to paragraph 6, The image classifier includes at least one of an object detection algorithm, an algorithm for estimating the trajectory of a moving object detected by the object detection algorithm, and a large-scale language model.

[0072] As an image classifier in the virtual space generation device according to paragraph 6, the object detection algorithm described in paragraph 7 (e.g., YOLO (You Only Look Once), SSD (Single Shot Multibox Detector)), an algorithm for estimating the trajectory of a moving object detected by the object detection algorithm (e.g., DeepSORT, StrongSORT), and / or large language models (LLM) can be suitably used. [Explanation of Symbols]

[0073] 1…Game System 10…Virtual Space Generator 11...Storage section 111...User information storage unit 112... Virtual space display data storage unit 113... Character data storage unit 114... Effect data storage unit 115...Image generator memory unit 116...Image discriminator memory unit 117...Image data storage unit 21...User Authentication Section 22…Virtual Space Generation Unit 23...Player character motion processing unit 24…Image Information Input Reception Department 25…Image generation unit 26…Image Recognition Unit 27…Image display processing unit 28…Judgment section 29…Effect generation unit 40...User terminal 41...Storage section 42…Attitude acquisition part 43…Voice input section 44…Audio output section 45...Input section 46…Display section 48...Data Processing Unit 5…Screen objects 51... Judgment Panel 511...Target area on the judgment panel 512…Incidence position 513...Player character silhouette area 52…Target area 53…Effects 54...Light source 6, 60...empty 61…sun 62, 621...cloud 7, 70...ground 71, 711…house 72, 721…road 8…Virtual space 9…Player character 91…Trajectory

Claims

1. A display data storage unit that stores display data for a virtual space and display data for a player character operating in that virtual space, A virtual space generation unit that generates a virtual space using the display data of the virtual space stored in the display data storage unit, A player character operation processing unit that uses the player character display data stored in the display data storage unit to operate the player character in the virtual space in response to external input operations, An image display processing unit that generates an image in the virtual space based on image data including a target region, A determination unit that determines whether or not a predetermined action performed on the image by the player character was performed on the target area, A determination result output unit that outputs the determination result from the determination unit. A virtual space generation device characterized by comprising the following features.

2. The image display processing unit places a screen object in the virtual space and displays the image on the screen object. A virtual space generation device according to claim 1, characterized in that...

3. The aforementioned screen object is a transparent object, The determination unit places a determination panel behind the screen object and performs the determination based on the range of actions performed by the player character on the image on the determination panel and the overlap between the target area. The virtual space generation device according to claim 2, characterized in that

4. moreover, An image generator storage unit that stores an image generator that generates images based on words or sentences, An image information input receiving unit that accepts input of words or sentences related to the aforementioned image. Equipped with, The image display processing unit generates image data by inputting the words or sentences entered into the image information input receiving unit into the image generator. A virtual space generation device according to claim 1, characterized in that...

5. The aforementioned image generator includes a large-scale language model The virtual space generation device according to claim 4, characterized in that

6. moreover, Image classifier storage unit that stores an image classifier that identifies objects in an image, which is composed of a trained model created by machine learning using training data consisting of a game image and a target region contained in the game image. Equipped with, The determination unit identifies the target region included in the image using the image classifier. A virtual space generation device according to claim 1, characterized in that...

7. The image classifier includes at least one of an object detection algorithm, an algorithm for estimating the trajectory of a moving object detected by the object detection algorithm, and a large-scale language model. The virtual space generation device according to claim 6, characterized in that

Images