Program for generating game scene on basis of emotional information stored in recording medium

Abstract

The present invention relates to a program for generating a game scene on the basis of emotional information stored in a recording medium and, more specifically, to a program for generating a game scene on the basis of emotional information stored in a recording medium, capable of generating an image corresponding to emotional information on the basis of artificial intelligence and automatically adjusting the color, lighting position, brightness, and contrast of a game scene on the basis of the generated image, so as to induce an emotion corresponding to a game scenario from a user playing a game or amplify an emotion currently felt by the user.

Description

Game scene generation program based on emotional information stored in recording media

[0001] The present invention relates to a game scene generation program based on emotional information stored in a recording medium, and more particularly, to a game scene generation program based on emotional information stored in a recording medium, which generates an image matching emotional information based on artificial intelligence and automatically adjusts the color, lighting position, brightness, contrast, etc. of the game scene based on the generated image, thereby inducing an emotion corresponding to a game scenario from a user playing the game or amplifying the emotion currently felt by the user.

[0002] Games used to have a somewhat negative image as elements that only provided entertainment and pleasure, but recently, they have been recognized as entertainment that can be applied to various fields such as education and treatment by utilizing positive elements that provide functions to enhance people's interest, curiosity, and immersion, and are not limited to entertainment elements, but are being recognized as entertainment that can be applied to various fields.

[0003] Research and development are being conducted to improve the gaming environment for users who play games, so that games can provide positive functions such as increased cultural diversity, positive self-development, active use of leisure time, stress relief, and positive socialization.

[0004] In the past, research and development was conducted in the direction of improving the graphics, sound effects, and intelligence of the game environment to further satisfy users playing the game. However, this was only research and development to provide high-quality games with technological advancements, and there were limitations in inducing emotional empathy, immersion, and interest in the game for users playing the game.

[0005] To improve this, we tried to increase immersion and interest in the game by inducing the emotions of the users playing the game by inducing them to play along the game storyline provided by the developer, creating and providing interactive elements that respond to the users' choices or reactions, or adjusting the color elements of the game scene to induce or amplify the emotions of the users playing the game.

[0006] In particular, since color is a factor that greatly influences human emotions and shows characteristics based on human tastes and living environments, adjusting the attribute elements such as color, brightness, and contrast of a game scene can easily induce or amplify human emotions and enhance immersion and interest in the game.

[0007] However, although there are many technologies for adjusting the attributes of current game scenes, such as color, brightness, and contrast, there is a problem in that it is difficult to create a new game scene that reflects emotional information because it is ambiguous to reflect abstract emotional information in a game scene that includes specific attribute elements.

[0008] Therefore, there is a need for a technology that can create a game scene that reflects emotional information by defining abstract emotional information as specific attribute elements and then correcting the attribute elements based on them in order to enhance immersion and interest in the game by inducing or amplifying the emotions of the person playing the game.

[0009] The present invention has been devised to solve the above-described problem, and the purpose of the present invention is to provide a game scene generation program based on emotional information stored in a recording medium, which can induce emotions corresponding to a game scenario from a user playing a game or amplify the emotions currently felt by the user by creating a game environment in which emotional information is reflected.

[0010] In order to achieve the above-described purpose, a game scene generation program based on emotional information stored in a recording medium is provided, characterized in that the computer functions as a reference image generation means for receiving natural language input about emotional information from a user, inputting a two-dimensional sample image and the natural language about the input emotional information into an image generation model to generate a reference image reflecting the emotional information; and a game scene adjustment means for automatically adjusting attribute elements of an initial game scene based on the reference image generated from the reference image generation means.

[0011] In a preferred embodiment, the method further includes an adjustment attribute confirmation means capable of confirming a change in the adjusted game scene in the order of attribute elements having a large influence on the adjusted game scene through the game scene adjustment means.

[0012] In a preferred embodiment, the adjustment attribute confirmation means includes: an influence score calculation means for calculating an influence (hereinafter, “influence score”) on a generated game scene (Generated Template) for each attribute element that adjusted the initial game scene; and an adjustment attribute visualization means for visualizing a change in a game scene applied in the order of attribute elements with the greatest influence on the generated game scene (Generated Template) based on the influence score calculated by the influence score calculation means.

[0013] In a preferred embodiment, the image generation model is a learning model constructed by training the WEBEmo image dataset collected based on Parrott's three-layer emotion model.

[0014] In a preferred embodiment, the image generation model is one of a diffusion model, a generative adversarial network (GAN), and a variational auto-encoder (VAE).

[0015] In a preferred embodiment, each image having a three-layer structure of the image dataset generates captions including emotional keywords for each image, the captions including: a first caption including emotional information; a second caption being either positive or negative; and a third caption including detailed emotional information; and the learning model is fine-tuned by training an image dataset consisting of pairs of each image and the captions.

[0016] In a preferred embodiment, the image dataset generates captions for each image using the CLIP model.

[0017] In a preferred embodiment, the emotional information is information about emotions including love, fun, surprise, sadness, anger, and fear, and in addition, can be expanded to include detailed emotional information including liveliness, confusion, frustration, jealousy, gratitude, disregard, pride, comfort, shame, and empathy.

[0018] In a preferred embodiment, the attribute elements are color, light and bloom.

[0019] In a preferred embodiment, the game scene adjustment means includes: an attribute error calculation means for calculating an error for each attribute element between the reference image and the initial game scene; and an attribute adjustment means for performing an adjustment to minimize an error value calculated from the attribute error calculation means so that the attribute elements of the initial game scene are close to the attribute elements of the reference image.

[0020] In a preferred embodiment, the attribute error calculation means uses a mean square error (MSE) function.

[0021] In a preferred embodiment, the attribute error calculation means uses a random search algorithm to derive an optimal attribute element error value.

[0022] In a preferred embodiment, the influence score calculation means calculates the influence score by calculating the average value of the mean square error (MSE) change calculated during N iterative adjustments in the game scene adjustment means.

[0023] In a preferred embodiment, the adjustment attribute confirmation means outputs game scenes adjusted in the order of influential attribute elements in the adjustment attribute visualization means, and can manually correct the correction values ​​of attribute elements of each adjusted game scene by receiving input from a game designer.

[0024] The present invention has the following excellent effects.

[0025] According to the game scene generation program based on emotion information stored in the recording medium of the present invention, by generating a game scene reflecting abstract information about emotions, it is possible to induce emotions corresponding to the game scenario from a user playing the game or to amplify the emotions currently felt by the user, thereby improving the immersion and interest in the game.

[0026] In addition, according to the game scene generation program based on emotional information stored in the recording medium of the present invention, a reference image for emotional information is generated, an error between the reference image and the game scene is calculated, and then, based on the calculated value, attribute elements including lighting, color, contrast, and brightness of the game scene are adjusted, thereby resolving difficulties caused by ambiguity in adjusting attribute elements including lighting, color, contrast, and brightness due to abstract information, and at the same time, minimizing the workload of a game scene designer and improving work speed.

[0027] FIG. 1 is a drawing for explaining the configuration of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention;

[0028] FIG. 2 is a drawing for explaining an operation method of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention, as an example.

[0029] FIG. 3 is a drawing for explaining a means for confirming adjustment properties of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention.

[0030] The terms used in the present invention are selected from the most widely used general terms as much as possible, but in certain cases, there are terms arbitrarily selected by the applicant. In such cases, the meaning of the terms should be understood by considering the meaning described or used in the detailed description of the invention, rather than the simple name of the term.

[0031] Hereinafter, the technical configuration of the present invention will be described in detail with reference to preferred embodiments illustrated in the attached drawings.

[0032] However, the present invention is not limited to the embodiments described herein and may be embodied in other forms, and like reference numerals represent like components throughout the specification.

[0033] FIG. 1 is a drawing for explaining the configuration of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention, and FIG. 2 is a drawing for explaining an operation method of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention, as an example.

[0034] Referring to FIGS. 1 and 2, the game scene generation program (100) based on emotional information stored in the recording medium of the present invention is a program that generates an image reflecting abstract emotions based on artificial intelligence and generates a game scene reflecting emotions by utilizing attribute elements of the generated image so as to induce or amplify the emotions of a user playing a game.

[0035] In addition, the game scene generation program (100) based on emotional information stored in the recording medium of the present invention is a program that can solve the problem of labor-intensive repetitive work of adjusting attributes of a game scene by adjusting attributes to minimize errors in attributes of an image reflecting abstract emotions and a game scene, and can solve the problem of ambiguity in attribute adjustment.

[0036] Meanwhile, the present invention can separately provide only a server in which a game scene generation program (100) based on emotional information stored in the recording medium is stored.

[0037] Here, the computer is a broad computer that includes not only a general personal computer, but also a server computer accessible through a communication network, a cloud system, smart devices such as smartphones and tablets, and an embedded system.

[0038] In addition, the game scene generation program (100) based on the above emotional information may be stored and provided in a separate recording medium, and the recording medium may be one that is specially designed and configured for the present invention or one that is known and available to a person having ordinary knowledge in the computer software field.

[0039] For example, the recording medium may be a hardware device specifically configured to store and execute program instructions, either singly or in combination, such as a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD or a DVD, a magneto-optical recording medium capable of both magnetic and optical recording, a ROM, a RAM, a flash memory, etc.

[0040] In addition, the game scene generation program (100) based on the above-mentioned emotional information may be a program composed of program commands, local data files, local data structures, etc. alone or in combination, and may be a program written in high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language code such as that created by a compiler.

[0041] Below, a game scene generation program (100) based on emotional information stored in the recording medium of the present invention is described in detail.

[0042] A game scene generation program (100) based on emotional information stored in the above recording medium comprises a reference image generation means (110) and a game scene adjustment means (120).

[0043] First, the reference image generation means (110) is a means for inputting natural language and a sample image (10) regarding emotional information (20) from a user into an image generation model (1) to generate a reference image (Reference Image, 30) in which the emotional information (20) is reflected in the sample image (10).

[0044] The above emotional information (20) is information about emotions including love, fun, surprise, sadness, anger, and fear, and in addition, it can be expanded to include detailed emotional information such as liveliness, confusion, frustration, jealousy, gratitude, disregard, pride, comfort, shame, and empathy.

[0045] Meanwhile, the image generation model (1) above is a learning model constructed by training a generative model with the WEBEmo image dataset generated based on Parrrot's three-layer emotion model.

[0046] Here, each image collected in the WEBEmo image dataset generates captions regarding emotional keywords for each of the three layers.

[0047] In detail, a caption containing broad emotional information is generated in the first layer of each image included in the image dataset, a caption containing either positive or negative emotional information is generated in the second layer, and a caption containing detailed emotional information of a sub-category is generated in the third layer.

[0048] Accordingly, the image dataset consisting of each image and a pair of the captions is trained on the image generation model (1) to perform fine-tuning to improve accuracy.

[0049] At this time, the present invention uses the CLIP model to generate captions for the image dataset, but is not limited thereto, and various models capable of generating captions for images can be used.

[0050] In addition, the generative model that inputs the image dataset and trains uses one of the generative models including the diffusion model, GAN (Generative Adversarial Networks), and VAE (Variational Auto-Encoder).

[0051] Meanwhile, the reference image generation means (110) of the present invention can, in addition to receiving natural language input regarding the emotional information (20) from the user, measure the emotions of the user playing the game, and generate a reference image by receiving the measured emotional information.

[0052] Next, the game scene adjustment means (120) is a means for automatically adjusting the attribute elements of the game scene based on the reference image (Reference Image, 30) generated from the reference image generation means (110).

[0053] Here, the above attribute elements refer to attributes that can create the atmosphere of a game scene, including color distribution, contrast, brightness, and lighting position, and in addition, various attribute elements that can create the atmosphere of a game scene may be further included.

[0054] In addition, the game scene adjustment means (120) includes an attribute error calculation means (121) and an attribute adjustment means (122).

[0055] The above attribute error calculation means (121) is a means for calculating an error for each attribute element between the reference image (Reference Image, 30) and the initial game scene (Initial Scene, 40). Specifically, the error value (distance value) of the attribute element is calculated using the mean square error (MSE) function for the attribute element value of the reference image (Reference Image, 30) and the attribute element value of the initial game scene (Initial Scene, 40).

[0056] Here, the mean square error (MSE) formula for calculating the above error value is as shown in Mathematical Formula 1 below.

[0057]

[0058] (Here, Y GT is the reference image, Y p' is an image of the game scene projected onto the camera with the lighting and post-processing properties of p', where W and H are the width and height of the reference image.)

[0059] However, since there is no prior knowledge about the relationship between the attribute elements and the appearance of the initial game scene (Initial Scene, 40), there is a problem that it is very complicated to derive the error value of the attribute elements as a mathematical solution simply by using the mean square error (MSE) function, making it difficult to calculate.

[0060] Accordingly, the attribute error calculation means (121) of the present invention determines the error value of an attribute element for reflecting emotional information in an image using a random search algorithm.

[0061] In detail, the attribute error calculation means (121) sets all attribute elements of the initial game scene (Initial Scene, 40) to a reference attribute element value (P0), and then repeatedly adjusts specific values ​​of the attribute elements in the initial game scene (Initial Scene, 40) N times in stages, calculates attribute element error values ​​of the reference image (Reference Image, 30) and the N-time adjusted game scene, and adjusts the attribute element error value (P) so that the Nth attribute element error value is smaller than the N-1th attribute element error value. n ) is derived.

[0062] Afterwards, the derived adjustment attribute element values ​​(P n) by adding or subtracting 1, 2 or the above specific value, and performing an additional M random cyclic adjustment to calculate the attribute element error value of the reference image (Reference Image, 30) and the M-adjusted game scene, and the final adjusted attribute element error value (P) is the value where the M-th attribute element error value is smaller than the M-1-th attribute element error value. m ) is decided.

[0063] Meanwhile, after random traversal adjustment, the above adjustment attribute element error value (P n ) if a better value is not found, the above adjustment attribute error value (P n ) is the final adjustment attribute element error value (P m ) is decided.

[0064] That is, the attribute error calculation means (121) of the present invention determines an optimal value for minimizing the attribute element error value between the reference image (Reference Image, 30) and the initial game scene (Initial Scene, 40), which is difficult to derive as a mathematical solution, by using the random search algorithm.

[0065] The above attribute adjustment means (122) adjusts the attribute elements of the initial game scene (Initial Scene, 40) based on the error value of the attribute elements determined by the attribute error calculation means (121).

[0066] In detail, the attribute adjustment means (122) performs an adjustment to minimize the attribute element error value so that the attribute element of the initial game scene (Initial Scene, 40) is close to the attribute element of the reference image based on the error value.

[0067] Meanwhile, the present invention may further include an adjustment attribute confirmation means (130) capable of confirming changes in game scenes applied in the order of attribute elements that had the greatest influence on the generated game scene (Generated Template, 50) among the attribute elements in the game scene adjustment means (120).

[0068] FIG. 3 is a drawing for explaining a means for confirming adjustment properties of a game scene generation program based on emotional information stored in a recording medium according to one embodiment of the present invention.

[0069] Referring to FIG. 3, the adjustment attribute confirmation means (130) can track the degree of change, change stage, etc. of attribute elements from the initial game scene (Initial Scene, 40) to the generation of the generated game scene (Generated Template, 50), and can output the adjusted game scene change (60) in the order of attribute elements having the greatest influence on the generated game scene (Generated Template, 50) by visualizing it, and can manually correct the attribute element correction value of each game scene by inputting it from a game designer.

[0070] In addition, the above adjustment property confirmation means (130) comprises the above influence score calculation means (131) and the adjustment property visualization means (132).

[0071] The above influence score calculation means (131) calculates the influence (hereinafter referred to as “influence score”) of the attribute elements that adjusted the initial game scene (Initial Scene, 40) on the generated game scene (Generated Template, 50).

[0072] In detail, the influence score calculation means (131) calculates the average value of the change in the mean square error (MSE) value calculated during N repeated adjustments in the game scene adjustment means (120) to calculate the influence score, and the formula for calculating the influence score is as shown in the following mathematical expression 2.

[0073]

[0074] (Here, P n-1 is the attribute vector at the nth iteration, e i is a unit vector, P, where all elements except the ith element are 0. n-1+eipi is the attribute vector at the n-1th iteration.)

[0075] The above game scene change output means (132) visualizes and outputs the game scene change (60) applied in the order of the attribute elements with the highest influence score, that is, the greatest influence on the generated game scene (Generated Template, 50), based on the influence score calculated by the influence score calculation means (131), and allows the user, that is, the game designer, to check the game scene change applied in the order of the attribute elements with the largest influence score, as shown in FIG. 3.

[0076] Accordingly, the game designer can manually correct each attribute element change step by step and input attribute element correction values ​​for areas that are lacking or need to be supplemented.

[0077] Therefore, the game scene generation program (100) based on emotion information stored in the recording medium of the present invention can induce emotions corresponding to the game scenario from a user playing the game or amplify the emotions currently felt by the user by generating a game scene reflecting abstract information about emotions, thereby improving the immersion and interest of the game, and has the advantage of minimizing the intensive labor of workers such as game designers to implement this.

[0078] As described above, the present invention has been illustrated and described with reference to preferred embodiments, but is not limited to the above embodiments, and various changes and modifications may be made by a person skilled in the art to which the invention pertains within a scope that does not depart from the spirit of the present invention.

[0079] [Explanation of symbols]

[0080] 1: Image generation model 10: Sample image

[0081] 20: Emotional information 30: Reference image

[0082] 40: Initial game scene 50: Generated game scene

[0083] 60: Game scene change

[0084] 100: Game scene generation program based on emotional information

[0085] 110: Means for generating emotional images 120: Means for adjusting game scenes

[0086] 121: Means for calculating attribute errors 122: Means for adjusting attributes

[0087] 130: Means for checking adjustment properties 131: Means for calculating influence scores

[0088] 132: Game scene change output means

[0089] The game scene generation program based on the emotion information stored in the recording medium of the present invention can be industrially utilized in the game industry field, which amplifies the emotions of users playing the game and enhances immersion by implementing abstract emotions as images, and in the game environment design industry field, which can minimize repetitive and intensive labor for game environment design by automatically adjusting low-level features of abstract emotions based on AI.

Claims

1. Computer, A reference image generation means for receiving natural language input about emotional information from a user, inputting a two-dimensional sample image and the natural language about the input emotional information into an image generation model to generate a reference image reflecting the emotional information; and A game scene generation program based on emotional information stored in a recording medium, characterized in that it functions as a game scene adjustment means that automatically adjusts attribute elements of an initial game scene based on a reference image generated from the above-mentioned reference image generation means.

2. In paragraph 1, A game scene generation program based on emotional information stored in a recording medium, characterized in that it further includes an adjusted attribute confirmation means capable of confirming a game scene change adjusted in the order of attribute elements having a large influence on the game scene adjusted through the above game scene adjustment means.

3. In paragraph 2, Means for checking the above adjustment properties: An influence score calculation means for calculating the influence (hereinafter referred to as “influence score”) on the generated game scene (Generated Template) for each attribute element that adjusted the above initial game scene; and A game scene generation program based on emotion information stored in a recording medium, characterized in that it includes an adjustment attribute visualization means for visualizing changes in a game scene applied in order of influential attribute elements to the generated game scene (Generated Template) based on the influence score calculated by the influence score calculation means.

4. In paragraph 1, A game scene generation program based on emotion information stored in a recording medium, characterized in that the image generation model above is a learning model constructed by learning the image dataset of WEBEmo collected based on Parrott's three-layer emotion model.

5. In paragraph 4, A game scene generation program based on emotional information stored in a recording medium, characterized in that the image generation model is any one of a diffusion model, a generative adversarial network (GAN), and a variational auto-encoder (VAE).

6. In paragraph 4, Each image in the above image dataset, which has a three-layer structure, generates captions containing sentiment keywords for each image. The captions above: The first caption contains emotional information; A second caption, either positive or negative; and Includes a third caption with detailed emotional information; A game scene generation program based on emotional information stored in a recording medium, characterized in that the learning model is fine-tuned by learning an image dataset consisting of pairs of each image and the captions.

7. In paragraph 6, The above image dataset is a game scene generation program based on emotion information stored in a recording medium, characterized in that it generates the captions for each image using the CLIP model.

8. In paragraph 1, A game scene generation program based on emotion information stored in a recording medium, characterized in that the above emotion information is information about emotions including love, fun, surprise, sadness, anger and fear, and in addition, can be expanded to emotion information including liveliness, confusion, frustration, jealousy, gratitude, disregard, pride, comfort, shame and empathy as detailed emotion information.

9. In paragraph 1, A game scene generation program based on emotional information stored in a recording medium, characterized in that the above attribute elements are color, light and bloom.

10. In paragraph 1, The above game scene adjustment means: An attribute error calculation means for calculating an error for each attribute element between the above-mentioned reference image and the above-mentioned initial game scene; and A game scene generation program based on emotion information stored in a recording medium, characterized in that it includes an attribute adjustment means for performing adjustment to minimize an error value calculated from the attribute error calculation means so that the attribute elements of the initial game scene (Initial Scene) become close to the attribute elements of the reference image (Reference Image).

11. In Article 10, The above-mentioned attribute error calculation means is a game scene generation program based on emotional information stored in a recording medium, characterized by using the mean square error (MSE) function.

12. In paragraph 11, A game scene generation program based on emotional information stored in a recording medium, characterized in that the above attribute error calculation means uses a random search algorithm to derive an optimal attribute element error value.

13. In paragraph 2, A game scene generation program based on emotion information stored in a recording medium, characterized in that the influence score calculation means calculates the influence score by calculating the average value of the mean square error (MSE) change calculated during N repeated adjustments in the game scene adjustment means.

14. In paragraph 3, A game scene generation program based on emotional information stored in a recording medium, characterized in that the above-mentioned adjustment attribute verification means outputs game scenes adjusted in the order of influential attribute elements in the above-mentioned adjustment attribute visualization means, and receives correction values ​​of attribute elements of each adjusted game scene from a game designer and can manually correct them.

Images