Scene content generation and evolution method based on multi-modal large language model fitness evaluation

Abstract

The invention belongs to the technical field of intelligent optimization, and relates to a scene content generation evolution method based on multi-modal large language model fitness evaluation. The method aims at constructing a generated content evolution framework guided by LLMs, and flexible and accurate evaluation of generated content in a complex multi-modal scene is achieved. According to the method, LLMs (such as ChatGPT-4) is used as an evaluation mechanism, and an auxiliary evolutionary algorithm (EAs) (such as CMA-ES and MAP-Elites) is used for comprehensively evaluating the fitness of the generated content, so that the generation model is helped to generate images with higher quality, the images visually show the richness and diversity of scenes, and the expression specificity and visual effect are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent optimization and relates to an evolutionary method for scene content generation based on fitness evaluation of a multimodal large language model. Background Art

[0002] Evolutionary algorithms (EAs) are widely used in various optimization and search fields by borrowing mechanisms such as crossover and mutation in biological evolution, as well as other evolutionary operations. Thanks to its high applicability and ease of use, EAs are outstanding in solving problems that traditional algorithms cannot cope with, such as NP-hard problems, operations research problems with large-scale decision variables, and complex engineering optimization problems. In the field of scene content generation, EAs have also achieved remarkable results, especially in generating high-quality and diverse solutions. With its powerful global search capabilities, EAs can continuously optimize features in multi-generation solution spaces, thereby efficiently optimizing the generation space of the generation model (such as hyperparameters, latent space, and network architecture), providing innovative ideas for solving complex problems.

[0003] EAs have shown outstanding advantages in assisting scene content generation. The following are two representative application examples: (1) By optimizing the hyperparameters of the neural network through the genetic algorithm (GA), a balance is achieved between the generation effect and reasoning performance of the generation model, thereby promoting the generation of high-quality images. (2) Using the evolutionary quality diversity (QD) algorithm, the latent space of the generator can be efficiently searched to discover game levels with a diverse combination of behavioral characteristics (BCs), thereby generating scenes with beautiful visual effects, excellent quality and high playability. These applications fully demonstrate the important role of EAs in improving the quality and diversity of generated content.

[0004] Although existing research has shown that using EAs to automatically search for hyperparameters and latent spaces of generative models can effectively discover optimal solutions, the current design of EAs is still highly dependent on human intervention and requires customized development for specific problems. As the complexity of problems continues to increase, automated or semi-automated design methods are expected to become the main trend in future algorithm development. In these designs, the construction of the fitness function is particularly critical because it directly determines the quality and preference of the generated solution, which in turn affects the overall performance of the algorithm.

[0005] At present, some studies have been devoted to designing fitness functions for complex optimization problems, including manual design and design using other machine learning techniques (such as support vector machines, decision trees, and neural networks). However, the existing fitness function design still has many limitations, especially when facing complex real-world scenarios, it is often difficult to take into account the diversity and complexity of the scenario. This makes it difficult for the fitness function to provide a comprehensive evaluation, and the generated solution tends to be biased towards certain single features. In addition, these design methods are usually time-consuming and resource-intensive, and there are obvious deficiencies in versatility and adaptability.

[0006] Recently, the emergence of large language models (LLMs) such as LLaMA, Claude, and ChatGPT series has promoted a new round of breakthroughs in the field of artificial intelligence. The powerful performance of LLMs is due to their self-supervised pre-training on massive multimodal datasets, deep neural network architecture, multi-layer structure, and large parameter scale. These characteristics make LLMs an expert tool with excellent perception ability, capable of recognizing complex patterns and parsing multidimensional datasets. In recent years, significant progress has been made in the research on the combination of LLMs and EAs, including using LLMs to optimize genetic operators, design adaptive algorithms, and solve complex problems through their rich domain knowledge and advanced generation capabilities, thereby significantly improving the performance of EAs. However, although the combination of LLMs and EAs has achieved certain results, there is still a lack of in-depth research on using LLMs to support EAs for evaluation, especially in the effectiveness verification and formal definition of problems in multimodal applications.

[0007] LLMs can flexibly and accurately evaluate the content in complex multimodal scenes through mechanisms such as perception, cross-modal association, context analysis, and prompt-based directional evaluation. Using the analytical capabilities of LLMs to evaluate and guide the evolutionary process aims to solve the problem of how to fairly and reasonably evaluate fitness in scene content generation, thereby helping to achieve a higher level of creativity and efficiency. At present, using LLMs to promote the evaluation of EAs in complex scene content generation is still an open problem that needs to be solved. This problem involves at least two key research directions: one is how to design sophisticated multimodal LLMs prompts to ensure that the generated solutions are both high-quality and diverse; the other is how to construct an interaction mechanism between EAs and LLMs to provide an accurate and efficient automated fitness evaluation method. Summary of the invention

[0008] In order to solve the above problems, the present invention proposes a scene content generation evolution method based on the fitness evaluation of a multimodal large language model. The method aims to build a generated content evolution framework guided by LLMs to achieve flexible and accurate evaluation of generated content in complex multimodal scenes. This method uses LLMs (such as ChatGPT-4) as an evaluation mechanism to assist evolutionary algorithms (EAs) (such as CMA-ES, MAP-Elites) to comprehensively evaluate the fitness of generated content, thereby helping the generative model to generate higher quality images. These images intuitively show the richness and diversity of the scene, and enhance the concreteness and visual effect of expression.

[0009] The present invention adopts the following technical solution:

[0010] A scene content generation evolution method based on multimodal large language model fitness evaluation includes the following steps:

[0011] Step 1: Construct evolutionary algorithms (EAs) to optimize the problem model of the generation space of the image generation model, and combine them with large language models (LLMs) to assist in improving the adaptability and flexibility of the evaluation.

[0012] (1.1) The process of optimizing image generation models by EAs can be viewed as an optimization task of latent vectors, network architectures, and specific parameter settings. Specifically, the generation space of image generation models contains all possible solutions, covering latent vectors, network architectures, and various parameter settings, which are collectively referred to as genotypes. Based on these genotypes, image generation models generate images that intuitively present the content of the target scene in a visual form. Subsequently, the image content is evaluated using a fitness function. The goal of optimization is to adjust the parameters in the genotype to maximize the value of the fitness function, thereby generating high-quality images.

[0013] (1.2) In order to solve the problem that traditional fitness functions lack universality and adaptability in image content generation, LLMs are combined with EAs, and the powerful analytical ability of LLMs is used to evaluate and guide the evolution process, thereby achieving a more comprehensive and flexible evaluation. LLMs process the input content by combining multimodal mapping and feature fusion methods, evaluate it and finally output the fitness score. Multimodal mapping is implemented based on technologies such as natural language processing (NLP) models and visual models, while feature fusion is based on attention mechanisms or linear transformations.

[0014] Therefore, the goal of LLMs-assisted EAs to optimize the generation space is defined as follows, as shown in formula (1):

[0015]

[0016] Where F represents the fitness function; Φmodal represents a multimodal function that can receive and process inputs from different modalities; m represents the content generated by the image generation model; P represents the text prompt describing the evaluation criteria; LLMs represents the large language model; encode represents an encoder that encodes the features of each modality into a form that LLMs can understand. In particular, for the input image content, Φ modal It is specifically defined as the image feature extraction function Φ img , as shown in formula (2):

[0017] Φ img (m,P)={φ style (m),φ layout (m),φ creativity (m),φ play (m)}(2)

[0018] Where m represents the content generated by the image generation model; P represents the text prompt describing the evaluation criteria, which determines the specific dimensions of the evaluation image; φ style (m) represents the extraction of artistic style features to evaluate whether the overall visual style and color matching are consistent with the theme; layout (m) represents the extraction of layout and structural features to evaluate whether the region division is clear and well-correlated; φ creativity (m) represents the evaluation of the creative dimension, identifying unique game mechanisms or hidden easter eggs in the map; play (m) represents the evaluation of playability, which includes the interactivity of elements in the image and the player's gaming experience.

[0019] Step 2 and Step 1 elaborated on the mathematical model of optimizing the generation space of the image generation model and the basic principles of evaluation; the following Step 2 will focus on the specific implementation framework. Construct a scene generation content optimization framework guided by large language models (LLMs), namely LEGCE. The core goal of this framework is to assist in evaluating the fitness value of the generated content by integrating LLMs into evolutionary algorithms (EAs).

[0020] (2.1) Initialize the population. First, randomly initialize a population pop1 consisting of multiple individuals, where each individual z1, z2, …, z n represents the genotype, and these parameters represent the latent vector of the image generation model, the network architecture, or specific settings in the image generation process.

[0021] (2.2) Generate image content and evaluate fitness. Each individual in the population generates an image m through the image generation model. i , and then according to step (1.2), the generated image is analyzed and evaluated in combination with LLMs to calculate its fitness value fit.

[0022] (2.3) Generate offspring. In each generation before reaching the maximum number of generations maxGen, the genetic operations of EAs (including crossover and mutation) are used to generate new offspring individuals based on the current population.

[0023] (2.4) Evaluate the candidate solution. Each new individual z i Generate new content m again through the image generation model i , and evaluate using LLMs according to step (1.2) to obtain the fitness value fit.

[0024] (2.5) Update the population to optimize the image content. The current population pop gen According to the fitness value, the newly generated population pop gen+1 Replace, retaining the better image content.

[0025] (2.6) Output the optimized image and generate the solution. This process continues until the stopping condition is met. Finally, the newly generated population pop gen+1 The individual with the highest fitness value is selected as the optimal solution.

[0026] Compared with the existing method, the present invention has the following beneficial effects:

[0027] This paper constructs a problem model of optimizing the generation space of image generation models by evolutionary algorithms (EAs), and explains the basic principle of LLMs evaluation. At the same time, the LEGCE framework is proposed, which uses LLMs as an evaluation mechanism to assist EAs in fitness evaluation of the content generated by the image generation model. The LEGCE framework can provide EAs with comprehensive and efficient evaluation, especially in complex scenes, thereby improving the quality of the content generated by the image generation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a specific structural diagram of an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of DCGAN in an embodiment of the present invention;

[0030] Figure 3 The figure is a framework flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0032] like Figure 1 As shown, the embodiment of the present invention is a scene content generation evolution method based on multimodal large language model fitness evaluation. This embodiment takes game content generation as an example and specifically includes the following steps:

[0033] Step 1: Construct evolutionary algorithms (EAs) to optimize the problem model of the game content generation model generation space, and combine large language models (LLMs) to assist in improving the adaptability and flexibility of the evaluation.

[0034] (1.1) As players’ demand for richer game content increases, the video game industry faces the high cost of manually creating game content. In this context, artificial intelligence technology has been widely studied and applied to automatically generate game content. In recent studies, the MAP-Elites algorithm and its variants have successfully generated game content with both quality and diversity, providing designers with rich inspiration and diverse choices. Specifically, MAP-Elites defines behavioral characteristics (BCs) by establishing an ordered mapping between game content (i.e., solutions) and its related attributes (such as the number of jumps, etc.). Before the search begins, MAP-Elites predefines n game attributes that need to be explored, thereby constructing a Cartesian (behavior) space of size n.

[0035] The goal of this problem is to find a set of game content that is both diverse and high-quality in the continuous search space Z. This goal can be formalized as an optimization problem, the core task of which is to fill the elite solution archive M and ensure that each grid in the behavior space is occupied by a solution that maximizes the fitness function, as shown in formula (3):

[0036]

[0037] Among them, M represents the archive storing the elite solution; b c is a feature vector that maps the solution vector z to a point in the behavior space; z represents the solution vector in the search space Z; p z is a symbolic encoding used to represent this patch type in the generated game content, usually mapping the genotype z to the phenotype p through an image generation model (such as GAN) z ; f is the fitness function, which is used to evaluate the generated content.

[0038] (1.2) In order to solve the problem that traditional fitness functions lack universality and adaptability in game content generation, LLMs are combined with EAs, and the powerful analytical ability of LLMs is used to evaluate and guide the evolution process, thereby achieving a more comprehensive and flexible evaluation. Next, the Super Mario Bros game is used as an application example to illustrate the basic principle of ChatGPT-4 combined with EAs to evaluate game content generation.

[0039] In the specific implementation, the specific steps included in step (1.2) are as follows:

[0040] (1.2.1) First, train the generative model GAN.

[0041] The training is performed using a deep convolutional generative adversarial network (DCGAN) combined with the WGAN algorithm. The training levels are encoded by assigning unique integers to 17 different brick types, which are then converted into one-hot encoded vectors and input into the discriminator. The entire architecture is shown in Figure 1. Figure 2 As shown in the figure, the generator takes a latent vector of size 32 as input, and this 32-dimensional latent vector constitutes a continuous search space. Through a series of transposed convolution operations, the generator gradually increases the spatial resolution of the output while reducing the number of channels, and finally generates a 17×64×64 matrix, where 17 channels correspond to 17 brick types. The discriminator takes the generated 17×64×64 matrix as input, and each channel represents a brick type. The input is processed through a series of convolutional layers and finally outputs a scalar to judge the authenticity of the input level. Through this architectural design, each training level is represented as a 64×64 matrix containing 17 channels, which not only ensures the compatibility of the data format with the GAN framework, but also retains the semantic information of each brick type, so that it can effectively learn to generate high-quality and diverse levels.

[0042] The DCGAN was trained using the RMSprop optimizer for 5000 iterations and the learning rate was set to 5×10 -5 , the batch size is 32. The discriminator iterates five times before each generator update. The training data for GAN comes from 15 classic levels in the Mario AI Framework. These level files are in plain text format. Each line represents a row of bricks in the level. All lines have the same length, and each level contains 16 bricks in height. In order to adapt GAN to process rectangular images of fixed size, each input image is generated by extracting a sliding window of 56 (width) × 16 (height) from the original level image. The sliding window moves from left to right, one brick at a time, to ensure that the generated input image maintains a consistent size.

[0043] After the GAN training is completed, the generator successfully learns the mapping relationship from genotype to phenotype. During the evolution process, the output of the generator is initially a 17×64×64 three-dimensional array. It is then pruned to a 17×16×56 array by retaining the first 16 elements of the second dimension and the first 56 elements of the third dimension. Next, an argmax operation is performed on the output vector of each brick to convert it to an integer value, thereby generating a complete level. These levels can be decoded and further evaluated through the Mario AI Framework.

[0044] (1.2.2) After GAN training is completed, ChatGPT-4 assists EAs in optimizing the generation space of GAN.

[0045] First, the latent vector z in the generated space is randomly initialized and input into the GAN to generate the corresponding Mario level. Then, the generated level is saved to the cloud server, and ChatGPT-4 is called to evaluate the level according to the predefined prompts by providing the URL of the image. The evaluation covers multiple aspects, including artistic style, functionality, creativity, and originality.

[0046] Specifically, ChatGPT-4 combines multimodal mapping and feature fusion methods to process and analyze the input content and finally output a fitness score. Among them, multimodal mapping is achieved through natural language processing (NLP) models, visual models and other technologies; feature fusion is based on attention mechanism or linear transformation. The optimization goal is shown in formula (1). In particular, for the input image content, Φ modal It is specifically defined as the image feature extraction function Φ img , as shown in formula (2).

[0047] Step 2 and Step 1 elaborate on the mathematical model of optimizing the generation space of the image generation model and the basic principles of evaluation; the following Step 2 will focus on the specific implementation framework. A generated content optimization framework guided by large language models (LLMs) is constructed, called LEGCE. In the specific application example of game content generation, the framework integrates ChatGPT-4 with CMA-ES and MAP-Elites algorithms, and constructs sub-frameworks LEGCE-CMAES and LEGCE-ME, respectively, to assist in evaluating the fitness value of the content generated by GAN.

[0048] (2.1) First, the subframework LEGCE-CMAES is implemented based on the CMA-ES algorithm, which dynamically adjusts the covariance distribution of the population, optimizes the latent vector and parameter settings, and thus searches for the optimal solution. The specific steps are as follows:

[0049] (2.1.1) Initialize the population. First, randomly initialize a population pop1 consisting of multiple individuals, where each individual z1, z2, …, z n is the genotype, representing the potential vector of the GAN generation space.

[0050] (2.1.2) Generate image content and evaluate fitness. Each population individual generates content m through the GAN model i ,Then according to step (1.2.2), ChatGPT-4 is used to analyze and evaluate the generated image and calculate its fitness value.

[0051] (2.1.3) Update the CMA-ES parameters, including the distribution mean m and the covariance matrix C. As shown in formula (4) and formula (5):

[0052] m gen+1 =f restart (m gen ,pop gen ),(4)

[0053] C gen+1 =f adjust (C gen ,pop gen ),(5)

[0054] Among them, f restart represents the function of adjusting the mean m of the distribution; f adjust Represents the function of adjusting the covariance matrix C according to the fitness value ranking of the previous generation solution; m gen Represents the distribution mean of the gen generation; m gen+1 represents the distribution mean of the gen+1 generation; C gen represents the covariance matrix of the gen generation; C gen+1 represents the covariance matrix of the gen+1 generation; pop gen Represents the population of generation gen.

[0055] (2.1.4) Generate offspring. In each generation before reaching the maximum number of generations maxGen, adjust the CMA-ES sampling distribution N(m,C) and generate new offspring individuals pop based on the current population. gen+1 .

[0056] (2.1.5) Evaluate candidate solutions. Each new individual z i Generate new content m again through GAN i , and evaluated by ChatGPT-4.

[0057] (2.1.6) Update the population to optimize the image content. The current population pop gen According to the fitness value, the newly generated population pop gen+1 Replace, retaining the better image content.

[0058] (2.1.7) Output the optimized image generation solution. This process continues until the stopping condition is met. Finally, the newly generated population pop gen+1 The individual with the highest fitness value is selected as the optimal solution.

[0059] (2.2) The subframework LEGCE-ME is implemented based on the MAP-Elites algorithm, such as Figure 3As shown in the figure, by dividing the search space into different feature regions and storing the solution with the highest fitness in each region, a high-quality and diverse solution set is generated. The specific steps are as follows:

[0060] (2.2.1) Initialize the archive. First, create an empty archive A and predefine behavioral characteristics (BCs) to store the optimal solution.

[0061] (2.2.2) Initialize the population. In each generation before reaching the maximum initial value maxInit, a random variable z is randomly initialized i , representing the potential vector of the GAN generation space.

[0062] (2.2.3) Generate image content and evaluate fitness. Each population individual generates content m through the GAN model i ,Then according to step (1.2.2), ChatGPT-4 is used to analyze and evaluate the generated image and calculate its fitness value.

[0063] (2.2.4) Generate offspring. When the number of generations exceeds maxInit, in each generation before the maximum number of generations maxGen is reached, a random individual z is selected from archive A. i , and then generate new offspring individuals through crossover mutation.

[0064] (2.2.5) Evaluate the candidate solution. The newly generated individual z i Generate new content m again through the GAN model i , and is evaluated by ChatGPT-4 to obtain the fitness value fit. At the same time, its feature descriptor b is calculated c , used to determine the storage location of the solution in archive A.

[0065] (2.2.6) Update archive A to optimize the image content. c The solutions of the corresponding cells are compared. If the cell is empty or the fitness value of the candidate solution is better, the archive is updated to store the better image content.

[0066] (2.2.7) Output the optimized archive A to generate the solution. This optimization process continues until the stopping condition is met. Finally, the individuals stored in archive A are the optimized optimal solution set.

[0067] (2.3) Experimental parameter setting and behavioral characteristics (BCs) definition.

[0068] The population size of CMA-ES is set to λ=17 and the mutation strength σ=0.5. For CMA-ME, five improved transmitters are deployed in a single run with a population size of λ=37. The mutation strength of both CMA-ME and MAP-Elites is set to σ=0.2. The initial population of MAP-Elites is set to 100. For comparison, the solutions generated by CMA-ES are assigned to grid locations according to their BCs and populated into a pseudo archive.

[0069] BCs are defined by KL divergence, and two distinctive hand-designed scenes from the Mario AI Framework are selected for analysis. A two-dimensional BC space is constructed by calculating the KL divergence of the tile pattern between the real scene and the generated scene. In the initial run of the experiment, the maximum and minimum BC values ​​covered by each algorithm are observed, providing a preliminary estimate of the range of each BC. The range of KL divergence is set to [0,4.5], with the real scene as a reference, and the resolution of the map is set to 60×60.

[0070] Step 3: Define the prompt mechanism for game content generation.

[0071] By setting up diverse scoring criteria and content comparison mechanisms, the effectiveness of content evaluation is improved, thereby supporting the generation of high-quality and diverse content. Simply adding instructions to the input text, a method called discrete prompts, can effectively guide ChatGPT-4 to complete specified tasks. Discrete prompts have the advantages of simple operation and minimal impact on computational cost, without access to all parameters and gradients of ChatGPT-4. However, the performance of this method is significantly affected by the choice of prompts, so it is crucial to design reasonable prompts. Effective prompts should have clear goals and specific evaluation criteria, and be combined with specific examples to improve the performance of the model.

[0072] To this end, an innovative prompt definition mechanism is proposed, which is specifically designed to optimize game content generation. When evaluating the quality of game levels, the focus is usually on aspects such as style, innovation, and layout design. Based on these key characteristics, a set of evaluation criteria are refined using ChatGPT-4 to guide the evaluation and improvement of Mario levels.

[0073] (1) Tile richness: A well-designed level should contain multiple types of tile elements, such as floors, walls, obstacles, and decorations. Diverse tile types not only enhance gameplay, but also improve visual appeal while adding depth and complexity to game strategies.

[0074] (2) Space Utilization: Level layouts must be carefully planned to ensure that each area has a clear function or purpose. By properly arranging the position and size of elements, large areas of useless blank areas can be avoided, thereby improving the overall design efficiency of the level.

[0075] (3) Player completion: To improve player completion, level design must ensure that players can successfully pass through the level from left to right. Designers should help players advance smoothly and achieve their goals through clear path guidance and sufficient interactive elements.

[0076] (4) Level difficulty: Mario levels need to be appropriately challenging. A reasonable balance of level difficulty allows players to enjoy the fun of the challenge while enhancing the playability and appeal of the level.

[0077] (5) Comprehensive quality: Comprehensive quality comprehensively evaluates the design and execution of game levels, covering not only tile richness and space utilization, but also multiple dimensions such as visual effects, style layout, etc., reflecting the overall design level and artistry of the level.

[0078] In addition, compared to directly evaluating a single image, combining reference images can help ChatGPT-4 understand the context and complexity of the task more comprehensively, thereby improving the accuracy of evaluation and reasoning. Assume that the reference image set input to ChatGPT-4 is M ref ={m ref,1 ,m ref,2 ,…,m ref,n}, and a generated image m to be evaluated eval During the evaluation process, ChatGPT-4 will integrate the information of the reference image into m eval Based on this, the optimization objective is defined as shown in formula (6):

[0079]

[0080] Among them, F represents the fitness function; Z is the generation space; z represents the genotype, that is, the potential vector of the GAN living space; Φ img represents the image feature extraction function; m eval represents the image content generated by GAN; P represents the text prompt describing the evaluation criteria; m ref,n represents the nth reference image; LLMs represents the large language model, i.e. ChatGPT-4; encode represents an encoder that encodes the features of each modality into a form that ChatGPT-4 can understand.

[0081] In addition, the generated levels need to meet certain design criteria (for example, each level should have a clear start and end point). The prompt clearly states that if the current level does not meet these requirements, its score should be appropriately reduced.

[0082] The LEGCE framework proposed in this invention makes full use of the powerful functions of large language models (LLMs) to conduct a comprehensive and detailed evaluation of the generated content from multiple aspects such as aesthetic appeal, creativity and functionality. Compared with the traditional fitness function, this framework provides a more superior evaluation system. Experimental results show that in the application of Mario game level generation, the LEGCE framework significantly improves the quality and diversity of the generated levels, and surpasses the traditional evaluation methods in terms of texture richness, duty cycle, challenge and playability indicators. In addition, the results of user studies also show that participants prefer the levels generated by the LEGCE framework, especially in terms of aesthetic design, challenge and playability.

Claims

1. A scene content generation evolution method based on multimodal large language model fitness evaluation, characterized in that: The steps include: Step 1: Construct an evolutionary algorithm EAs to optimize the problem model of the image generation model generation space, and combine it with the large language model LLMs to assist in improving the adaptability and flexibility of the evaluation; (1.1) The process of EAs optimizing image generation models is viewed as an optimization task of latent vectors, network architectures, and specific parameter settings; specifically, the generation space of image generation models contains all possible solutions, covering latent vectors, network architectures, and various parameter settings, which are collectively referred to as genotypes; based on these genotypes, image generation models generate images that intuitively present the content of the target scene in a visual form; Subsequently, the image content is evaluated by the fitness function; the goal of optimization is to adjust the parameters in the genotype to maximize the fitness function value, thereby generating high-quality images; (1.2) Combine LLMs with EAs and use the powerful analytical capabilities of LLMs to evaluate and guide the evolutionary process; The goal of LLMs-assisted EAs to optimize the generation space is defined as follows, as shown in formula (1): Where F represents the fitness function; Φ modal represents a multimodal function that can receive and process inputs from different modalities; m represents the content generated by the image generation model; P represents a text prompt describing the evaluation criteria; LLMs represents a large language model; encode represents an encoder that encodes the features of each modality into a form that LLMs can understand; in particular, for the input image content, Φ modal It is specifically defined as the image feature extraction function Φ img , as shown in formula (2): F img (m,P)={φ style (m),φ layout (m),φ creativity (m),φ play (m)}(2) Where m represents the content generated by the image generation model; P represents the text prompt describing the evaluation criteria, which determines the specific dimensions of the evaluation image; φ style (m) represents the extraction of artistic style features to evaluate whether the overall visual style and color matching are consistent with the theme; layout (m) represents the extraction of layout and structural features to evaluate whether the region division is clear and well-correlated; φ creativity (m) represents the evaluation of the creative dimension, identifying unique game mechanisms or hidden easter eggs in the map; play (m) represents the evaluation of playability, including the interactivity of elements in the image and the player's gaming experience; Step 2: Build a large language model-guided scenario generation content optimization framework, namely LEGCE. The goal of this framework is to assist in evaluating the fitness value of generated content by integrating LLMs into evolutionary algorithms EAs. The details are as follows: (2.1) Initialize the population; First, randomly initialize a population pop1 consisting of multiple individuals, where each individual z1, z2, …, z n represents the genotype, and these parameters represent the latent vector of the image generation model, the network architecture, or the specific settings in the image generation process; (2.2) Generate image content and evaluate fitness; each individual in the population generates an image m through the image generation model i , then according to step (1.2), the generated image is analyzed and evaluated in combination with LLMs to calculate its fitness value fit; (2.3) Generate offspring; in each generation before reaching the maximum number of generations maxGen, use the genetic operations of EAs, including crossover and mutation, to generate new offspring individuals based on the current population; (2.4) Evaluate candidate solutions; each new individual z i Generate new content m again through the image generation model i , and use LLMs to evaluate according to step (1.2) to obtain the fitness value fit; (2.5) Update the population to optimize the image content; the current population pop gen According to the fitness value, the newly generated population pop gen+1 Replace,retaining the better image content; (2.6) Output the optimized image and generate a solution; this process continues until the stopping condition is met; finally, the newly generated population pop gen+1 The individual with the highest fitness value is selected as the optimal solution.

Images