Computer systems and programs

Abstract

To provide a new technology that enables users to obtain content with the specifications and content they desire when generating content and retrieving it from the generation unit. [Solution] When a server system that controls the provision of content receives a given generation instruction information, it generates and obtains candidate products by providing instruction information based on the said generation instruction information to a predetermined generation unit. Then, it generates evaluation criteria for evaluating the candidate products based on the generation instruction information, evaluates the candidate products based on the evaluation criteria, and performs control to output candidate products that satisfy the given passing conditions as presented products.

Description

Technical Field

[0001] The present invention relates to computer systems and the like.

Background Art

[0002] Patent Document 1 describes training an AI (Artificial Intelligence) model from multiple game plays of a scenario of a game application using training state data collected from multiple game plays of the scenario and success criteria associated with each of the multiple game plays.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is generative AI that can generate data in various content categories such as text, characters, images, and voices. In the field of game production, there is a search for using a generation unit typified by generative AI in game production.

[0005] In order to cause the generation unit to generate data, it is necessary to give the generation unit information for generation instructions (generation instruction information) called a prompt. The generation instruction information includes multiple types of contexts that describe what to generate, under what preconditions to generate, and what data to refer to for generation.

[0006] Users create generation instructions to obtain content (products) with their desired specifications and content, but the generated content is not always as expected. If the generated content does not meet the user's specifications and content, the user revises the generation instructions and provides them to the generation unit again to restart the generation process. Users must repeat this process, which is one reason why production efficiency does not improve as expected despite utilizing the generation unit.

[0007] These problems are not limited to the use of generation units in video game production. For example, the same applies when using generation units in content that provides users (players) with virtual experiences in a virtual space. For instance, if a user specifies the content they want to play each time they play, and the generation unit generates and provides unique content each time, the problem of providing content that does not meet the user's request may occur.

[0008] The problem that this invention aims to solve is to provide a new technology that makes it possible for a user to obtain content with the specifications and content desired by the user when generating content in a generation unit and obtaining that content. [Means for solving the problem]

[0009] The first invention for solving the above problems is a generation instruction information receiving means for receiving given generation instruction information (for example, the generation instruction information receiving unit 206 in Figure 14, step S10 in Figure 16), A candidate product acquisition control means (for example, the candidate product acquisition control unit 212 in Figure 14, step S34 in Figure 16) performs control to acquire N different candidate products by providing instruction information based on the aforementioned generation instruction information to a predetermined generation unit, Based on the production instruction information, an evaluation criteria generation means (for example, the evaluation criteria generation unit 232 in Figure 14, step S48 in Figure 16) generates evaluation criteria for evaluating the candidate product, The computer system includes an evaluation means (for example, the evaluation unit 234 in Figure 14, step S50 in Figure 16) for evaluating the candidate product based on the aforementioned evaluation criteria.

[0010] The second invention is a computer system in which, in the above-described computer system, the evaluation means performs control to output as a presented product a candidate product whose evaluation result satisfies a given acceptance condition (for example, YES in step S52 of Figure 16 → step S120).

[0011] According to the first or second invention, the computer system causes the generation unit to generate candidate products based on the received generation instruction information, and evaluates the candidate products according to the evaluation criteria based on the said generation instruction information. According to the second invention, if the evaluation result is satisfactory, it becomes possible to output it.

[0012] In other words, the user only needs to set the generation instruction information, eliminating the need to check each time a candidate product is generated whether it matches their desired specifications and content. Therefore, when the generation unit generates content and retrieves it, it becomes possible to obtain content that matches the user's desired specifications and content.

[0013] The third invention is a computer system in which, in the above-described computer system, the candidate product acquisition control means repeatedly modifies the instruction information and provides it to the generation unit (for example, steps S56 to S58 in Figure 16) to perform control for acquiring N candidate products.

[0014] According to the third invention, the computer system can modify the instruction information to create various variations and provide them to the generation unit. Therefore, by simply setting the generation instruction information, it becomes possible to obtain various candidate products based on various instruction information.

[0015] The fourth invention is a computer system in which, in the above-described computer system, the candidate product acquisition control means provides the generation unit with feedback information based on the evaluation results of the evaluation means for past products which are candidate products that have been generated in the past, and new instruction information based on said past products, and repeatedly performs control to acquire new candidate products, thereby performing control to acquire N candidate products (for example, step S54 in Figure 16).

[0016] According to the fourth invention, the computer system can generate new candidate products by providing the generation unit with evaluation results for previously generated candidate products and instruction information based on those candidate products. In other words, it becomes possible to repeatedly retries and refine a given candidate product.

[0017] The fifth invention is a computer system in which, in the above-described computer system, the candidate product acquisition control means performs the above repetition a given number of times.

[0018] According to the fifth invention, the computer system can obtain candidate products by repeating the process a given number of times.

[0019] The sixth invention is a computer system in which the feedback information is information for making an evaluation result that does not meet a given acceptance condition based on the evaluation result satisfy the acceptance condition.

[0020] According to the sixth invention, the computer system can provide feedback to the generation unit to generate new candidate products that meet the acceptance criteria, replacing candidate products that do not meet the acceptance criteria.

[0021] The seventh invention is a computer system in the above computer system, wherein the passing conditions include a plurality of requirements, and the feedback information is information for satisfying the requirements included in the passing conditions that the evaluation result does not meet.

[0022] According to the seventh invention, the computer system can generate feedback information as information for satisfying the requirements included in the passing conditions that the evaluation result does not meet.

[0023] The eighth invention is a computer system in the above computer system, wherein the evaluation criteria is a criterion for determining the degree to which the content of the generation instruction information is satisfied.

[0024] According to the eighth invention, the computer system can evaluate the degree to which the content of the generation instruction information is satisfied.

[0025] The ninth invention is a computer system in the above computer system, wherein the evaluation criteria generation means analyzes the content of the generation instruction information for each given analysis item, and generates criteria for each analysis item (for example, context items in FIGS. 7, 8, and 9) for determining the degree to which the content of the generation instruction information is satisfied. The given passing conditions based on the evaluation result include requirements based on the criteria for each analysis item, and the feedback information is information related to the requirements included in the passing conditions that the evaluation result does not meet.

[0026] According to the ninth invention, the computer system can provide feedback to improve the analysis items related to the requirements that the evaluation result does not meet.

[0027] The tenth invention is a computer system in which, in the above-described computer system, the generation unit has a content element-specific generation unit (for example, a content element generation AI4 in Figure 1) that generates content elements of content playable by the user, and the generation instruction information receiving means receives the generation instruction information as an instruction to generate the content or one or more of the content elements.

[0028] According to the tenth invention, the computer system can evaluate and acquire content elements of playable content as candidate products. Here, "playing" includes not only executing games, but also playing and viewing images, audio, and video.

[0029] The eleventh invention is a computer system in which the content is game content.

[0030] According to the eleventh invention, a computer system becomes capable of evaluating and acquiring game content as candidate products.

[0031] The twelfth invention is a computer system in which, in the above-described computer system, the candidate product acquisition control means performs the following actions: selecting an element-specific production unit to produce the candidate product based on the production instruction information (for example, step S16 in Figure 16); and providing the selected element-specific production unit with the instruction information based on the production instruction information (for example, step S34 in Figure 16).

[0032] Furthermore, the thirteenth invention is a computer system in which, in the above-mentioned computer system, the candidate product acquisition control means creates the instruction information to be given to the selected element-specific generation unit based on the generation instruction information and the selected element-specific generation unit (for example, element-specific generation instruction information 710 in Figure 15, step S30 in Figure 16).

[0033] According to the 12th or 13th invention, the computer system can select a generation unit responsible for generation and generate candidate products.

[0034] The fourteenth invention is a computer system in which, in the above-described computer system, the candidate product acquisition control means performs control to variably determine the order in which the element-specific generation units that provide the instruction information are selected when a plurality of the element-specific generation units are selected (for example, step S32 in Figure 16).

[0035] According to the 14th invention, the computer system can improve the efficiency of acquiring candidate products by variably determining the order in which instruction information is given to the element-specific generation unit.

[0036] The fifteenth invention is a computer system in which, in the above-described computer system, the evaluation means controls the timing of evaluation of the generated candidate products based on the generation status of N candidate products (for example, step S50 in Figure 16).

[0037] According to the 15th invention, the computer system can change the evaluation timing for the candidate product based on the generation status of the candidate product, and perform an appropriate evaluation according to the composition of the candidate product.

[0038] Furthermore, the sixteenth invention is a computer system in which, in the above-described computer system, the evaluation means controls the timing of evaluation of the generated candidate product based on the type of the candidate product (for example, the type of content category selected in the category setting UI 10 in Figure 4) (for example, step S22 in Figure 19).

[0039] "Candidate product generation status" refers to the status of the candidate product during the process of its creation. For example, if a candidate product has multiple components, it refers to the status at each stage where the components are output from the generation unit. In the case of a candidate product with multiple components, there may be close relationships between the components. In such cases, a correct evaluation cannot be made unless all of the closely related components have been generated.

[0040] Specifically, for example, if the candidate product is an RPG (Role Playing Game), it may be possible to perform a test play on a computer in a virtual execution environment and evaluate it based on the game log. In this case, the candidate product has components such as a game field, player characters, enemy NPCs, etc. Therefore, a correct evaluation is not possible unless all of these are generated. If the candidate product is an actionable "character," it has components such as a character model, skin, motion, etc. Therefore, a correct evaluation is not possible unless all of these are generated.

[0041] On the other hand, in cases where the candidate product is a single component such as music or a still image, a correct evaluation is possible even if the evaluation is performed only when that candidate product is generated. The same applies when the candidate product is a "character form".

[0042] According to the 16th invention, the computer system can change the evaluation timing for a candidate product based on the type of candidate product, and perform an appropriate evaluation according to the composition of the candidate product.

[0043] The 17th invention is a program for causing a computer system to function as: a generation instruction information receiving means for receiving given generation instruction information; a candidate product acquisition control means for performing control to acquire N different candidate products (N≧2) by providing instruction information based on the generation instruction information to a predetermined generation unit; an evaluation criterion generation means for generating evaluation criteria for evaluating the candidate products based on the generation instruction information; and an evaluation means for evaluating the candidate products based on the evaluation criteria.

[0044] According to the 17th invention, it is possible to realize a program that can enable a computer system to perform the same functions as the first invention. [Brief explanation of the drawing]

[0045] [Figure 1] A system configuration diagram showing an example of a content delivery system. [Figure 2] A diagram illustrating the process of generating and providing new content. [Figure 3] A diagram illustrating the process of generating and providing new content. [Figure 4] A diagram illustrating an example of a user interface. [Figure 5] A diagram illustrating an example of a user interface. [Figure 6] A diagram showing an example of setting up element decomposition and a role generation AI. [Figure 7] A diagram illustrating examples of requirements and evaluation criteria. [Figure 8] A diagram illustrating examples of requirements and evaluation criteria. [Figure 9] A diagram illustrating examples of requirements and evaluation criteria. [Figure 10] A diagram illustrating machine learning in feedback AI. [Figure 11] A diagram illustrating machine learning in feedback AI. [Figure 12] A diagram illustrating machine learning in feedback AI. [Figure 13] A diagram showing examples of programs and data stored by a server system. [Figure 14] A diagram showing an example of the functional configuration of the server processing unit. [Figure 15] A diagram showing an example of the data structure of generated and managed data. [Figure 16] A flowchart illustrating the process flow related to content creation and delivery. [Figure 17] A diagram illustrating a modified example. [Figure 18] This figure shows an example of the data structure of the step-by-step generation definition data in a modified example. [Figure 19] Flowchart in a modified example. [Figure 20] A flowchart illustrating the step-by-step generation process in the modified example. [Figure 21] A diagram illustrating the machine learning process used by an AI for generating evaluation criteria for specific categories. [Figure 22] A diagram illustrating the machine learning process of an AI that generates element-specific generation instruction information. [Modes for carrying out the invention]

[0046] Examples of embodiments of the present invention will be described below, but it goes without saying that the embodiments to which the present invention can be applied are not limited to the following embodiments.

[0047] Figure 1 is a system configuration diagram showing an example of the configuration of a content provision system according to this embodiment. The content provision system 1000 is a computer system that provides content offering virtual experiences in a virtual space, such as video games, virtual activities, and shopping, to multiple registered users in parallel. In other words, the content provision system 1000 is able to constantly provide users with new virtual experiences.

[0048] The content provision system 1000 is a computer system that includes a server system 1100 and user terminals 1500 for each user, all connected via a network 9 for data communication.

[0049] Network 9 refers to a communication path capable of data transmission. In other words, Network 9 includes not only LANs (Local Area Networks) using dedicated lines (dedicated cables) or Ethernet (registered trademark) for direct connections, but also telephone networks, cable networks, and the Internet.

[0050] The server system 1100 is a computer system that performs various processes such as managing and controlling registered user information and various controls related to content provision (for example, controlling the progress of a game).

[0051] The server system 1100 has a control board 1150 mounted on the main unit 1101. The control board 1150 is equipped with various microprocessors such as a CPU (Central Processing Unit) 1151, a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor), various IC memories 1152 such as VRAM, RAM, and ROM, and a communication device 1153. Some or all of the functions mounted on the control board 1150 may be implemented using an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a SoC (System on a Chip).

[0052] In Figure 1, the server system 1100 is depicted as a single server device, but it may also be configured with multiple devices. For example, the server system 1100 may be configured with multiple servers, each handling a different function, and connected to each other via an internal bus or network 9 for data communication. Furthermore, the server system 1100 may include a database and online storage.

[0053] The server system 1100 has a content generation AI 3 that generates content based on given generation instruction information (e.g., prompts). The content generation AI 3 is implemented by a machine learning-based AI model on hardware employing a multi-core architecture (e.g., a group of GPUs and memory, a group of AI chips, etc.).

[0054] The content generation AI3 is not limited to being exclusively for server system 1100. For example, the content generation AI 3 may be implemented as an external device that can be connected to the server system 1100 via the network 9, and may also be an external system that can be used by customers other than the server system 1100.

[0055] The content generation AI3 may be implemented as a single multimodal generation AI model, or it may be configured as a group of AIs having one or more content element generation AIs 4 (4a, 4b, ...) for each type of content element. Hereafter, we will describe it as having multiple content element generation AIs 4.

[0056] Examples of content elements include, for example, if the content is a video game, the game scenario, game stage maps, characters appearing in the game, character forms (including the overall size and shape of individual parts, skin including color and pattern, etc.), character dialogue, character actions (motions), etc. Other examples include game background music, background images, background objects, etc.

[0057] For example, if the content is a virtual underwater dive experience (one of the virtual activities), the types of content elements may include the space (stage) to dive into, the types of creatures that appear in the water, and artificial objects that appear in the water (e.g., ships, submarines, shipwrecks, etc.).

[0058] There may be at least one content element generation AI4 for each type of content element. The types of content elements may be further subdivided, and a configuration may be provided for each sub-sub

[0059] User terminal 1500 serves as a Man-Machine Interface (MMIF) for user 2 to play content as a player. For example, if the content to be played is a game, the user terminal 1500 for user 2, who is the player, functions as a game playing terminal. Although only one user terminal 1500 is depicted in Figure 1, in actual operation, it is common for multiple user terminals 1500 to communicate and connect to the server system 1100 simultaneously.

[0060] The user terminal 1500 is a computer system that can connect to the network 9, such as a personal computer, smartphone, wearable computer, portable game console, home game console, or tablet computer.

[0061] The user terminal 1500 is a computer comprising an operation input device, an image display device, a communication device, and a control board 1550 that performs calculation processing. Examples of the operation input device include a touch panel 1506, a keyboard, a game controller, and a mouse. Examples of the image display device include a touch panel 1506, a head-mounted display, and a glasses-type display.

[0062] The control board 1550 is equipped with a CPU 1551, various microprocessors such as a GPU and DSP, various IC memories 1552 such as VRAM, RAM, and ROM, and a communication module 1553 that connects to the network 9. These elements mounted on the control board 1550 are electrically connected via bus circuits and the like, enabling data reading and writing, and signal transmission and reception. Part or all of the control board 1550 may be an ASIC, FPGA, or SoC.

[0063] The control board 1550 stores programs and various data necessary to realize the functions of the user terminal 1500 in the IC memory 1552. The user terminal 1500 realizes the functions of a play terminal by executing a predetermined application program (for example, a client program).

[0064] In the following sections, the content provision system 1000 will be explained using an example of providing video game content.

[0065] Figures 2 and 3 are diagrams illustrating the process of generating and providing new content in the content provision system 1000. As shown in Figure 2, user 2 can request the content provision system 1000 to generate and provide new content by performing a predetermined request operation on the user terminal 1500. In this case, the new content may be game content recognized as a game title to be played, or it may be content elements used in a pre-prepared game title (e.g., game stages, characters, etc.).

[0066] Figure 4 shows an example of a user interface related to request operations, specifically an example of a case where a "new game stage" is requested. Figure 5 shows an example of a user interface related to request operations, specifically an example of a case where a "new character" is requested. Note that the number and classification of content categories are not limited to the examples shown in these figures.

[0067] When the server system 1100 detects that a predetermined request function has been invoked on the user terminal 1500, it displays a category setting UI (User Interface) 10 on the user terminal 1500 that accepts the user selection operation for the category of newly generated content.

[0068] The options in Category Setting UI10 can be configured as needed. The "Game Title" option is for when you want to request game content with a new title that is different from the pre-selected game titles.

[0069] The "Game Scenario" option is for those who want a new game scenario that retains the world setting of a pre-defined game title. A single game scenario consists of one or more new game stages. If the game scenarios for pre-prepared game titles are considered original scenarios, then new game scenarios are equivalent to additional scenarios or bonus scenarios.

[0070] The "Game Stage" option is used when requesting additional game stages to be added to pre-defined game titles. These new game stages are positioned as "Extra Stages."

[0071] The options "Characters" and "Items" are for requesting the provision of characters or items to be added to pre-defined game titles.

[0072] When User 2 selects one of the multiple content categories shown in Category Setting UI 10, one or more detailed setting UIs 12 (12a, 12b, ...) are displayed.

[0073] The detailed settings UI12 displays detailed generation conditions and required quality for the content of the selected category for each setting item and accepts selection operations from User 2. The content and number of items in the detailed settings UI12 are predetermined by the content category. Each type of detailed settings UI12 corresponds to an item in the context of the generation instruction information (prompt) given to the generation AI, and the options in each detailed settings UI12 correspond to the specified values ​​in that context.

[0074] Returning to Figure 2, the server system 1100 generates origin generation instruction information 702, which describes the content of the request operation. The origin generation instruction information 702 includes the content category selected in the category setting UI 10 (what to generate), generation conditions (prerequisites) and required quality according to the result selected in the detailed setting UI 12, supplementary information, and so on.

[0075] The server system 1100 determines the types of content elements that make up the content requested by user 2. This is called "element decomposition". The type of content elements that make up each category of content are determined by the category. In other words, the type of content elements that need to be generated to provide the content required by the origin generation instruction information 702 is predetermined by the category.

[0076] The types of content elements obtained through element decomposition match the types of content elements that content generation AI3 can generate, or the types of content elements that each of the multiple content element generation AI4 can generate (see Figure 1).

[0077] For example, if the content category is "Game Title," then the content elements would include the game scenario, the game stages that appear in the scenario, the terrain and backgrounds that make up the game stages, and the characters that appear in the game stages. Other content elements would include sound effects, visual effects, and background music.

[0078] For example, if the content category is "Character," then content elements would include character models, skins, various motions, dialogue text, voiceovers for dialogue, equipment items, and initial settings for ability parameter values.

[0079] For example, content categories such as "BGM," "Sound Effects," and "Visual Effects" are considered standalone content elements.

[0080] For each type of content element determined by element decomposition, the server system 1100 determines which generation AI will be responsible for generating that type of content element from among the content element generation AIs 4 that are capable of generating that type of content element.

[0081] Figure 6 shows an example of element decomposition and AI assignment settings when the content category is "Game Stage". As a result of the element decomposition, the types of content elements are identified as "Map", "Background", "Character", "Visual Effects", "Sound Effects", "Item", etc. An AI is then assigned to each type of content element.

[0082] Of course, in cases where the content category indicated by origin generation instruction information 702 is a standalone content element, such as "sound effect," only one generation AI will be selected.

[0083] Returning to Figure 2, the server system 1100 creates element-specific generation instruction information 710 for each assigned generation AI to generate the content elements required for the content requested in the origin generation instruction information 702.

[0084] The element-specific generation instruction information 710 is created according to the generation instruction format of the assigned generation AI. Each element-specific generation instruction information 710 includes the type of content element to be generated (what to generate), generation conditions (prerequisites), required quality, and supplementary information. The supplementary information may include a unique element ID assigned at the time of generation, an origin ID indicating the origin generation instruction information 702 from which the element-specific generation instruction information 710 originates, and so on.

[0085] The server system 1100 sets the order in which to give element-specific generation instruction information 710 (instruction information) to the content element generation AI 4 based on the generation status of each responsible generation AI, and then gives the element-specific generation instruction information 710 in that order to acquire data for new content elements.

[0086] The "generation status of the assigned generation AI" refers to the processing load and workload status of the generation AI, and is represented by generation status information. This generation status information includes the processing load rate, memory usage, and number of pending executions for the generation AI.

[0087] The order in which the element-specific generation instruction information 710 is given to the responsible generation AI is such that the generation AI with the highest load and busiest status will be given priority. This increases the likelihood of synchronizing the timing of the generation and assembly of content elements obtained from element decomposition.

[0088] Furthermore, depending on the content category indicated by the origin generation instruction information 702, an appropriate instruction order for the types of content elements is predetermined. In such cases, once the generation of higher-level content element types is completed, element-specific generation instruction information 710 is provided to the AI ​​responsible for the lower-level content element types. At that time, it is also possible to correct the element-specific generation instruction information 710 to match the content of the higher-level content element before providing it to the responsible generation AI.

[0089] For example, in the content category "Game Stage," the content elements include a map and the number of enemy characters. Once the map that achieves the specified difficulty level is determined, the number of types of enemy characters to appear in that map, the abilities of the enemy characters (for example, the ratio of their abilities to those of the player character), and so on are determined.

[0090] If maps and enemy characters are generated simultaneously, and the generated map content does not match the content of the concurrently generated enemy characters, the enemy characters will need to be regenerated. For content categories where the settings of some content elements affect the settings of other content elements, this unnecessary work can be avoided by pre-determining the generation order for each type of content element.

[0091] When the content elements of the types obtained through element decomposition are generated and assembled, one candidate product 12 is prepared. The candidate product 12 is a candidate for the content that will ultimately be presented to the user 2 in response to the origin generation instruction information 702. The types and number of content elements contained in one candidate product 12 will differ depending on the content category indicated by the origin generation instruction information 702 and the numerical conditions specified in the generation conditions.

[0092] Incidentally, focusing on the number of generation cycles, candidate product 12 in Figure 2 corresponds to the "initial content" generated on the first attempt.

[0093] Moving to Figure 3, the server system 1100 determines whether the initial content, candidate product 12, is suitable as the final content to be output to user 2, and if it is, provides it to user 2.

[0094] The server system 1100 is involved in the evaluation of candidate products and includes a virtual execution unit 230 and a feedback AI 50.

[0095] The virtual execution unit 230 provides a simulation environment for virtually executing the content, which is the candidate product 12. For example, this can be realized by virtual machine technology that virtually implements the computational processing functions of the user terminal 1500. The virtual execution unit 230 then virtually executes the game content with the computer acting as the player, and outputs various game data describing what happened during execution as a game log.

[0096] The feedback AI 50 is implemented using a machine learning-based AI model that functions as an evaluation criteria generation unit 232, an evaluation unit 234, and a feedback information generation unit 236.

[0097] The evaluation criteria generation unit 232 generates evaluation criteria for evaluating candidate products based on the origin generation instruction information 702. The "evaluation criteria" are standards for each analysis item used to determine the extent to which the content of the origin generation instruction information 702 is satisfied, based on a given analysis of the content of the origin generation instruction information 702. The evaluation criteria generation unit 232 sets the "requirements" that the candidate product 12 must satisfy in order to pass as a product having the specifications and content required by the origin generation instruction information 702, and generates "evaluation criteria" for evaluating whether the requirements are met.

[0098] For candidate product 12 to be recognized as a product to be provided to user 2, it must meet several requirements. In other words, the acceptance criteria for candidate product 12 are described by one or more requirements.

[0099] "Requirements" are the conditions that break down the content specifications and content requested in the origin generation instruction information 702 into individual items. Therefore, requirements can be said to be intermediate data set based on the type of content category, the analysis items (context items) of the origin generation instruction information 702, and their specified values. What kind of requirements are set will differ depending on the content of the training dataset in the machine learning of the feedback AI 50, that is, the outcome of the machine learning.

[0100] A single requirement is described by one or more evaluation results based on the evaluation criteria. The "evaluation criteria" define the types of content elements to be evaluated and are described as thresholds or ranges that must be met in relation to the specifications and content of those content elements estimated from the origin generation instruction information 702. Depending on the corresponding requirements, the evaluation criteria may be described as the types of data, thresholds or ranges for values ​​that must be met for each item of the game log generated by the virtual execution unit 230.

[0101] The evaluation unit 234 evaluates the candidate product 12 based on the evaluation criteria and controls the output of the candidate product 12 that meets the given passing conditions described by multiple requirements as a presented product, i.e., the content that will ultimately be presented and provided to the user 2.

[0102] Specifically, the evaluation unit 234 has an element-specific evaluation unit 235 corresponding to the type of content element, and each content element of the candidate product 12 to be evaluated is evaluated by the element-specific evaluation unit 235 corresponding to its type.

[0103] The evaluation unit 234 summarizes all evaluation results for the candidate product 12 and its content elements to determine whether the candidate product 12 is pass or fail. It determines the pass or fail status of each content element in light of the evaluation criteria, and if all (or a predetermined percentage or more, for example, 80% or more) of the content elements pass, it determines that the requirement has been met. If all (or a predetermined percentage or more) of the requirements are met, the candidate product 12 may be judged as passable.

[0104] Figures 7 through 9 show examples of requirements and evaluation criteria. In the example in Figure 7, the content category indicated by the origin generation instruction information 702 is "Game Title". This category is selected when User 2 requests a game title other than those pre-prepared in the content provision system 1000. Furthermore, the "Image Quality" as a generation condition for the content category "Game Title" is "High Quality". In this case, the requirement is "Image Quality = High Quality".

[0105] To determine whether the requirements are met, the generated content elements are evaluated against the evaluation criteria. As shown in Figure 7, the content element types involved in the requirement "Image Quality = High Quality" include "Characters," "Items," and "Background Objects."

[0106] The evaluation criteria generation unit 232 generates evaluation criteria related to the requirement for each type of content element involved in the requirement. Figure 7 shows an example in which evaluation criteria for the number of polygons constituting the character model and evaluation criteria for the number of colors used in the character's color scheme are generated for the content element "character".

[0107] The evaluation criteria generation unit 232 generates a series of graded criteria for determining which of the following image quality levels—"high quality," "standard quality," or "low quality"—the polygon count falls under (the degree of satisfaction). Specifically, the evaluation criteria for the polygon count are the threshold or range of the polygon count.

[0108] The evaluation unit 234, in relation to the polygon count evaluation criteria, refers to the character's model data (content element data) and determines whether the polygon count falls within the range of low quality, standard quality, or high quality. If it falls into the high quality category, the character is judged to have passed the polygon count evaluation criteria.

[0109] Furthermore, the evaluation criteria generation unit 232 generates a series of graded criteria for determining which of the following image quality levels—"high quality," "standard quality," or "low quality"—the image meets (the degree of satisfaction) as an evaluation criterion for the number of colors. Specifically, the evaluation criteria for the number of colors are generated using thresholds and ranges for the number of colors used in the character's color scheme.

[0110] The evaluation unit 234 refers to the character's skin data and determines whether the number of colors used falls within the range of low quality, standard quality, or high quality. If it falls into the high quality category, the character is judged to have passed the evaluation criteria for the number of colors.

[0111] The evaluation unit 234 then determines that the candidate product 12 has passed the requirements if it has passed all requirements related to the analysis item "image quality" (one requirement in Figure 7) and all evaluation criteria for each requirement (including the two evaluation criteria illustrated in the example in Figure 7).

[0112] Furthermore, the character evaluation criteria related to the "image quality = high quality" requirement may include evaluation criteria other than polygon count and color count. For example, evaluation criteria such as the setting of character ability values ​​and the number of character motion variations may be included.

[0113] In the example in Figure 8, the content category indicated by the origin generation instruction information 702 is "Game Title," and the context specifying the design style (theme) as a generation condition is "Solemn." In this case, the requirement becomes "Design Style = Solemn."

[0114] To determine whether a requirement is met, the generated content elements are evaluated against the evaluation criteria. In Figure 8, the requirement "Design Style = Solemn" includes content elements such as "Characters," "Items," and "Background Objects."

[0115] The evaluation criteria generation unit 232 generates evaluation criteria related to the requirement for each type of content element involved in the requirement. Figure 8 shows an example where, for the content element "character," evaluation criteria for the color scheme of the character model and evaluation criteria for the form of the character model are generated. Of course, other evaluation criteria can also be set as appropriate.

[0116] The evaluation criteria generation unit 232 generates multiple color palettes based on the differences in the impression they give to the viewer, as evaluation criteria for color.

[0117] The evaluation unit 234 determines the conformance rate for each color palette used in the color scheme (skin color scheme) of the character being evaluated. If the color palette with the highest conformance rate is the "heavy" color palette, and the conformance rate is above a predetermined standard value, the evaluation unit 234 determines that the character has passed the evaluation criteria for color combination.

[0118] Furthermore, the evaluation criteria generation unit 232 generates multiple ranges of dimensional ratios for each part of the form, such as the head, chest, abdomen, waist, upper arms, lower arms, hands, thighs, lower legs, and feet, as evaluation criteria for the form, for example, if the character is humanoid, to represent the differences in the impression the viewer gets from the form. The ranges of dimensional ratios used as evaluation criteria are generated to correspond to contextual specified values ​​(for example, "heavy," "bright and vibrant," etc.).

[0119] The evaluation unit 234 determines the dimensional ratio of each part of the character being evaluated and calculates the conformance rate for each range of dimensional ratios. If the conformance rate is highest for the range of dimensional ratios deemed "imposing" in light of the evaluation criteria, and the degree of conformance is above a predetermined standard value, the evaluation unit 234 determines that the character has passed the evaluation criteria for form.

[0120] The evaluation unit 234 then determines that the character being evaluated has passed the requirements if all evaluation criteria related to the character being evaluated, such as the evaluation criteria for color combinations and the evaluation criteria for form, are met.

[0121] The evaluation unit 234 determines that if all content element types involved in the requirement "Design style = solemn" in Figure 8 meet the requirement, the candidate product 12 will be deemed to meet that requirement.

[0122] In the example in Figure 9, the content category indicated by the origin generation instruction information 702 is "Game Stage," and the generation conditions are type "Labyrinth" and game difficulty with the context "HARD." The example shows that four requirements were generated for this case: "Difficulty of labyrinth game map = HARD," "Enemy character appearance locations = Many," "Enemy character ability ratio (compared to player character) = 5 times or more," and "Game log play time (average time) is within about 1 hour."

[0123] The evaluation criteria generation unit 232 generates thresholds and ranges for the degree of mazeness for each difficulty level as evaluation criteria corresponding to the first requirement of "difficulty level of the labyrinth game map = HARD". The degree of mazeness can be calculated using a function that has a positive correlation with variables such as the number of T-junctions in the entire map, the number of corners in the entire map, and the number of corners in the correct route.

[0124] The evaluation unit 234 determines the degree of maze in the generated game map, and if the degree of maze reaches the range of maze equivalent to a predetermined HARD difficulty level, it determines that the evaluation criterion has been passed.

[0125] Furthermore, the evaluation criteria generation unit 232 generates thresholds and ranges for the number of enemy character appearance locations set in the labyrinth game map, categorized by difficulty level, as evaluation criteria corresponding to the second requirement, "Enemy character appearance locations = many".

[0126] The evaluation unit 234 refers to the settings of the game map generated as a content element and determines that the evaluation criterion has been passed if the number of enemy character spawn locations reaches a range equivalent to a predetermined HARD difficulty level.

[0127] Furthermore, the evaluation criteria generation unit 232 generates thresholds and ranges for the abilities (e.g., level values) of enemy characters and player characters, according to the difficulty level, as evaluation criteria corresponding to the second requirement of "enemy character ability ratio = 5 times or more".

[0128] The evaluation unit 234 compares the ability parameter values ​​of the enemy character generated as a content element with the ability parameter values ​​of the player character, and if they reach a predetermined range equivalent to HARD, it determines that the evaluation criterion has been passed.

[0129] Furthermore, the evaluation criteria generation unit 232 generates thresholds and ranges for play time, categorized by difficulty level, as evaluation criteria corresponding to the requirement that "play time in the game log = 1 hour or more".

[0130] The evaluation unit 234 refers to the game log obtained as a result of the virtual execution, and if the play time reaches the predetermined HARD certification threshold, it determines that the evaluation criterion has been passed.

[0131] Then, the evaluation unit 234 determines that if the candidate product 12 passes all (or a predetermined percentage or more) of the evaluation criteria for each of the first to fourth requirements, then the candidate product 12 is deemed to have passed the analysis item.

[0132] Returning to Figure 3, the evaluation unit 234 determines that the candidate product 12 is acceptable as a provided product if it passes all analysis items related to the content category indicated by the origin generation instruction information 702. The evaluation unit 234 determines that the candidate product 12 is unacceptable as a provided product if it fails any analysis item (any requirement) related to the content category indicated by the origin generation instruction information 702.

[0133] If the candidate product 12 is unsatisfactory, the feedback information generation unit 236 generates feedback information 736 based on the evaluation result from the evaluation unit 234.

[0134] "Feedback information" is information used to change an evaluation result that does not meet the passing criteria to meet the passing criteria, and is information used to change the requirements that the evaluation result did not meet among the requirements that describe the passing criteria.

[0135] For example, in the example in Figure 7, suppose the evaluation criteria for the content element "character" are not met. The feedback information generation unit 236 generates information (extended prompt / additional context) as feedback information 736, which adds a first range of polygon counts considered equivalent to high quality as a prerequisite during generation.

[0136] For example, suppose in the example in Figure 8, the evaluation criteria for the color combination related to the content element "character" are not met. The feedback information generation unit 236 generates feedback information 736, which is an additional information that is assumed at the time of generation to use a color palette that satisfies the requirements.

[0137] Similarly, in the example in Figure 8, suppose the content element "character" does not meet the evaluation criteria for its form. The feedback information generation unit 236 generates feedback information 736, which is an additional information that is assumed during generation to be that the dimensional ratio of each part falls within the evaluation criteria that conform to the context-specified value.

[0138] For example, in the example in Figure 9, suppose that "enemy character appearance location = many" falls short of the range indicated by the evaluation criteria. The feedback information generation unit 236 generates information as feedback information 736, which adds the range equivalent to HARD as a prerequisite during generation.

[0139] Furthermore, the feedback information 736 may also include data on the content elements that were evaluated this time (content elements that have been generated in the past) and element-specific generation instruction information 710 (past instruction information) that was given to the content element generation AI4 for the generation of the content elements in question.

[0140] For example, in the example in Figure 8, suppose the content element "character" passes the evaluation criteria for form but fails the evaluation criteria for color combination. The feedback information generation unit 236 generates feedback information 736, which is based on the premise that the character model data (the data of the passed form) will be colored, and that a color palette that conforms to the evaluation criteria will be used.

[0141] Returning to Figure 3, when the server system 1100 generates feedback information 736, it causes the unsuccessful content element to be recreated. That is, the server system 1100 adds the feedback information 736 to the element-specific generation instruction information 710 previously given to the generation AI responsible for that content element. If the element-specific generation instruction information 710 created for the initial generation is called the "initial prompt," then the feedback information 736 corresponds to the "extended prompt" that is added to the initial prompt and given to the content element generation AI 4 during the next generation.

[0142] Furthermore, the regeneration of unsuccessful content elements may be performed by a different content element generation AI 4 than the one used previously. In other words, a different AI may be selected from the assigned generation AIs. In this case, the element-specific generation instruction information 710 given to the newly selected AI will have the feedback information 736 added to it from the beginning (element-specific generation instruction information 710b).

[0143] The AI ​​model for Feedback AI50 can be built, for example, using supervised machine learning (deep learning). For example, if we were to build Feedback AI50 as a standalone AI model, as shown in Figure 10, we would train it to take input such as origin generation instruction information, candidate product data, virtual execution game logs, and annotations, and output pass / fail judgments for candidate products, evaluation results for each content element, and feedback information. Note that a training dataset with "none" virtual execution game logs may also be prepared.

[0144] Feedback AI50 may be provided for each type of content element. One feedback AI 50 is constructed by training it to take origin generation instruction information, element-specific instruction information, content element data, and annotations as input, and output evaluation results and feedback information for each content element, as shown in Figure 11. Note that a training dataset with "no" element-specific instruction information may also be used. Alternatively, a training dataset that includes virtual execution game logs as input may also be used.

[0145] As shown in Figure 12, the feedback AI 50 may be constructed using an AI 52 for generating evaluation criteria for each content element type, an evaluation AI 54, and a feedback information generation AI 56. In that case, each of these would be created using supervised machine learning (deep learning).

[0146] The evaluation criteria generation AI 52 corresponds to the evaluation criteria generation unit 232. The evaluation criteria generation AI 52 is created using supervised machine learning (deep learning) to take, for example, origin generation instruction information, element-specific generation instruction information, and annotations as input and output element-specific evaluation criteria.

[0147] The evaluation AI 54 corresponds to the evaluation unit 234 and the element-specific evaluation unit 235. The evaluation AI 54 is created using supervised machine learning (deep learning) to take, for example, evaluation criteria, content element data, and annotations for corresponding content elements as input and output evaluation results. The input may also include a training dataset containing virtual execution game logs.

[0148] The feedback information generation AI 56 corresponds to the feedback information generation unit 236. The feedback information generation AI 56 is created using supervised machine learning (deep learning) to take evaluation criteria, content element data, evaluation results, and annotations as input and output feedback information.

[0149] Furthermore, the feedback AI 50 in this configuration is built to be able to work in conjunction with a search-extended system that incorporates it. That is, during operation, the evaluation AI 54 can search and refer to the evaluation criteria generated by the evaluation criteria generation AI 52. In addition, the feedback information generation AI 56 can search and refer to the evaluation criteria generated by the evaluation criteria generation AI 52 and the evaluation results generated by the evaluation AI 54.

[0150] Figure 13 shows an example of programs and data stored by the server system 1100. The server system 1100 stores the server program 501, the distribution client program 503, the content initial setup data 510, the trained generation AI model 512, and the element decomposition pattern data 514 in the IC memory 1152.

[0151] The server system 1100 also stores instruction sequence definition data 520, user registration data 600, generation status information 602, content play information 650, generation management data 700, and the current date and time 900. Of course, other data may also be stored as appropriate.

[0152] The server system 1100 performs server processing 200s functions as a server processing unit 200s, as shown in Figure 14, by executing and processing the server program 501 on the CPU 1151. The server program 501 may include a generation AI program 502 for realizing the functions of a content generation AI 3 and a content element generation AI 4. Alternatively, the generation AI program 502 may be stored separately.

[0153] The server processing unit 200s performs various controls from the generation of content using the AI ​​to its provision. Specifically, the server processing unit 200s includes a generation status information acquisition control unit 204, a generation instruction information reception unit 206, a responsible generation AI setting unit 208, and a candidate product acquisition control unit 212. The server processing unit 200s also includes a content generation unit 220, a virtual execution unit 230, an evaluation criteria generation unit 232, an evaluation unit 234, a feedback information generation unit 236, and a timing unit 280.

[0154] The generation status information acquisition control unit 204 controls the acquisition of generation status information for each of the content generation AI3 and content element generation AI4. "Generation status information" is information indicating processing load status and request congestion status. "Processing load status information" is, for example, CPU load rate and memory consumption rate (usage rate). "Information indicating congestion status" is, for example, the number of pending executions.

[0155] The generation instruction information receiving unit 206 receives the given generation instruction information. Specifically, the given generation instruction information is the origin generation instruction information 702 (see Figure 2). The generation instruction information receiving unit 206 controls the user terminal 1500 to display an input screen for receiving request operations (see Figure 4) and generates the origin generation instruction information 702 based on the result of the request operation.

[0156] Alternatively, the display control of the input screen and the generation of the origin generation instruction information 702 may be performed on the user terminal 1500 instead of the server system 1100. In that case, the generation instruction information receiving unit 206 will perform control to acquire the origin generation instruction information 702 from the user terminal 1500.

[0157] The AI ​​generation setting unit 208 determines the types of content elements required for the newly generated content, as indicated by the generation instruction information received by the generation instruction information receiving unit 206. Specifically, it performs element decomposition by referring to the element decomposition pattern data 514 (see Figure 13). Then, from among the content element generation AIs 4, it sets the AI ​​responsible for generation according to the type of content element obtained through element decomposition.

[0158] The candidate product acquisition control unit 212 controls the acquisition of N (N≧2) different candidate products by providing instruction information based on the generation instruction information received by the generation instruction information receiving unit 206 to a predetermined generation AI. Specifically, the candidate product acquisition control unit 212 creates element-specific generation instruction information 710 for the assigned generation AI from the origin generation instruction information 702 and sets the instruction order (see Figure 2). Then, it provides the element-specific generation instruction information 710 to each assigned generation AI according to the instruction order.

[0159] When setting the instruction order, the candidate product acquisition control unit 212 refers to the instruction order definition data 520 (see Figure 13) and, if there is definition data that matches the content category indicated by the origin generation instruction information 702, it adopts the instruction order defined by that data. If there is no matching definition data, it refers to the generation status information 602 (see Figure 13) of the assigned generation AI and sets the instruction order so that the assigned generation AI in a high-load state comes first, and then the assigned generation AI for content elements with large data sizes as content elements comes first.

[0160] The content generation unit 220 generates the content specified in the generation instruction information (prompt) based on the given generation instruction information, by referring to supplementary information to satisfy the specified generation conditions and required quality.

[0161] The content generation unit 220 is a content generation AI 3 (see Figure 1). The content generation unit 220 has multiple content element generation units 222 which are implemented by the content element generation AI 4.

[0162] However, neither the content generation AI3 of the content generation unit 220 nor the content element generation AI4 of the content element generation unit 222 are perfect. The generation AI may produce output that does not satisfy the content specified in the given generation instruction information, or output that includes elements that were not specified. Such phenomena are called "hallucination" or "convenient lies."

[0163] The evaluation criteria generation unit 232 (see Figure 3) generates evaluation criteria for determining which content elements of the candidate product 12 requested by the origin generation instruction information 702 are acceptable content elements that do not contain "hallucination" or "convenient lies".

[0164] The evaluation unit 234 evaluates the content elements and candidate products 12 in light of the generated evaluation criteria, identifies candidate products 12 that are acceptable as provided products, and outputs and presents them to the user 2.

[0165] For unacceptable content elements of the unacceptable candidate product 12, the feedback information generation unit 236 generates feedback information 736 (see Figure 3), and the candidate product acquisition control unit 212 controls the regeneration. Focusing on this regeneration process, the candidate product acquisition control unit 212 controls the acquisition of N (N≧2) different candidate products by providing instruction information based on the origin generation instruction information 702 to a predetermined generation AI.

[0166] The timing unit 280 uses the system clock to determine the current date and time (900) and various other timings.

[0167] Returning to Figure 13, the distribution client program 503 is the original client program provided to the user terminal 1500.

[0168] The content initial setup data 510 is prepared separately for each preset content that the content provision system 1000 has prepared in advance, and stores various initial setup data for that preset content. In this embodiment, since the content provided by the content provision system 1000 is a game, the content initial setup data 510 corresponds to game initial setup data.

[0169] The trained generative AI model 512 is provided for both the content element generation AI 4 (see Figure 1) and the feedback AI 50 (see Figure 3). One trained generative AI model 512 includes the generative AI type and AI model data. For the AI ​​model of the content element generation AI 4, information on the types of elements that can be generated is stored.

[0170] Element decomposition pattern data 514 is prepared for each piece of content containing multiple content elements, from the categories shown as options in the category setting UI 10 (see Figure 4). Each piece of element decomposition pattern data 514 stores the content category and a list of content element types obtained through element decomposition.

[0171] The instruction order definition data 520 defines the instruction order (see Figure 2) for instructing the content generation AI to generate content elements within a specific type of content category. A single instruction order definition data 520 stores the content category type as an application requirement for the definition data, and a list of instruction order by content element type.

[0172] User registration data 600 is prepared for each user who has completed the prescribed user registration procedure and stores various information (user information) associated with that user.

[0173] The generation status information 602 is prepared separately for content generation AI3 and content element generation AI4, and stores the generation status information of the respective generation AI. One generation status information 602 stores, for example, the generation AI ID, processing load rate, memory consumption rate, number of executions waiting, etc. The generation status information 602 is constantly updated by the generation status information acquisition control unit 204 to show the latest generation status.

[0174] Content play information 650 is prepared for each content play and stores various information related to that content play. One piece of content play information 650 stores, for example, the player account (the user account of the player user) and game progress information (content progress information). When the candidate product 12 passes and is provided to user 2 as a provided product, the data related to that provided product is added to the content play information 650 and becomes available for use.

[0175] The generated management data 700 is prepared for each request from a single user 2 and stores various data related to content generation. The generation management data 700 includes, for example, as shown in Figure 15, origin generation instruction information 702, a content element type list 706 which is the result of element decomposition, element-specific generation instruction information 710, responsible generation AI setting data 714, and instruction order setting data 716. The generation management data 700 also includes candidate product data 720 for each of the candidate products 12, a game log 730, evaluation criteria data 732, evaluation result data 734, and feedback information 736. Of course, other information may also be included as appropriate.

[0176] The instruction order setting data 716 stores the instruction order for each type of content element to which it is applied (see Figure 2).

[0177] The candidate product data 720 includes a candidate product ID 722, an origin ID 724 indicating which origin generation instruction information 702 it was generated based on, and one or more content element data 726. Of course, other information may also be included as appropriate.

[0178] Game log 730 stores the log ID, the execution target candidate product ID indicating the virtually executed candidate product 12, and the log body data in association with each other.

[0179] The evaluation criteria data 732 stores the evaluation criteria ID, the requirement ID of the requirement to which it belongs, the origin ID indicating which origin generation instruction information 702 the evaluation criteria pertains to, and the evaluation criteria body data (such as the range data exemplified in Figures 7 to 9).

[0180] The evaluation result data 734 stores the ID of the content element to be evaluated, the evaluation criterion ID of the evaluation criterion used, the evaluation result, and the pass / fail result.

[0181] Feedback information 736 stores a feedback ID, an applicable content element ID indicating which content element it relates to, and body data (corresponding to the contents of the extended prompt).

[0182] Figure 16 is a flowchart illustrating the flow of processing related to content generation and provision executed by the server system 1100.

[0183] The server system 1100 receives a request operation from user 2 (step S10) and creates origin generation instruction information 702 in response to the request operation (step S12).

[0184] Next, the server system 1100 decomposes the content category indicated by the origin generation instruction information 702 to obtain a list of the types of content elements that need to be generated (step S14). Then, the server system 1100 sets the responsible generation AI for each type of content element obtained by the decomposition (step S16).

[0185] Next, the server system 1100 creates element-specific generation instruction information 710 (step S30). Then, the server system 1100 sets the instruction order (step S32), and in the order of the instructions, provides the element-specific generation instruction information 710 to the responsible generation AI to give instructions for generating content elements and controls the acquisition of content elements (step S34).

[0186] When content elements are generated for all types of content elements obtained through element decomposition, candidate product 12 is ready. If the candidate product is a new game title, the server system 1100 virtually executes it and obtains a game log 730 (step S36). Then, the server system 1100 sets requirements based on the origin generation instruction information 702 and creates evaluation criterion data 732 (step S48), and evaluates each content element based on the evaluation criterion data 732 (step S50).

[0187] The server system 1100 determines whether a candidate product is acceptable or unacceptable based on the evaluation results for each required content element (step S52). If a candidate product fails to meet the requirements as a presented product (NO in step S52), the server system 1100 creates feedback information 736 (step S54). Then, the server system 1100 adds the feedback information 736 to the element-specific generation instruction information 710 given to the generation AI responsible for the content element that failed to meet the evaluation criteria, and updates it (step S56).

[0188] The server system 1100 provides the updated element-specific generation instruction information 710 to the generation AI responsible for the content element that failed to meet the evaluation criteria, causing the content element to be regenerated (step S58). Accordingly, the server system 1100 updates the candidate product data 720 of the candidate product 12 prepared in step S34 with the regenerated content elements. Alternatively, the candidate product data 720 of the candidate product 12 prepared in step S34 may be duplicated as the base for the candidate product data 720 of the new candidate product 12, and the unsuccessful content elements may be updated with the regenerated content elements.

[0189] The server system 1100 returns to step S36 and evaluates the candidate product 12 with the regenerated content elements again (step S50). This process is repeated until the candidate product 12 passes.

[0190] If candidate product 12 is acceptable (YES in step S52), the server system 1100 presents and outputs candidate product 12 to user 2 as a presented product (step S120). If candidate product 12 is a game title, gameplay begins. If candidate product 12 is content that can be used in a pre-prepared game (game stage, character, item, music, etc.), it is made available for use in that game.

[0191] In summary, according to this embodiment, when generating content using a generation AI and acquiring that content, it becomes possible to acquire content with the specifications and content desired by the user. The generation AI cannot always produce the perfect generation that the user desires. It may generate or output products that lack the specifications or content requested in the origin generation instruction information 702, or products that contain "hallucination" or "convenient lies."

[0192] According to this embodiment, such products can be automatically identified and excluded before being output and presented to the user, and the products can be regenerated to present to user 2 only those products that meet the specifications and content obtained from the origin generation instruction information 702. User 2 does not need to evaluate the candidate products 12 themselves and repeat the request operation, making it highly convenient.

[0193] [Variation] Although examples of embodiments to which the present invention is applied have been described above, the forms to which the present invention can be applied are not limited to the above forms, and components can be added, omitted, or modified as appropriate.

[0194] (Variation 1) In the above embodiment, the content provision system 1000 also generated new content and provided an environment for playing the provided content, but is not limited to this. For example, the content provision system 1000 could be positioned as a tool for game creators to generate content data, and its function could be limited to generating new content and outputting data of the provided product.

[0195] (Variation 2) For example, although the content provision system 1000 was exemplified as a client-server type, multiple user terminals 1500 may be implemented using a P2P (Peer to Peer) architecture. In this case, programs and data corresponding to the functional division are stored in the user terminals 1500, and the functions corresponding to the server processing unit 200s in the above embodiment are distributed and implemented by the user terminals 1500, which act as P2P nodes. The same effects as in the above embodiment can be obtained with this configuration as well.

[0196] (Variation 3) Furthermore, in the above embodiment, the content provision system 1000 may be implemented not as a client-server type, but as a single computer system that was the user terminal 1500 in the above embodiment.

[0197] Specifically, all the data that the server system 1100 in the above embodiment is supposed to store is stored on the user terminal 1500 (see Figure 13). However, instead of the server program 501 and the distribution client program 503, a content provision program is provided as an application program for the user terminal 1500.

[0198] In the above embodiment, the content provision program implements all of the functional units of the server system 1100 (see Figure 14) on the user terminal 1500. In this modified example, the content provision program is executed on the user terminal 1500. The processing flow in the above embodiment (see Figure 16) can be interpreted by replacing the execution entity from the server system 1100 to the user terminal 1500.

[0199] (Modification #4) Furthermore, the user interface for request operations may not be an icon-based system where users select from pre-defined options, as shown in Figures 4 and 5, but rather a system where user 2 inputs text using natural language.

[0200] Specifically, instead of displaying the category setting UI10 or detailed setting UI12 (see Figure 4) on the user terminal 1500, the server system 1100 displays a request setting screen W10 that allows input using natural language text, such as shown in Figure 17.

[0201] The server system 1100 may use a large-scale language model type AI to determine the context items and context specification values ​​of the origin generation instruction information 702 from the text entered on the request setting screen W10, and generate the origin generation instruction information 702.

[0202] (Variation 5) In the above embodiment, the server system 1100 is configured to generate one candidate product 12 per generation cycle, but it is also possible to generate multiple candidate products 12 at once.

[0203] (Variation #6) In the above embodiment, the generation of content elements essentially involved the simultaneous generation of multiple content elements, although the timing of providing the element-specific generation instruction information 710 to the content element generation AI 4, which became the responsible generation AI, may be adjusted based on the order of the instructions. However, it is also possible to perform serial generation, where a certain type of content element is generated and evaluated, and if that content element passes, the next type of content element is generated.

[0204] For example, if the content category is "Game Scenario," then the game scenario here refers to a description of how the game progresses and which game stages it branches into. Depending on the specifications and content of the newly generated game scenario, the number of game stages to be generated and the content of those game stages will change. And depending on the content of the game stages, the number and type of characters and items that need to be generated for those game stages will change.

[0205] In other words, in this case, focusing on the inclusion relationships of content elements, the higher-level content element "game scenario" contains the intermediate-level content element "game stage," and the specifications and content required of the intermediate-level content element change depending on the specifications and content of the higher-level content element. Furthermore, the intermediate-level content element "game stage" contains the lower-level content elements "character" and "item," and the specifications and content required of the lower-level content elements change depending on the specifications and content of the intermediate-level content element.

[0206] In this case, if the generation is carried out simultaneously as in the above embodiment, there remains a possibility that the resulting product will use game stages that do not conform to the specifications and content of the game scenario, and use characters and items that do not conform to the use and content of the game stages.

[0207] To address this concern, the server system 1100 may perform serial generation. Specifically, the server system 1100 pre-stores stage generation definition data 530, as shown in Figure 18. The stage generation definition data 530 stores a specific category type indicating a specific type of content category to which the definition data applies, and a list of content element types in the order of stage generation, in association with each other.

[0208] The tiered generation order content element type list is a list that describes the types of content elements obtained by decomposing a content category of a specific category type, arranged in order from the highest to the lowest level of content inclusion relationships. In this list, the highest-level content is ranked 1st in generation priority, the middle-level content 2nd, and the lowest-level content 3rd. The final ranking changes depending on the number of inclusion relationships.

[0209] Figure 19 is a flowchart illustrating the processing flow in this modified example. It is basically the same as that of the above embodiment, but after step S16, the server system 1100 determines whether the content category indicated by the origin generation instruction information 702 corresponds to a specific category type indicated by the stage generation definition data 530 (step S20).

[0210] If negative (NO in step S20), the process is the same as in the embodiment described above. If positive (YES in step S20), the server system 1100 executes the step generation process (step S22).

[0211] Figure 20 is a flowchart illustrating the flow of the step-by-step generation process. In the step generation process, the server system 1100 refers to the appropriate step generation definition data 530 (step S80) and initializes the execution generation order to "1" (step S82).

[0212] The server system 1100 then generates evaluation criteria for the content elements in the execution generation order (step S84). At this time, it sets requirements as appropriate. Furthermore, the evaluation criteria are set to conform to the specifications and content of the content element one level higher that contains the content element in the execution generation order.

[0213] The evaluation criteria may be generated by preparing a specific category evaluation criterion generation AI58 for each type of content element. As shown in Figure 21, one specific category evaluation criterion generation AI58 is trained to take origin generation instruction information, element-specific instruction information for the next higher level content element, and annotations as input and output evaluation criteria. Note that a training dataset containing data for the next higher level content element may be used as input.

[0214] Returning to Figure 20, the server system 1100 then creates element-specific generation instruction information 710 for the AI ​​responsible for generating content elements in the execution generation order (step S86). At this time, the element-specific generation instruction information 710 is created so as to specify the evaluation criteria set in step S84 as the generation conditions or required quality.

[0215] The element-specific generation instruction information 710 may also be generated by preparing an element-specific generation instruction information generation AI 59 for each content element type and having it generate the information. As shown in Figure 22, the AI ​​59 for generating element-specific generation instructions is trained to take origin generation instruction information, evaluation criteria, and annotations as input and output element-specific generation instruction information 710. Alternatively, a training dataset containing data from the next higher-level content element may be used as input.

[0216] Returning to Figure 20, the server system 1100 then generates content elements in the order of execution generation (step S88) and evaluates them against the evaluation criteria obtained in step S84 (step S100).

[0217] If the evaluation result is unsatisfactory (NO in step S100), the server system 1100 generates feedback information 736 (step S102), and updates the element-specific generation instruction information 710 by adding the generated feedback information 736 (step S104). Then, it regenerates the content elements in the execution generation order (step S106).

[0218] If the evaluation result is a pass (YES in step S100), the server system 1100 determines whether the execution generation order was the lowest (step S110). Alternatively, it may determine whether all content elements passed.

[0219] Then, in the case of negation (NO in step S110), that is, if there are still content elements remaining that are to be generated, the server system 1100 increases the execution generation order by "1" (step S112) and returns to step S84. If the answer is affirmative (YES in step S110), the stepwise generation process ends and the system returns to the flowchart in Figure 20.

[0220] (Variation 7) Furthermore, as an alternative way to address the same concerns as in Modification 6, the system can determine whether a relatively lower-level content element (e.g., an enemy character) generated earlier is compatible with a relatively higher-level content element (e.g., a map) generated later, based on the results of essentially simultaneous processing. If incompatibility is determined, subsequent processing from that point onward can be stopped.

[0221] The determination of suitability, as referred to here, may be implemented, for example, as follows: The AI ​​responsible for generating the content element to be evaluated is given the content element to be evaluated, along with predetermined evaluation instruction information that instructs it to classify the content element and provide an impression that the content element gives to the viewer. The degree of match and similarity of this response to the classification and impression in the generation conditions and prerequisites of the element-specific generation instruction information for relatively higher-ranking content elements is then determined, and if these meet predetermined criteria, it is determined to be suitable.

[0222] For example, if the design style classification of an enemy character is the same as or similar to the design style classification of a map, it is determined to be a match; however, if the design styles are different, it is determined to be a match.

[0223] If the result is unsuitable, the system will issue a stop command to the AI ​​responsible for generation, which has not yet completed the generation process, thereby halting subsequent processing.

[0224] This configuration offers the advantage of faster processing when a relatively lower-level content element, generated earlier, satisfies the generation conditions and prerequisites for a relatively higher-level content element, which is generated later.

[0225] (Variation #8) In the above embodiment, the evaluation criteria generation unit 232 generates the evaluation criteria, but the embodiment is not limited to this. The evaluation criteria generation unit 232 may be replaced by a vector search engine that searches for evaluation criteria pre-stored in a vector database that the server system 1100 can connect to, according to the content element to be evaluated.

[0226] Specifically, for example, the content generation AI3 and content element generation AI4 are trained to accept generation instruction information written in natural language, and the element-specific generation instruction information 710 is written in natural language. Furthermore, evaluation criterion definition text written in natural language is stored in a vector database that the server system 1100 can connect to. The evaluation criterion definition text may be written in the following format, for example, "The evaluation criteria when (content element type) is generated to satisfy (content of generation conditions / prerequisites) are as follows: (a sentence explaining one or more evaluation criteria)."

[0227] The vector search engine vectorizes the text text of the generation conditions / prerequisites for each element of the content to be evaluated, performs a nearest neighbor search on the evaluation criteria definition text in the vector database, and uses this as the evaluation criterion for the content to be evaluated.

[0228] (Modification #9) The functionality of the feedback AI 50 may be implemented using algorithms rather than AI. In that case, the requirements and evaluation criteria shown in Figures 7 to 9 are prepared in advance as evaluation criterion definition data, and the server system 1100 stores this data beforehand. Specifically, the evaluation criterion definition data is prepared according to the application requirements, with the "contents of the origin generation instruction information" defined as the application requirements for the definition data in Figures 7 to 9. The system then stores the data for the corresponding "requirements and corresponding evaluation criteria." [Explanation of symbols]

[0229] 2…User 3… Content generation AI 4… AI for generating content elements 12...Candidate product 50…Feedback AI 52… AI for generating evaluation criteria 54…Evaluation AI 56…Feedback Information Generation AI 200s... Server Processing Unit 206...Generation Instruction Information Reception Unit 208... AI generation setting department 212... Candidate Product Acquisition Control Unit 220...Content Generation Unit 222...Content element generation unit 232...Evaluation Criteria Generation Unit 234…Evaluation Department 235... Element-Specific Evaluation Department 236...Feedback Information Generation Unit 501…Server program 512... Pre-trained generative AI models 514... Element decomposition pattern data 520... Instruction sequence definition data 700…Generated and managed data 702…Origin generation instruction information 706…List of Content Element Types 710…Element-specific generation instruction information 720… Candidate product data 726…Content element data 732…Evaluation criteria data 734…Evaluation result data 736…Feedback Information 1000... Content delivery system 1100…Server System 1500... User terminal

Claims

1. A means for receiving generation instruction information that receives given generation instruction information, A candidate product acquisition control means that performs control to acquire N different candidate products (N≧2) by providing instruction information based on the aforementioned generation instruction information to a predetermined generation unit, An evaluation criteria generation means that generates evaluation criteria for evaluating the candidate product based on the production instruction information, An evaluation means for evaluating the candidate product based on the evaluation criteria, A computer system equipped with the following features.

2. The evaluation means controls the output of candidate products as presented products if the evaluation result satisfies the given passing conditions. The computer system according to claim 1.

3. The candidate product acquisition control means repeatedly modifies the instruction information and provides it to the generation unit to perform control for acquiring N candidate products. The computer system according to claim 1.

4. The candidate product acquisition control means provides the generation unit with feedback information based on the evaluation results of the evaluation means for past products which are candidate products that have been generated in the past, and new instruction information based on said past products, and repeatedly performs control to acquire new candidate products, thereby performing control to acquire N candidate products. The computer system according to claim 1.

5. The candidate product acquisition control means performs the repetition a given number of times. The computer system according to claim 4.

6. The aforementioned feedback information is information for making an evaluation result that does not meet the given passing conditions based on the evaluation result meet the passing conditions. The computer system according to claim 4.

7. The aforementioned passing conditions include multiple requirements, The aforementioned feedback information is information that helps to satisfy the requirements among the requirements included in the passing conditions that the evaluation result has not yet met. The computer system according to claim 6.

8. The aforementioned evaluation criteria are criteria for determining the extent to which the content of the generation instruction information is satisfied. The computer system according to claim 1.

9. The evaluation criteria generation means analyzes the content of the generation instruction information for each given analysis item and generates criteria for each analysis item to determine the degree to which the content of the generation instruction information is satisfied. The given passing conditions based on the evaluation results include the requirements based on the criteria for each of the analysis items, The aforementioned feedback information is information relating to the requirements included in the passing conditions that the evaluation result did not meet. The computer system according to claim 4.

10. The generation unit has a content element-specific generation unit that generates content elements for content that the user can play, The generation instruction information receiving means receives the generation instruction information as an instruction to generate the content or one or more of the content elements. The computer system according to claim 1.

11. The aforementioned content is game content. The computer system according to claim 10.

12. The candidate product acquisition control means is Based on the generation instruction information, the element-specific generation unit is selected to generate the candidate product, The selected element-specific generation unit is given the instruction information based on the generation instruction information, Execute The computer system according to claim 10.

13. The candidate product acquisition control means creates the instruction information to be given to the selected element-specific generation unit based on the generation instruction information and the selected element-specific generation unit. The computer system according to claim 12.

14. The candidate product acquisition control means performs control to variably determine the order in which the element-specific generation units that provide the instruction information are selected when a plurality of element-specific generation units are selected. The computer system according to claim 12.

15. The evaluation means controls the timing of evaluation of the generated candidate products based on the generation status of N candidate products. The computer system according to claim 3.

16. The evaluation means controls the timing of the evaluation of the generated candidate product based on the type of the candidate product. The computer system according to claim 3.

17. Computer systems, A means for receiving generation instruction information that receives a given generation instruction information, Candidate product acquisition control means that performs control to acquire N different candidate products (N≧2) by providing instruction information based on the aforementioned generation instruction information to a predetermined generation unit. Evaluation criteria generation means that generates evaluation criteria for evaluating the candidate product based on the production instruction information, Evaluation means for evaluating the candidate product based on the evaluation criteria, A program designed to function as such.

Images