Virtual scene generation method and system

Abstract

The present disclosure relates to the fields of computer and artificial intelligence, and provides a method and system for generating a virtual scene, wherein the method comprises: a planning agent receives a scene generation instruction input by a user; the planning agent selects a programmatic content generation plug-in for generating the target scene from the preset resource library according to the plug-in annotation information in the preset resource library, and generates a task plan for generating the target scene; an execution agent uses the programmatic content generation plug-in selected from the preset resource library to execute the task plan to generate the target scene. The present disclosure can solve the problem of difficulty in improving the work efficiency of generating virtual scenes, and can automatically generate the target scene without the user having to deeply participate in the scene generation process such as the selection, application, and adjustment of the PCG plug-in, thereby improving work efficiency.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer science and artificial intelligence, and in particular to a virtual scene generation method and system. Background Art

[0002] Procedural Content Generation (PCG) is an algorithm- and data-driven technology used to automatically generate virtual environments and scenes. In the field of computer graphics and virtual reality, PCG has been widely used to generate various complex virtual worlds, such as cities, terrains, and vegetation. The advantage of this technology is that it can automatically create large-scale and highly realistic virtual scenes based on preset rules, parameters, or input data, greatly saving labor costs and time.

[0003] However, with the increasing application of virtual scenes, the demand for virtual scenes in various fields is increasing, and the requirements for scene generation speed and efficiency are also increasing. Even if PCG technology is used, it is difficult to further improve work efficiency due to the need for manual participation in the scene generation process such as PCG plug-in selection, application, and adjustment. It is impossible to meet the rapidly growing demand for scene generation. Summary of the invention

[0004] The present disclosure provides a virtual scene generation method and system to at least solve the problem in the related art that it is difficult to improve the work efficiency of generating virtual scenes. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a virtual scene generation method is provided, which is executed in a virtual scene generation system, and the virtual scene generation system includes a planning agent and an execution agent, wherein the virtual scene generation method includes: the planning agent receives a scene generation instruction input by a user, wherein the scene generation instruction includes a description of a target scene to be generated; the planning agent selects a programmatic content generation plug-in for generating the target scene from a preset resource library according to plug-in annotation information in the preset resource library, and generates a task plan for generating the target scene, wherein the preset resource library includes a plurality of the programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information is used to describe the corresponding programmatic content generation plug-in; the execution agent uses the programmatic content generation plug-in selected from the preset resource library to execute the task plan to generate the target scene.

[0006] Optionally, the execution agent receives task information of the current subtask from the planning agent, wherein the task information indicates a target programmatic content generation plug-in that matches the current subtask; the execution agent selects the target programmatic content generation plug-in from the preset resource library based on the task information received from the planning agent, and executes the current subtask.

[0007] Optionally, the virtual scene generation system also includes an assistant agent, wherein, before the planning agent sends the task information of the current subtask to the execution agent, the virtual scene generation method also includes: the assistant agent verifies whether the current task plan is correct, and sends the verification result to the planning agent, wherein the verification of whether the current task plan is correct includes: verifying whether the execution conditions of the current subtask are met and / or verifying whether the timing of executing the current subtask is correct, wherein the verification of whether the execution conditions of the current subtask are met includes verifying whether the scene contents match and / or verifying whether the task objectives of the current subtask comply with the description in the scene generation instructions; the planning agent responds to the verification result indicating that the current task plan is incorrect, corrects the task plan, and sends the corrected task plan to the assistant agent for re-verification; the planning agent responds to the verification result indicating that the current task plan is correct, determines the current subtask in the current task plan, so as to execute the step of sending the task information of the current subtask to the execution agent.

[0008] Optionally, the virtual scene generation system also includes an evaluation agent, wherein the virtual scene generation method also includes: the execution agent sends the execution result of each target programmatic content generation plug-in to the evaluation agent after executing the current subtask; the evaluation agent evaluates the execution result according to the code logic of the target programmatic content generation plug-in and the visual effect obtained based on the execution result, and feeds back the evaluation result to the planning agent; the planning agent responds to the evaluation result indicating that the execution result of the current subtask meets the expected requirements, and sends the task information of the next subtask to the execution agent to execute the next subtask; the planning agent responds to the evaluation result indicating that the execution result of the current subtask does not meet the expected requirements, and adjusts the parameters in the task information of the current subtask, and sends the adjusted task information of the current subtask to the execution agent to re-execute the current subtask.

[0009] Optionally, the virtual scene generation method also includes: the planning agent or the assistant agent displays the execution progress of the task plan in real time, wherein the display of the execution progress of the task plan includes: displaying the current task plan, displaying the task information of the current subtask or the next subtask, displaying the success or failure of the current subtask and / or displaying the task parameters required for the execution of the current subtask that require user input; the planning agent or the assistant agent adds the task parameters to the task information of the current subtask in response to receiving the task parameters input by the user.

[0010] Optionally, the virtual scene generation system also includes an annotation agent, which adds a programmatic content generation plug-in to the preset resource library in the following manner: obtaining structured packaging information of the programmatic content generation plug-in to be added, wherein the structured packaging information includes basic information of the programmatic content generation plug-in to be added, input and output quantities of the programmatic content generation plug-in to be added, and restriction information for using the programmatic content generation plug-in to be added; based on the structured packaging information, classifying the programmatic content generation plug-in to be added to add one or more classification tags to the programmatic content generation plug-in to be added; based on the classification tags and the structured packaging information, generating plug-in annotation information corresponding to the programmatic content generation plug-in to be added; adding the programmatic content generation plug-in to be added and the corresponding plug-in annotation information to the preset resource library.

[0011] According to a second aspect of the present disclosure, a virtual scene generation system is provided, the virtual scene generation system comprising a planning agent and an execution agent, the planning agent receiving a scene generation instruction input by a user, wherein the scene generation instruction comprises a description of a target scene to be generated; the planning agent selects a programmatic content generation plug-in for generating the target scene from a preset resource library according to plug-in annotation information in the preset resource library, and generates a task plan for generating the target scene, wherein the preset resource library comprises a plurality of the programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, the plug-in annotation information being used to describe the corresponding programmatic content generation plug-in; the execution agent executes the task plan using the programmatic content generation plug-in selected from the preset resource library to generate the target scene.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to execute the virtual scene generation method according to the first aspect of the present disclosure.

[0013] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the virtual scene generation method according to the first aspect of the present disclosure.

[0014] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer executable instructions, which, when executed by at least one processor, implement the virtual scene generation method according to the first aspect of the present disclosure.

[0015] The technical solution provided by the present disclosure brings at least the following beneficial effects:

[0016] According to the present disclosure, a virtual scene generation system can be used to select a programmatic content generation plug-in for generating a target scene based on the scene generation instructions input by the user and the plug-in annotation information in the preset resource library, generate a task plan, and use the programmatic content generation plug-in in the preset resource library to execute the task plan and generate the target scene. In this way, the target scene can be automatically generated based on the target scene description input by the user, without the user having to deeply participate in the scene generation process such as the selection, application, and adjustment of the PCG plug-in, thereby improving work efficiency.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0019] Figure 1 is a schematic flowchart of a virtual scene generation method according to an exemplary embodiment of the present disclosure.

[0020] Figure 2 is a schematic block diagram of an example of a virtual scene generation system according to an exemplary embodiment of the present disclosure.

[0021] Figure 3 is a schematic diagram of an overall example of executing a task plan in a virtual scene generation method according to an exemplary embodiment of the present disclosure.

[0022] Figure 4 is Figure 3 A schematic diagram of a specific example of a preprocessing program generated plug-in in the example.

[0023] Figure 5It is a schematic flowchart of the steps of adding a programmatic content generation plug-in to a preset resource library in a virtual scene generation method according to an exemplary embodiment of the present disclosure.

[0024] Figure 6 is Figure 3 A schematic diagram of a specific example of executing the current subtask in the example.

[0025] Figure 7 is a schematic block diagram of a virtual scene generation system according to an exemplary embodiment of the present disclosure.

[0026] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the present application will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0030] As mentioned above, in the related art, it is difficult to further improve the work efficiency of generating virtual scenes because of the need for manual participation in parameter setting and adjustment of PCG technology.

[0031] To address such issues, the exemplary embodiments of the present invention take into account technologies in multiple fields such as programmatic content generation, computer vision and image processing, and natural language processing, and rationally utilize the combination and optimization of these technologies, aiming to improve the efficiency, authenticity, and personalization of virtual scene generation, and meet the needs of different application scenarios for high-quality and diversified urban scenes.

[0032] Exemplary embodiments of the present disclosure provide a virtual scene generation method, a virtual scene generation system, an electronic device, a computer-readable storage medium, and a computer program product, which can solve or at least alleviate the above problems.

[0033] In a first aspect of an exemplary embodiment of the present disclosure, a virtual scene generating method is provided.

[0034] The virtual scene generation method according to the exemplary embodiment of the present disclosure can be applied to scenarios in which users interact with software. For example, a virtual scene generation system can be loaded on a user terminal, and the virtual scene generation system can include a planning agent and an execution agent. A user can input a scene generation instruction on the user terminal, and the user terminal can generate a virtual scene by executing the virtual scene generation method according to the exemplary embodiment of the present disclosure.

[0035] Specifically, the user terminal may receive a scene generation instruction input by a user, wherein the scene generation instruction includes a description of a target scene to be generated.

[0036] The user terminal can also select a programmatic content generation plug-in for generating a target scene from the preset resource library according to the plug-in annotation information in the preset resource library, and generate a task plan for generating the target scene, wherein the preset resource library includes multiple programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information is used to describe the corresponding programmatic content generation plug-in.

[0037] The user terminal can also use the programmatic content generation plug-in selected from the preset resource library to execute the task plan to generate the target scene.

[0038] The above-mentioned user terminal can be, for example, a tablet computer, a laptop computer, a digital assistant, a wearable device, etc. However, the implementation scenario of the above method is only an example scenario. The virtual scene generation method according to the exemplary embodiment of the present disclosure can also be applied to other application scenarios. For example, the user can send a scene generation instruction to the server through the network at the user terminal (for example, a mobile phone, a desktop computer, a tablet computer, etc.), and the server can complete the request by executing the virtual scene generation method according to the exemplary embodiment of the present disclosure. Here, the server can be an independent server, a server cluster, a cloud computing platform or a virtualization center.

[0039] According to the virtual scene generation method of the exemplary embodiment of the present disclosure, it is possible to automatically generate a target scene according to a target scene description input by a user without the user having to deeply participate in the scene generation process, thereby improving work efficiency.

[0040] The following will refer to Figures 1 to 6 An example of a virtual scene generation method according to an embodiment of the present disclosure is described. The virtual scene generation method can be executed in a virtual scene generation system, the virtual scene generation system can include multiple agents, and each step in the virtual scene generation method can be completed by an agent.

[0041] Here, each agent in the virtual scene generation system can be implemented by a large language model (Large Language Model, LLM), and the large language model can be, for example, but not limited to, ChatGPT (Chat Generative Pre-trained Transformer) and the like.

[0042] The virtual scene generation system can include a planning agent and an execution agent, and can also include other optional agents according to actual needs. A multi-agent framework can be constructed. For example, the functions and work contents of different agents can be defined through tools such as AutoGen, and interaction and collaboration between agents can be allowed. Here, when multiple agents are used, it is more conducive to supervision and optimization between agents, and by dividing different work contents into different agents, the impact of erroneous operations of individual agents on the global work results can be reduced, thereby improving the reliability of the system.

[0043] The following is an example code for initially defining each agent, where "Planner" can represent the planning agent described below, "pipeliner" can represent the execution agent described below, and "renderer" can represent the evaluation agent described below:

[0044] "{

[0045] Planner = Conversableagent(

[0046] name="planner",

[0047] system_message="You are a blender process designer.You completetasks by using different pcg."

[0048] “First call get_legal pcas()first,to get list of legal pcg.”

[0049] “Then call execute_pcg(pcg_name)to execute pcg.Before executing,payattention to whether the input of pcg exists”,

[0050] llm_config=llm_config, )

[0052] pipeliner=ConversableAgent(

[0053] name=“pipeliner”,

[0054] system_message=“You are a blender pipeline designer.You are givenVarious pcg,and you need to complete tasks based on these pcg.”

[0055] f“{pcg_info}”

[0056] “you can retrievel osm file first”

[0057] “You need to think about each step first”

[0058] “then you can call get_pcg_and_run()”

[0059] “Don’t write any form of code yourself!!!You only need to give thePcg you need to use in each step”

[0060] “If user say‘edit’,please just calleditor pcg,not other”,

[0061] llm_config=llm_config, )

[0063] renderer = ConversableAgent(

[0064] name = "renderer",

[0065] system_message="You are responsible for Blender's rendering work andprovide rendering results to the planner."

[0066] "call render(),to get rendering results.",

[0067] llm_config=llm_config,

[0068] }".

[0069] As an example, the virtual scene generation system may have Figure 2 The structure shown in FIG. 1 will be described in detail below in conjunction with the execution process of the virtual scene generation method. However, Figure 2 The system shown is only an example, and the virtual scene generation system according to the embodiment of the present disclosure may also have other structures as long as the corresponding virtual scene generation method can be executed.

[0070] like Figure 1 As shown, the virtual scene generation method may include the following steps:

[0071] In step S110, the planning agent may receive a scene generation instruction input by a user.

[0072] Here, the scene generation instruction may include a description of the target scene to be generated. For example, the user may describe the basic features of the scene to be generated, such as: Figure 3 As shown, the user can enter the description "Please Generate aModern city to me".

[0073] As an example, the target scene can be a virtual scene containing any visual content, such as but not limited to a virtual city, a virtual game space, a virtual indoor environment, etc. It can be a scene in any field, such as but not limited to a virtual testing environment for equipment in the industrial field, a virtual experimental environment in the medical field, a virtual training environment in the military field, a virtual teaching environment in the educational field, etc.

[0074] In addition, the embodiments according to the present disclosure also support users to input multimodal data to better instruct the virtual scene generation system to generate scenes, so that the generated scenes better meet the requirements and expectations of users.

[0075] As an example, the scene generation instruction may include multimodal input data. Specifically, in addition to the text description of the target scene to be generated input by the user, the scene generation instruction may also include input data of other modalities besides text, such as Figure 3 As shown, the scene generation instructions may also include but are not limited to OSM data, semantic maps and / or satellite images, etc.

[0076] Specifically, the user can provide the virtual scene generation system with a description of the target scene and data related to the scene to be generated or data to be added to the scene, so as to guide the system to better understand the user's description and generate a target scene that better meets the user's needs.

[0077] In step S120, the planning agent may select a programmatic content generation plug-in for generating a target scene from the preset resource library according to the plug-in annotation information in the preset resource library, and generate a task plan for generating the target scene.

[0078] Here, the preset resource library may include multiple programmatic content generation plug-ins (also referred to as "PCG plug-ins") and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information may be used to describe the corresponding programmatic content generation plug-in. As an example, the preset resource library may also be referred to as a preset asset library, and the resources in the preset resource library (such as programmatic content generation plug-ins) may also be referred to as assets.

[0079] As an example, Figure 2 and Figure 3 As shown, the planning agent 220 can formulate a detailed task plan for generating the target scene based on the user's input description and multimodal data. The task plan includes determining the overall structure of the generated scene, selecting a suitable PCG plug-in, and / or planning the execution order and priority of each subtask.

[0080] As an example, the planning agent 220 can generate a preliminary task plan based on the user's scenario generation instructions, and during the task execution process, dynamically plan the next operation according to the task goal and workflow of the current subtask.

[0081] Specifically, the planning agent 220 formulates a task plan at the beginning of each task, and the task plan includes a rough workflow, such as Figure 3As shown, in the task planning of the first stage, a task plan can be generated, which may include X subtasks. The planning agent 220 can dynamically plan the next subtask action according to the current subtask during the execution of the specific subtask until the user's needs are met, which will be described in detail below.

[0082] In step S120, the planning agent can select a programmatic content generation plug-in suitable for generating the current target scene according to the plug-in annotation information in the preset resource library. Here, the task plan can include one or more subtasks, and the generation of the target scene can be achieved by executing these subtasks in sequence. Each subtask can include the programmatic content generation plug-in to be called and the plug-in annotation information of the plug-in.

[0083] A PCG plug-in may refer to a process of creating content through a predetermined algorithm or rule. Different plug-ins may implement different functions, including but not limited to object creation, asset management, large scene layout generation, scene setting, etc. As an example, the preset resource library may include asset-type PCG plug-ins and / or functional PCG plug-ins, wherein an asset-type PCG plug-in refers to a PCG of a node group without UI (User Interface) code, and the user cannot customize the parameters of the plug-in; a functional PCG plug-in refers to a PCG of a node group with UI code, and the user can edit the parameters of the PCG plug-in by inputting or outputting at the node.

[0084] Here, object creation may refer to the generation of new objects such as three-dimensional (3D) objects in a scene. Asset management may refer to the management and use of different materials or resources, such as models, materials, etc., in a scene, which may involve the retrieval and loading of different plug-ins, materials, and 3D models into the scene as described below. Large scene layout generation may be related to the arrangement and distribution of objects in a scene, for example, may include placing objects in a large scene according to specific rules or layouts. Scene setting may involve creating (or setting) a background for a scene, for example, including specifying weather, lighting, etc. in a scene, and its main task is usually to add background elements or environment to the entire scene.

[0085] According to the embodiments of the present disclosure, the programmatic content generation plug-in can be any existing plug-in, such as an open source program for an object in a scene that an artist has completed designing, etc. The programmatic content generation plug-in can be dynamically collected and added to the preset resource library according to the embodiments of the present disclosure. In addition, in the embodiments of the present disclosure, there is no particular limitation on the specific type, function, format, and acquisition method of the programmatic content generation plug-in.

[0086] Here is an example of building and dynamically updating a preset resource library. Specifically, Figure 3 and Figure 4 As shown, before executing the scene generation task, the programmatic content generation plug-in can be preprocessed to obtain a preset resource library. The preprocessing stage can be a preparation stage before the scene generation task starts. The collected PCG plug-ins with various functions, different inputs and / or different outputs can be formatted and packaged according to the preset PCG management protocol, and classified and labeled by the intelligent agent in the virtual scene generation system to obtain the labeling results, such as the plug-in annotation information corresponding to each PCG plug-in described above. The plug-in annotation information and the PCG plug-in can together constitute a PCG platform (bench) as a preset resource library for use in the scene generation task process.

[0087] As an example, the virtual scene generation system may include a labeling agent 210, such as Figure 5 As shown, the annotation agent 210 can add a programmatic content generation plug-in to the preset resource library in the following manner:

[0088] In step S510, structured packaging information of the programmatic content generation plug-in to be added may be obtained.

[0089] Here, the structured packaging information may include basic information of the programmatic content generation plug-in to be added, input quantity and output quantity of the programmatic content generation plug-in to be added, and restriction information for using the programmatic content generation plug-in to be added.

[0090] As an example, in this step, a structured packaging template can be generated according to a preset programmatic content generation (PCG) management protocol; based on the user's input to the structured packaging template, the programmatic content generation plug-in to be added can be structured and packaged to obtain the above-mentioned structured packaging information.

[0091] The PCG management protocol can be used to standardize and manage the integration and operation of different PCG plug-ins. Its content can be set according to actual needs. For example, the protocol can specify custom content that needs to be entered by the user in the structured packaging template.

[0092] The structured packaging template can be a packaging document provided to the user, which may include custom content that needs to be input by the user, so as to guide the user to perform the PCG packaging operation and reduce the technical threshold for beginners to package the PCG program into a PCG plug-in with structured annotations. By providing a structured packaging template, even non-professionals can package the PCG plug-in for subsequent processing by the annotation agent 210.

[0093] The custom content in the structured packaging template may include, for example, but is not limited to, basic information of the programmatic content generation plug-in, input and output quantities of the programmatic content generation plug-in, and restriction information for using the programmatic content generation plug-in.

[0094] Here, the basic information of the programmatic content generation plug-in may include, but is not limited to, the function or purpose of the programmatic content generation plug-in; the restriction information for using the programmatic content generation plug-in may include, but is not limited to, the restriction on the input amount.

[0095] According to an embodiment of the present disclosure, when a PCG plug-in is obtained, a structured encapsulation template can be generated according to the PCG management protocol, and the structured encapsulation template can be displayed to the user to receive the user's input for the custom content in the structured encapsulation template. As an example, the above process of obtaining structured encapsulation information can be implemented by Python language. However, the method of obtaining structured encapsulation information is not limited to the above example. For example, the above structured encapsulation information can also be obtained from the plug-in provider or source when the PCG plug-in is obtained.

[0096] In addition, the PCG management protocol described above may also include: provisions for a dynamic application programming interface (Application Programming Interface, API) (hereinafter referred to as the "dynamic API conversion interface"), where the dynamic API conversion interface can achieve the connection and conversion between input quantities and output quantities with non-corresponding formats, thereby allowing free combination between different PCG plug-ins. Specifically, the dynamic API conversion interface can be implemented in the following ways: collect and compile the input format and output format of all APIs, and define a dynamic API conversion interface that covers all input formats and all output formats. The dynamic API conversion interface solves the problem of the lack of a unified communication protocol in the existing PCG API, ensures that different PCGs can be freely combined, allows communication between APIs of different formats, and increases the diversity of generated content.

[0097] Alternatively or additionally, the PCG management protocol may also include: retrieving and expanding the preset resource library through text description and image rendering (i.e., adding plug-ins to the preset resource library). Here, the preset resource library can be retrieved in the following manner: the image rendered from the plug-in can be encoded into a plug-in vector using a pre-trained CLIP (Contrastive Language-Image Pre-training) model, and the description input by the user is matched with the plug-in vector by calculating the cosine similarity, and one result is selected from the most similar results and imported into the target scene. The process of retrieving in the preset resource library can be implemented, for example, by the planning agent described below. Here, the use of the pre-trained CLIP model for text-to-image retrieval can meet the needs of diversified scene generation based on multimodal data.

[0098] In step S520, the programmatic content generation plug-in to be added may be classified based on the structured packaging information, so as to add one or more classification tags to the programmatic content generation plug-in to be added.

[0099] As an example, the programmatic content generation plug-in to be added can be classified according to the preset plug-in category (class). According to the function of the plug-in, the plug-in category can include, but is not limited to, the object creation class, asset management class, large scene layout generation class, scene setting class, etc. described above. Here, a programmatic content generation plug-in can be classified into one or more plug-in categories. For example, the same programmatic content generation plug-in can belong to both the object creation class and the large scene layout generation class.

[0100] When the plug-in category of the programmatic content generation plug-in to be added is determined, a corresponding classification tag may be added to the plug-in, for example, Figure 3 As shown, the plug-in "scatter" can be classified as "Scene Layout", that is, the corresponding label can be added to it.

[0101] In step S530, plug-in annotation information corresponding to the programmatic content generation plug-in to be added may be generated based on the classification tag and the structured packaging information.

[0102] In this step, the annotation agent 210 can use the classification label and the structured encapsulation information as the plug-in annotation information for the plug-in. Figure 2 As shown, the plugin annotation information of the plugin "scatter" may include the following:

[0103] {

[0104] "class":"Scene Layout",

[0105] "description":"Place the object.",

[0106] "input": "the name of the object being placed and The name of the object to support.",

[0107] "limitation":"only place objects on flat terrain

[0108] }

[0109] Here, "class" indicates the plug-in category; "description" indicates the basic information of the plug-in, such as the description of the plug-in; "input" indicates the input amount of the plug-in; and "limitation" indicates the limitation information of the plug-in.

[0110] In step S540, the programmatic content generation plug-in to be added and the corresponding plug-in annotation information may be added to the preset resource library.

[0111] In this step, the plug-in and the corresponding plug-in annotation information can be added to the preset resource library for initial construction of the preset resource library or for dynamic updating of the preset resource library after the initial construction is completed. For example, new PCG plug-ins can be obtained regularly or in response to plug-in acquisition conditions and added to the preset resource library. If storage conditions permit, the resources or plug-ins in the preset resource library can be added indefinitely.

[0112] In addition, according to an embodiment of the present disclosure, the virtual scene generation system may further include a common message pool, and each intelligent agent in the system may access the common message pool, and may share information in the common message pool so that each intelligent agent may exchange information in real time. Here, the above-mentioned plug-in annotation information may also be stored in the common message pool for access by each intelligent agent, thereby facilitating each intelligent agent to better understand or use the corresponding plug-in by accessing the plug-in annotation information.

[0113] In the example described above, classification and labeling of PCG plug-ins can be achieved by labeling intelligent agents, thereby generating plug-in annotation information, which can be used to retrieve plug-ins, call plug-ins, etc. during subsequent task execution, so that the intelligent agent can efficiently access and understand each plug-in, perform actions at different plug-in levels, and enable the intelligent agent to reasonably use the plug-ins in the preset resource library for scene generation.

[0114] Although it is described above that resources are added to the preset resource library by labeling an intelligent agent, the embodiments of the present disclosure are not limited to this. The above process can also be implemented by one or more other intelligent agents in the virtual scene generation system of the embodiments of the present disclosure, or can also be implemented by manual or other external systems, as long as the above preset resource library can be constructed.

[0115] According to the embodiments of the present disclosure, a dynamic API conversion interface, structured encapsulation, and unlimited preset resource libraries can be provided, achieving seamless integration and efficient management of multiple plug-ins.

[0116] Return to reference Figure 1 In step S130, the executing agent may utilize the programmatic content generation plug-in selected from the preset resource library to execute the task plan to generate the target scene.

[0117] In this step, the target scene can be generated according to the task plan using the PCG plug-in in the task.

[0118] As an example, a task plan includes one or more subtasks. Each subtask may correspond to a part of the scene content in the target scene, such as Figure 3 As shown, the overall task plan can be divided into X subtasks, and subtask 1, subtask 2, subtask 3 to subtask X can be executed in sequence to obtain the final scene generation result.

[0119] like Figure 3 and Figure 6 As shown, for any current subtask, the task plan can be executed in the following manner: the execution agent 230 receives the task information of the current subtask from the planning agent 220; the execution agent 230 selects the target programmatic content generation plug-in from the preset resource library based on the task information received from the planning agent 220, and executes the current subtask.

[0120] Here, the task information may indicate a target programmatic content generation plug-in that matches the current subtask.

[0121] The executive agent 230 may be responsible for managing all PCG plug-ins and utilizing the annotated PCG plug-ins to process program generation or resource manipulation subtasks in the 3D design software.

[0122] The execution agent 230 can select a suitable PCG plug-in from the PCG platform according to the parameters provided by the planning agent 220, and execute the generation and editing of the target scene in a 3D design software such as Blender. During the execution process, the execution results and related resources of each PCG plug-in will be recorded and transmitted to the evaluation agent 250 described later for comprehensive evaluation, which will be described in detail below.

[0123] by Figure 6 For example, when the current subtask is subtask 1, the planning agent 220 may send task information of the current subtask to the execution agent 230. The task information may include:

[0124] {"pcg_name":"assetretrievel"

[0125] "pcg_info":"OpenStreetMap data for city structure"}.

[0126] The execution agent 230 can select a target programmatic content generation plug-in from a preset resource library and execute the current subtask according to the task information received from the planning agent 220. For example, the execution agent 230 can call and execute a PCG plug-in, such as an action function "assetretrievel", according to the received task information.

[0127] Through the above method, the planning agent can generate a task plan and send the subtasks in the task plan to the execution agent. The execution agent calls the PCG plug-in to perform the corresponding subtask actions to generate the scene content to be generated by the current subtask, thereby realizing automated task execution.

[0128] As an example, the virtual scene generation system may further include an assistant agent 240, wherein before the planning agent 220 sends the task information of the current subtask to the execution agent 230, the generation method may further include:

[0129] The assistant agent 240 can verify whether the current task plan is correct and send the verification result to the planning agent 220.

[0130] In response to the verification result indicating that the current task plan is incorrect, the planning agent 220 can correct the task plan and send the corrected task plan to the assistant agent 240 for re-verification. In response to the verification result indicating that the current task plan is correct, the planning agent 220 determines the current subtask in the current task plan to perform the step of sending the task information of the current subtask to the execution agent 230.

[0131] Here, verifying whether the current task plan is correct includes: verifying whether the execution conditions of the current subtask are met and / or verifying whether the timing of executing the current subtask is correct, wherein verifying whether the execution conditions of the current subtask are met may include verifying whether the scene contents match and / or verifying whether the task objectives of the current subtask comply with the description in the scene generation instructions.

[0132] Specifically, the assistant agent 240 can verify the current subtask itself, the overall process of the task plan, and the execution order or priority between the subtasks in the task plan.

[0133] For example, in the task verification phase, the assistant agent 240 can verify the subtask to be executed, determine whether the task content is correct and whether the conditions for executing the subtask are currently met. After ensuring that the execution conditions are met, the assistant agent 240 can inform the planning agent 220 to generate parameters and pass them to the execution agent 230. Then, in the action execution phase, the execution agent 230 can select the corresponding PCG plug-in in the PCG platform according to the parameters provided in the task verification phase.

[0134] The assistant agent 240 verifies each subtask before the task begins to ensure that the execution conditions are met and the task is correct. For example, it verifies whether the layout of the building is consistent with the map data, whether the weather setting is consistent with the description entered by the user, etc. The verification results are fed back to the planning agent 220 by the assistant agent 240 for further parameter generation and adjustment.

[0135] For example, Figure 6 As shown, the assistant agent 240 can obtain the current subtask 1 "Retrieve the osmfile for the city" and verify its correctness. If the assistant agent 240 verifies that the subtask 1 is correct, the verification result can be sent to the planning agent 220, and the planning agent 220 can send the task information of the subtask to the execution agent 230 for action execution; if the assistant agent 240 verifies that the subtask 1 is incorrect, the verification result can be sent to the planning agent 220, for example, informing the planning agent 220 "You are wrong! You should load osmfile", and the planning agent 220 can correct the parameters of the current subtask, and send the corrected subtask "Load theosm file" to the assistant agent 240 for re-verification until the verification result indicates that the current subtask 1 is correct.

[0136] In the above manner, an assistant agent can be set up to verify or supervise the task plan of the planning agent, thereby avoiding the possibility of deviation or unreasonable planning due to the use of a single agent for task planning.

[0137] As an example, the virtual scene generation system also includes an evaluation agent 250. In this example, the generation method may also include: the execution agent 230 sends the execution result of each target programmatic content generation plug-in to the evaluation agent after executing the current subtask; the evaluation agent 250 evaluates the execution result according to the code logic of the target programmatic content generation plug-in and the visual effect obtained based on the execution result, and feeds back the evaluation result to the planning agent 220.

[0138] Here, in response to the evaluation result indicating that the execution result of the current subtask meets the expected requirements, the planning agent 220 sends the task information of the next subtask to the execution agent 230 to execute the next subtask; in response to the evaluation result indicating that the execution result of the current subtask does not meet the expected requirements, the planning agent 220 adjusts the parameters in the task information of the current subtask and sends the adjusted task information of the current subtask to the execution agent 230 to re-execute the current subtask.

[0139] Specifically, the evaluation agent 250 can comprehensively consider the two aspects of code logic and visual effects, such as the programming results of LLM and the visual effects generated by Blender software, evaluate and feedback the results generated by each subtask, and guide the planning agent 220 to take the next action. If the results of both aspects meet expectations (for example, matching the scene generation instructions input by the user), the next subtask will continue to be executed; if at least one does not meet expectations, it will fall back to the action execution stage, and adjust and iterate through the execution agent 230 until the evaluation of the evaluation agent 250 achieves the expected effect.

[0140] As an example, for code logic, the evaluation agent 250 can read code from a language program such as Python and determine whether the task is performed correctly from the code level. For visual effects, the evaluation agent 250 can obtain scene information, such as object positions and rendered images, from design software such as Blender, and generate a visualization scene based on the scene information. The visualization scene can be similar to Figure 2 The evaluation agent 250 can compare such a visual scene with the expected goal of the scene generation instruction input by the user, and evaluate the difference between the two, such as calculating the distance or similarity between pixels, so as to evaluate whether the current subtask is correct at the visual level.

[0141] After executing in the 3D design software, the execution agent 230 will pass the execution result to the evaluation agent 250. The evaluation agent 250 can evaluate whether the current execution result is reasonable in terms of code logic and visual effects, and can inform the planning agent 220 of the evaluation result. If the evaluation result indicates that the execution result is accurate, the planning agent 220 will start the next subtask; otherwise, the planning agent 220 will fall back to loop and iterate the current subtask until the execution result meets expectations. By looping and iterating the above process, each subtask is executed in turn until the entire generation task is completed, which can continuously optimize the execution effect of each subtask, achieve highly realistic and large-scale 3D city scene generation, and meet the needs of diversification and fine control.

[0142] In addition, as an example, during the above execution process, the user can interact with each agent to meet the needs of fine control and personalized editing.

[0143] For example, the virtual scene generation system according to an embodiment of the present disclosure may also include a user interaction interface, which enables the user to interact with the intelligent agent in the multi-agent framework in real time during the scene generation process to meet personalized editing and fine control requirements.

[0144] The generation method may also include: the planning agent 220 or the assistant agent 240 displays the execution progress of the task plan in real time; the planning agent 220 or the assistant agent 240 adds the task parameters to the task information of the current subtask in response to receiving the task parameters input by the user.

[0145] Here, displaying the execution progress of the task plan may include: displaying the current task plan, displaying task information of the current subtask or the next subtask, displaying success or failure of the current subtask execution and / or displaying task parameters required for the execution of the current subtask that require user input.

[0146] In the above manner, during the entire task generation process, the user can interact with the planning agent 220 and / or the assistant agent 240 to adjust the parameters and requirements of the generation task, thereby achieving fine control and personalized editing of the target scene generation process. During the task execution process, the system can find feasible solutions through multiple rounds of interaction and visual feedback with the user, thereby completing the target task and simplifying the city generation process of multi-modal input.

[0147] According to the virtual scene generation method of the embodiment of the present disclosure, scene generation can be implemented based on technologies in multiple fields such as computer science, artificial intelligence (AI), image processing and three-dimensional modeling.

[0148] Specifically, this generation method takes into account the important role of computer vision and image processing technology in programmatic content generation. It can use feature extraction, object recognition, image segmentation and other technologies to process and analyze image data obtained from multimodal input data, which can help the system understand and process complex virtual scene data, thereby supporting accurate content generation.

[0149] In addition, natural language processing (NLP) technology is widely used to extract and understand information from text descriptions. In virtual scene generation, NLP technology can help parse the descriptions provided by users, such as scene style, weather conditions, and specific requirements, and guide the program to generate adaptable urban scenes that meet user expectations. In particular, the design and optimization of a multi-agent system can also be used. The multi-agent system can effectively manage the complex multi-round interactions and visual feedback in the generation task, solving the problem of complex hierarchical structures of controllable elements in 3D design software such as Blender. In addition, in this generation method, by manipulating resources (such as PCG plug-ins) in the scene generation task, adjusting weather, and modifying the scene customization, the specific scene content can be flexibly and accurately controlled.

[0150] In general, the virtual scene generation method according to the embodiment of the present disclosure can provide a controllable programmatic content generation method for generating realistic large-scale 3D virtual scenes, which can build a system based on two key components: the PCG management protocol and the multi-agent framework. The PCG management protocol provides a unified standard for various PCG plug-ins, and through dynamic API conversion interfaces, structured packaging and preset resource libraries, it can achieve flexible integration and expansion of different plug-ins, significantly reducing the user's usage threshold; the multi-agent framework can effectively manage the complex interactions between the large language model (LLM) and Blender, and through the collaboration between multiple agents, it can achieve the generation and customization of high-quality 3D virtual scenes.

[0151] Experimental verification shows that the virtual scene generation method according to the embodiment of the present disclosure can generate highly realistic large-scale urban scenes using multimodal input data including OSM data, semantic maps, satellite images, and text descriptions, and is significantly superior to scene generation methods in related technologies in terms of generation quality and diversity. User evaluation and comparative experiments have proven the effectiveness and flexibility of this generation method in scene generation tasks, and it has broad application prospects.

[0152] In a second aspect of the exemplary embodiments of the present disclosure, a virtual scene generation system is provided, such as Figure 7 As shown, the virtual scene generation system 700 may include a planning agent 710 and an execution agent 720. In addition, the virtual scene generation system 700 may also have the same Figure 2 , Figure 3 Same structure.

[0153] The planning agent 710 receives a scene generation instruction input by a user, wherein the scene generation instruction includes a description of a target scene to be generated.

[0154] The planning agent 710 selects a programmatic content generation plug-in for generating a target scene from the preset resource library according to the plug-in annotation information in the preset resource library, and generates a task plan for generating the target scene, wherein the preset resource library includes multiple programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information is used to describe the corresponding programmatic content generation plug-in.

[0155] The execution agent 720 generates a plug-in using programmatic content selected from a preset resource library and executes the task plan to generate a target scene.

[0156] As an example, the task plan includes one or more subtasks, wherein the task plan is executed in the following manner: the execution agent 720 receives task information of the current subtask from the planning agent 710, wherein the task information indicates a target programmatic content generation plug-in that matches the current subtask; the execution agent 720 selects a target programmatic content generation plug-in from a preset resource library based on the task information received from the planning agent 710, and executes the current subtask.

[0157] As an example, the virtual scene generation system also includes an assistant agent, wherein, before the planning agent 710 sends the task information of the current subtask to the execution agent 720, the assistant agent verifies whether the current task plan is correct and sends the verification result to the planning agent 710, wherein verifying whether the current task plan is correct includes: verifying whether the execution conditions of the current subtask are met and / or verifying whether the timing of executing the current subtask is correct, wherein verifying whether the execution conditions of the current subtask are met includes verifying whether the scene contents match and / or verifying whether the task objectives of the current subtask comply with the description in the scene generation instructions; the planning agent 710, in response to the verification result indicating that the current task plan is incorrect, corrects the task plan, and sends the corrected task plan to the assistant agent for re-verification; the planning agent 710, in response to the verification result indicating that the current task plan is correct, determines the current subtask in the current task plan to execute the step of sending the task information of the current subtask to the execution agent 720.

[0158] As an example, the virtual scene generation system also includes an evaluation agent, and the execution agent 720 sends the execution result of each target programmatic content generation plug-in to the evaluation agent after executing the current subtask; the evaluation agent evaluates the execution result according to the code logic of the target programmatic content generation plug-in and the visual effect obtained based on the execution result, and feeds back the evaluation result to the planning agent 710; in response to the evaluation result indicating that the execution result of the current subtask meets the expected requirements, the planning agent 710 sends the task information of the next subtask to the execution agent 720 to execute the next subtask; in response to the evaluation result indicating that the execution result of the current subtask does not meet the expected requirements, the planning agent 710 adjusts the parameters in the task information of the current subtask, and sends the adjusted task information of the current subtask to the execution agent 720 to re-execute the current subtask.

[0159] As an example, the planning agent 710 or the assistant agent displays the execution progress of the task plan in real time, wherein displaying the execution progress of the task plan includes: displaying the current task plan, displaying the task information of the current subtask or the next subtask, displaying the success or failure of the current subtask execution and / or displaying the task parameters required for the execution of the current subtask that require user input; the planning agent 710 or the assistant agent adds the task parameters to the task information of the current subtask in response to receiving the task parameters input by the user.

[0160] As an example, the virtual scene generation system also includes an annotation agent, which adds a programmatic content generation plug-in to the preset resource library in the following way: obtaining structured packaging information of the programmatic content generation plug-in to be added, wherein the structured packaging information includes basic information of the programmatic content generation plug-in to be added, input and output quantities of the programmatic content generation plug-in to be added, and restriction information for using the programmatic content generation plug-in to be added; based on the structured packaging information, classifying the programmatic content generation plug-in to be added, so as to add one or more classification tags to the programmatic content generation plug-in to be added; based on the classification tags and the structured packaging information, generating plug-in annotation information corresponding to the programmatic content generation plug-in to be added; adding the programmatic content generation plug-in to be added and the corresponding plug-in annotation information to the preset resource library.

[0161] Regarding the device in the above-mentioned embodiment, the specific manner in which each intelligent agent performs operations has been described in detail in the embodiment of the method. Each intelligent agent in the virtual scene generation system can execute the corresponding steps in the method according to the virtual scene generation method in the method embodiment of the first aspect above, which will not be elaborated in detail here.

[0162] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Figure 8 As shown, the electronic device 10 includes a processor 101 and a memory 102 for storing processor executable instructions. Here, when the processor executable instructions are executed by the processor, the processor is prompted to execute the virtual scene generation method as described in the above exemplary embodiment.

[0163] As an example, the electronic device 10 is not necessarily a single device, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or in combination. The electronic device 10 may also be part of an integrated control system or system manager, or may be configured as a server that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0164] In the electronic device 10, the processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not a limitation, the processor 101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0165] The processor 101 may execute instructions or codes stored in the memory 102, wherein the memory 102 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0166] The memory 102 may be integrated with the processor 101, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 102 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 102 and the processor 101 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 101 can read files stored in the memory 102.

[0167] In addition, the electronic device 10 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 10 may be connected to each other via a bus and / or a network.

[0168] In an exemplary embodiment, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can execute the virtual scene generation method as described in the above exemplary embodiment. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: read-only memory (ROM), random access memory (RAM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0169] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided. The computer program product includes computer executable instructions. When the computer executable instructions are executed by at least one processor, the virtual scene generation method according to the exemplary embodiment of the present disclosure is implemented.

[0170] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0171] In addition, it should be noted that although several examples of each step are described above with reference to specific drawings, it should be understood that the embodiments of the present disclosure are not limited to the combinations given in the examples, and the steps appearing in different drawings may be combined, and the order of execution of the steps may be changed, which is not exhaustive here.

[0172] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A virtual scene generation method, characterized in that: The generation method is executed in a virtual scene generation system, the virtual scene generation system includes a planning agent, an execution agent and a labeling agent, wherein the virtual scene generation method includes: The planning agent receives a scene generation instruction input by a user, wherein the scene generation instruction includes a description of a target scene to be generated; The planning agent selects a programmatic content generation plug-in for generating the target scenario from the preset resource library according to the plug-in annotation information in the preset resource library, and generates a task plan for generating the target scenario, wherein the preset resource library includes a plurality of the programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information is used to describe the corresponding programmatic content generation plug-in; The execution agent generates a plug-in using programmatic content selected from the preset resource library to execute the task plan to generate the target scene. The annotation agent adds a programmatic content generation plug-in to the preset resource library in the following manner: obtaining structured packaging information of the programmatic content generation plug-in to be added; classifying the programmatic content generation plug-in to be added based on the structured packaging information to add one or more classification tags to the programmatic content generation plug-in to be added; generating plug-in annotation information corresponding to the programmatic content generation plug-in to be added based on the classification tags and the structured packaging information; adding the programmatic content generation plug-in to be added and the corresponding plug-in annotation information to the preset resource library, The structured packaging information includes basic information of the programmatic content generation plug-in to be added, input and output of the programmatic content generation plug-in to be added, and restriction information for using the programmatic content generation plug-in to be added. The annotation agent generates a structured packaging template according to a preset programmatic content generation management protocol, and based on the user's input to the structured packaging template, performs structured packaging on the programmatic content generation plug-in to be added to obtain the structured packaging information, wherein the programmatic content generation management protocol is used to standardize and manage the integration and operation of the programmatic content generation plug-in, and the structured packaging template is a packaging document provided to the user, and the packaging document includes custom content that requires user input.

2. The virtual scene generation method according to claim 1, characterized in that: The task plan includes one or more subtasks, wherein the task plan is executed in the following manner: The execution agent receives task information of the current subtask from the planning agent, wherein the task information indicates a target programmatic content generation plug-in matching the current subtask; The execution agent selects the target programmed content generation plug-in from the preset resource library according to the task information received from the planning agent, and executes the current subtask.

3. The virtual scene generation method according to claim 2, characterized in that: The virtual scene generation system further includes an assistant agent, wherein before the planning agent sends the task information of the current subtask to the execution agent, the virtual scene generation method further includes: The assistant agent verifies whether the current task plan is correct, and sends the verification result to the planning agent, wherein the verification of whether the current task plan is correct includes: verifying whether the execution condition of the current subtask is met and / or verifying whether the timing of executing the current subtask is correct, wherein the verification of whether the execution condition of the current subtask is met includes verifying whether the scene contents match each other and / or verifying whether the task goal of the current subtask meets the description in the scene generation instruction; In response to the verification result indicating that the current task plan is incorrect, the planning agent corrects the task plan and sends the corrected task plan to the assistant agent for re-verification; In response to the verification result indicating that the current task plan is correct, the planning agent determines the current subtask in the current task plan to execute the step of sending the task information of the current subtask to the execution agent.

4. The virtual scene generation method according to claim 2, characterized in that: The virtual scene generation system further includes an evaluation agent, wherein the virtual scene generation method further includes: The execution agent sends the execution result of each target programmatic content generation plug-in to the evaluation agent after executing the current subtask; The evaluation agent evaluates the execution result according to the code logic of the target programmatic content generation plug-in and the visual effect obtained based on the execution result, and feeds back the evaluation result to the planning agent; The planning agent sends the task information of the next subtask to the execution agent in response to the evaluation result indicating that the execution result of the current subtask meets the expected requirements, so as to execute the next subtask; In response to the evaluation result indicating that the execution result of the current subtask does not meet the expected requirements, the planning agent adjusts the parameters in the task information of the current subtask and sends the adjusted task information of the current subtask to the execution agent to re-execute the current subtask.

5. The virtual scene generation method according to claim 3, characterized in that: The virtual scene generation method also includes: The planning agent or the assistant agent displays the execution progress of the task plan in real time, wherein the display of the execution progress of the task plan includes: displaying the current task plan, displaying the task information of the current subtask or the next subtask, displaying the success or failure of the execution of the current subtask and / or displaying the task parameters required for the execution of the current subtask that need to be input by the user; In response to receiving the task parameters input by the user, the planning agent or the assistant agent adds the task parameters to the task information of the current subtask.

6. A virtual scene generation system, characterized in that: The virtual scene generation system includes a planning agent, an execution agent and a labeling agent. The planning agent receives a scene generation instruction input by a user, wherein the scene generation instruction includes a description of a target scene to be generated; The planning agent selects a programmatic content generation plug-in for generating the target scenario from the preset resource library according to the plug-in annotation information in the preset resource library, and generates a task plan for generating the target scenario, wherein the preset resource library includes a plurality of the programmatic content generation plug-ins and plug-in annotation information corresponding to each programmatic content generation plug-in, and the plug-in annotation information is used to describe the corresponding programmatic content generation plug-in; The execution agent generates a plug-in using programmatic content selected from the preset resource library to execute the task plan to generate the target scene. The annotation agent adds a programmatic content generation plug-in to the preset resource library in the following manner: obtaining structured packaging information of the programmatic content generation plug-in to be added; classifying the programmatic content generation plug-in to be added based on the structured packaging information to add one or more classification tags to the programmatic content generation plug-in to be added; generating plug-in annotation information corresponding to the programmatic content generation plug-in to be added based on the classification tags and the structured packaging information; adding the programmatic content generation plug-in to be added and the corresponding plug-in annotation information to the preset resource library, The structured packaging information includes basic information of the programmatic content generation plug-in to be added, input and output of the programmatic content generation plug-in to be added, and restriction information for using the programmatic content generation plug-in to be added. The annotation agent generates a structured packaging template according to a preset programmatic content generation management protocol, and based on the user's input to the structured packaging template, performs structured packaging on the programmatic content generation plug-in to be added to obtain the structured packaging information, wherein the programmatic content generation management protocol is used to standardize and manage the integration and operation of the programmatic content generation plug-in, and the structured packaging template is a packaging document provided to the user, and the packaging document includes custom content that requires user input.

7. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor executable instructions, When the processor executable instructions are executed by the processor, the processor is prompted to execute the virtual scene generation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the virtual scene generating method according to any one of claims 1 to 5.

9. A computer program product comprising computer executable instructions, characterized in that: When the computer executable instructions are executed by at least one processor, the virtual scene generation method according to any one of claims 1 to 5 is implemented.

Images