Light source assembly control method and device of rendering engine, electronic equipment and storage medium

Abstract

The invention discloses a light source assembly control method and device of a rendering engine, electronic equipment and a storage medium. The method comprises the steps that light source information of all light source examples in a virtual scene is acquired; acquiring a light source demand text input by a user; inputting the light source information and the light source demand text of the light source instance into a light source setting prediction model to obtain a light source setting instruction corresponding to the light source demand text; the light source setting prediction model is used for outputting a light source setting instruction capable of being recognized by the rendering engine and achieving a light source effect corresponding to the light source demand text according to the light source demand text, and the light source setting instruction comprises light source attributes and corresponding numerical values; and according to a light source setting instruction corresponding to the light source demand text, controlling a light source assembly of the light source instance in the virtual scene through a rendering engine. A user does not need to master related operation knowledge of the light source assembly of the rendering engine, and can quickly adjust the light source assembly in the virtual scene only by inputting the light source demand text, so that an expected illumination effect is achieved.

Description

Technical Field

[0001] The present application relates to the field of rendering technology, and in particular to a method, device, electronic device and computer-readable storage medium for controlling a light source component of a rendering engine. Background Art

[0002] 3D rendering engines can provide a complete set of development tools for creating virtual scenes. Lighting is a key element in virtual scenes. It not only affects the visual effects, but also enhances the overall experience quality. It has an important impact on the atmosphere, functions and user experience of virtual scenes. Reasonable use of lighting can significantly improve the expressiveness and attractiveness of virtual scenes. However, the control of the light source component of the rendering engine often requires setting many parameters and requires manual adjustment by senior developers. There are problems such as high difficulty in light source control and high technical barriers. In particular, it is not friendly to novice users, who need to learn a lot of professional knowledge before they can get started. At the same time, when the rendering engine needs to be updated iteratively, developers need to learn and master the new document content to use it correctly, which takes a lot of time. Summary of the invention

[0003] The present application provides a method, device, electronic device and computer-readable storage medium for controlling a light source component of a rendering engine, so as to reduce the difficulty of controlling the light source of the rendering engine and improve the control efficiency of the light source component.

[0004] In a first aspect, an embodiment of the present application provides a method for controlling a light source component of a rendering engine, the method comprising:

[0005] Acquire light source information of a light source instance in a virtual scene; wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information;

[0006] Acquire input light source requirement text, where the light source requirement text is used to represent the light source effect requirement in a virtual scene created by a rendering engine;

[0007] Inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model to obtain a light source setting instruction corresponding to the light source requirement text; wherein the light source setting prediction model is used to output a light source setting instruction that can be recognized by the rendering engine and realizes the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values;

[0008] According to the light source setting instruction, the light source component of the light source instance in the virtual scene is controlled by the rendering engine.

[0009] In a second aspect, an embodiment of the present application provides a light source component control device for a rendering engine, the device comprising:

[0010] A first acquisition module is used to acquire light source information of a light source instance in a virtual scene; wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information;

[0011] A second acquisition module is used to acquire an input light source requirement text, where the light source requirement text is used to represent the light source effect requirement in the virtual scene created by the rendering engine;

[0012] A prediction module, used for inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model, and obtaining a light source setting instruction corresponding to the light source requirement text; wherein the light source setting prediction model is used for outputting a light source setting instruction that can be recognized by the rendering engine and realizes the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values;

[0013] A control module is used to control the light source components of the light source instance in the virtual scene through the rendering engine according to the light source setting instruction.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0015] A memory and a processor, wherein the memory and the processor are coupled;

[0016] The memory is used to store one or more computer instructions;

[0017] The processor is used to execute the one or more computer instructions to implement the light source component control method of the rendering engine described in any one of the first aspects above.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having one or more computer instructions stored thereon, characterized in that the instruction is executed by a processor to implement the light source component control method of the rendering engine described in any one of the first aspects above.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the light source component control method of the rendering engine described in any one of the first aspects above.

[0020] Compared with the prior art, this application has the following advantages:

[0021] The light source component control method of the rendering engine provided in the present application obtains the light source information of the light source instance in the virtual scene. Wherein, the virtual scene is created by the rendering engine, and the light source information includes light source attribute information and light source position information. The input light source requirement text is obtained, and the light source requirement text is used to characterize the light source effect requirement in the virtual scene created by the rendering engine. The light source information and the light source requirement text of the light source instance are input into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text. Wherein, the light source setting prediction model is used to output the light source setting instruction that the rendering engine can recognize and realize the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes the light source attribute and its corresponding value. According to the light source setting instruction, the light source component of the light source instance in the virtual scene is controlled by the rendering engine. Through this application, the user does not need to master the relevant operating knowledge of the rendering engine light source component, and only needs to input the light source requirement text to quickly adjust the light source component in the virtual scene so that the virtual scene presents the lighting effect reflected by the light source requirement text, which greatly reduces the user's operation threshold and greatly improves the control efficiency of the light source component through the rendering engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A schematic flow chart of a method for controlling a light source component of a rendering engine provided in the first embodiment of the present application;

[0024] Figure 2 A schematic diagram of a flow chart of a method for controlling a light source component of a rendering engine provided in a second embodiment of the present application;

[0025] Figure 3 A schematic diagram of the training process of the light source setting prediction model provided in the third embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of a light source component control device of a rendering engine provided in a fourth embodiment of the present application;

[0027] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in the fifth embodiment of the present application.

[0028] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] In order to make the purpose, advantages and features of the present application clearer, the present application is described clearly and completely below in conjunction with the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments, and all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0030] It should be noted that, in the description of the present application, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance, and a specific order or sequence. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, the term "multiple" refers to two or more. The term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can be represented: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship. The terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In order to facilitate understanding of the technical solution of the present application, the relevant concepts involved in the present application are first introduced.

[0032] A rendering engine is a software system in computer graphics that converts 3D models, textures, lighting and other information into 2D images for display on monitors or other devices. A rendering engine is an indispensable tool in the fields of 3D animation, game development, virtual reality, architectural design, etc. It is responsible for handling all calculations and graphics processing tasks from scene description to final image generation. Common rendering engines include Unreal Engine (UE), Unity, etc.

[0033] Among them, Unreal Engine is the world's largest open source 3D rendering engine, which can provide a complete set of development tools for any user who needs real-time technology. From design visualization and movie-like experience to the production of high-quality games, it covers multiple fields such as PC, console, mobile device, VR and AR platforms, and it can also provide all the tools needed for project startup and delivery. Unreal Engine has powerful real-time rendering capabilities, supports high-quality lighting and shadows, rich material and shader systems, and is widely used in game development and virtual reality. The writing concept of Unreal Engine is to make content production and development programming easier. In the case of less program development content, abstract programs are used to freely create virtual environments, providing programmers with efficient modules and scalable development frameworks to create, test and complete various types of software testing work.

[0034] Light Actor: In the rendering engine, a light actor is an entity that can be directly placed and edited in the level, such as Directional Light Actor, Point Light Actor, SpotLight Actor, etc. These light actors themselves contain one or more light components and provide a graphical user interface to set various properties of the light source.

[0035] Light Instance: In the rendering engine, a light instance refers to an instance of a light Actor that exists in the level. When you place a light Actor in the level editor, you are actually creating a light instance. Each light Actor has a specific instantiation location and property settings in the level.

[0036] Light Component: In the rendering engine, a Light Component is a component inside a Light Actor that is responsible for the actual lighting calculations. Light Components can be attached to any Actor, not just Light Actor. This means that you can attach a Light Component to a Static Mesh Actor or other type of Actor to make it a light source. Light Components contain the specific properties of a light source, such as color, intensity, range, etc.

[0037] In order to solve the above problems, the present application provides a method for controlling a light source component of a rendering engine, a device for controlling a light source component of a rendering engine corresponding to the method, an electronic device that can implement the method for controlling a light source component of a rendering engine, and a computer-readable storage medium. The following embodiments are provided to describe the above method, device, electronic device, and computer-readable storage medium in detail.

[0038] In order to make the purpose and technical solution of the present application clearer and more intuitive, the method provided by the embodiment of the present application will be described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It is understood that the following embodiments may exist separately, and the following embodiments and features in the embodiments may be combined with each other when there is no conflict between the embodiments provided in the present application, and the same or similar content will not be repeated in different embodiments. In addition, the step sequence in the following method embodiments is only an example, not a strict limitation, and in some cases, the steps shown or described may be performed in a different order from this.

[0039] The present application provides a method, device, electronic device and computer-readable storage medium for controlling a light source component of a rendering engine. Specifically, the method for controlling a light source component of a rendering engine in one embodiment of the present application can be executed by a computer device, wherein the computer device can be a terminal or a server. The terminal can be a terminal device such as a smart phone, a tablet computer, a laptop computer, a touch screen, a game console, etc. The terminal can also include a client, which can be a game application client, a browser client carrying a game program, or an instant messaging client, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and basic cloud computing services such as big data and artificial intelligence platforms.

[0040] First embodiment

[0041] Next, combine Figure 1 , the light source component control method of the rendering engine provided by the first embodiment of the present application is described, Figure 1 A schematic flow chart of a method for controlling a light source component of a rendering engine provided in the first embodiment of the present application.

[0042] like Figure 1 As shown, the light source component control method of the rendering engine includes steps S101-S104:

[0043] S101, obtaining light source information of a light source instance in a virtual scene, wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information.

[0044] S102: Acquire a light source requirement text input by a user, where the light source requirement text is used to represent a light source effect requirement for a light source component in a virtual scene created by a rendering engine.

[0045] S103, inputting the light source information and the light source requirement text of the light source instance into the light source setting prediction model, and obtaining the light source setting instruction corresponding to the light source requirement text. The light source setting prediction model is used to output the light source setting instruction that the rendering engine can recognize and realize the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes the light source attributes and their corresponding values.

[0046] S104. According to the light source setting instruction, the light source component of the light source instance in the virtual scene is controlled by a rendering engine.

[0047] Next, steps 101 to S104 are described in detail.

[0048] In rendering engines (such as Unreal Engine, Unity and other 3D rendering engines), a light source Actor is a game object used to simulate lighting effects. A light source Actor is a special type of Actor. In the virtual scene, a light source Actor does not display any geometric shapes, but it affects other virtual objects in the virtual scene. Light sources can be static or dynamic. Place light sources in the virtual scene editor, and adjust properties such as position, intensity, and color as needed. Among them, lighting visual effects include but are not limited to simulating natural lighting, emphasizing scene elements, creating atmosphere effects, and achieving shadow effects. Light source Actor is responsible for generating lighting in the virtual scene to help render realistic lighting visual effects.

[0049] Common light source actor types include, but are not limited to: point light, spotlight, directional light, area light, and ambient light. Among them, point light: emits light evenly from a point to all directions. Spotlight: similar to a flashlight or stage spotlight, with a central axis and emits light within a certain angle range. Directional light: simulates the effect of sunlight, emitting parallel light from an infinite distance. Area light: usually a rectangular or circular plane light source that simulates a large area of ​​luminous surface. Ambient light: provides basic lighting for the scene, usually without clear directionality.

[0050] In an embodiment of the present application, before the user controls the light source component in the virtual scene through the rendering engine to achieve the desired lighting effect, the light source information of all light source instances currently existing in the virtual scene is obtained. The light source information includes light source attribute information and light source position information. Among them, the light source type includes but is not limited to direct light, point light source, spot light source, and rectangular light source. Different types of light source instances may have the same attributes or different attributes. Specifically, the core attributes of direct light include but are not limited to intensity, color, color temperature, and source angle. The core attributes of point light sources include but are not limited to intensity, color, attenuation radius, color temperature, volume scattering intensity, and indirect lighting intensity. The core attributes of spot light sources include but are not limited to intensity, color, attenuation radius, cone inner angle, cone outer angle, and color temperature. The core attributes of rectangular light sources include but are not limited to intensity, color, attenuation radius, source width, source height, light barrier angle, light barrier length, and color temperature.

[0051] Exemplarily, taking the rendering engine as Unreal Engine as an example, all light source instance information currently existing in the virtual scene created by Unreal Engine is obtained. The light source instances include the main light DirectionalLight, point light PointLight, spot light SpotLight, rectangular light RectLight, and sky light SkyLight. It should be noted that when there are multiple main lights, only the first main light source is retained. The light source attribute information of the light source instance, taking the main light DirectionalLight as an example, the light source attribute information of the main light DirectionalLight is as follows:

[0052] DirectionalLight = {

[0053] 'intensity':1000,

[0054] 'light_color':(1.0,1.0,1.0,1.0),

[0055] 'cast_shadows':True,

[0056] 'shadow resolution':1024,

[0057] 'direction':(0,0,1)}.

[0058] Above, the light source position information of all light source instances, the light source position information data format can be: [DirectionalLight: (0,0,0), PointLight: (328,658,206), SpotLight: (500,245,150), RectLight: (280,468,306), SkyLight: (400,550,302)].

[0059] In an embodiment of the present application, a light source requirement text input by a user is obtained, and the light source requirement text is used to characterize the user's demand for the light source effect of the light source component in the virtual scene created by the rendering engine. Among them, the light source requirement text is a natural language input by the user to describe the desired lighting effect and scene lighting atmosphere. For example, if the light source requirement text input by the user is "I want an outdoor landscape scene in the evening when the sun is about to set. I want the sunlight to be golden in hue and moderate in intensity to illuminate the entire scene, but not too strong. I want the color of the sky to reflect the feeling of the sunset, with obvious shadows on the ground and trees." For another example, the light source requirement text input by the user is "I need a forest adventure scene where the sun shines through the treetops during the day. I want the sunlight to be white and moderate in intensity, but not very strong. I hope to see some mottled light and shadow effects on the ground to simulate the lighting effect of sunlight penetrating the leaves."

[0060] In an embodiment of the present application, a light source setting prediction model is obtained, and the light source setting prediction model is used to output a light source setting instruction that can be recognized by a rendering engine and realizes a light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values. Next, the light source information and the light source requirement text of the light source instance are input into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text output by the light source setting prediction model.

[0061] In an optional implementation manner, a possible implementation manner of "inputting the light source information and the light source requirement text of the light source instance into the light source setting prediction model" in step S103 includes step S1031:

[0062] S1031. Based on a preset prompt template, the light source information and the light source requirement text of the light source instance are spliced ​​to obtain a target prompt, and the target prompt is input into a light source setting prediction model.

[0063] As mentioned above, the preset prompt template is a user-defined prompt template, which is used to splice the light source information and light source requirement text of the light source instance. For example, the preset prompt template can be: {all light source instance information in the virtual scene} + now you need to adjust the light source information + {light source requirement text entered by the user (that is, the natural language entered by the user to describe the desired lighting effect and scene lighting atmosphere)}, or it can be {atmosphere description text of the entire virtual scene} + {all light source instance information in the virtual scene} + now you need to adjust the light source information + {light source requirement text entered by the user}, which is only an example, and the embodiments of the present application do not impose any restrictions on this.

[0064] In the embodiment of the present application, based on the preset prompt template, the light source information and the light source requirement text of the light source instance are spliced ​​to obtain the target prompt. The target prompt is input into the light source setting prediction model. According to the target prompt, the light source setting prediction model can more easily grasp the light source information of all light source instances in the current virtual scene and understand the user's light source effect requirements, thereby prompting the light source setting prediction model to output a light source setting instruction that better meets the user's light source effect requirements.

[0065] For example, the target prompt is: This is a misty forest scene, all light source instance information, namely, light source attribute information and light source position information, (DirectionalLight = {

[0066] 'intensity':1000,

[0067] 'light_color':(1.0,1.0,1.0,1.0),

[0068] 'cast_shadows':True,

[0069] 'shadow resolution':1024,

[0070] 'direction':(0,0,1)},

[0071] PointLight = {……},

[0072] SpotLight = {……},

[0073] RectLight={……},

[0074] SkyLight = {...}

[0075] DirectionalLight:(0,0,0),

[0076] PointLight:(328,658,206),

[0077] SpotLight:(500,245,150),

[0078] RectLight:(280,468,306),

[0079] SkyLight:(400,550,302)]) Now you need to adjust the light source information. I need a forest adventure scene with sunlight shining through the tree canopy during the day. I want the sunlight to be white and moderate in intensity, but not very strong. I want to see some mottled light and shadow effects on the ground to simulate the lighting effect of sunlight penetrating through the leaves.

[0080] Input the target prompt into the light source setting prediction model to obtain the light source setting instruction output by the light source setting prediction model. Assuming that there are N light sources that need to be adjusted, the corresponding light source setting instruction can be as follows:

[0081] [data1={

[0082] 'intensity':2500,

[0083] 'light_color':(180,190,200),

[0084] 'source_angle':8,

[0085] 'source_soft_angle':16,

[0086] 'use_temperature':True,

[0087] 'temperature':7000,

[0088] 'cast_shadows':True,

[0089] 'indirect_lighting_intensity':1.3,

[0090] 'volumetric_scattering_intensity':2.5

[0091] },data2={…},

[0092] …

[0093] ,dataN={……}].

[0094] It should be noted that the lighting setting prediction model can batch output light source information for multiple light sources at one time, or it can output light source information for one light source instance multiple times to avoid the output being too long and truncated due to exceeding the token limit (maximum number of outputs). This can be done by limiting the maximum number of outputs in the target prompt using the lighting text requirement text.

[0095] In an embodiment of the present application, a light source setting instruction corresponding to the light source requirement text is obtained based on the above steps, and the light source component of the light source instance in the virtual scene is controlled by the rendering engine to realize the lighting effect corresponding to the light source requirement text in the virtual scene.

[0096] The light source component control method of the rendering engine provided in the embodiment of the present application obtains the light source information of all light source instances in the virtual scene. Wherein, the virtual scene is created by the rendering engine, and the light source information includes light source attribute information and light source position information. The input light source requirement text is obtained, and the light source requirement text is used to characterize the light source effect requirement for the light source component in the virtual scene created by the rendering engine. The light source information and the light source requirement text of the light source instance are input into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text. Wherein, the light source setting prediction model is used to output the light source setting instruction that the rendering engine can recognize and realize the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes the light source attribute and its corresponding value. According to the light source setting instruction, the light source component of the light source instance in the virtual scene is controlled by the rendering engine. The user of the present application does not need to master the relevant operating knowledge of the rendering engine light source component, and only needs to input the light source requirement text to quickly adjust the light source component in the virtual scene so that the virtual scene presents the lighting effect reflected by the light source requirement text, which greatly reduces the user's operation threshold and greatly improves the control efficiency of the light source component through the rendering engine.

[0097] Based on the above first embodiment, the light source component control method of the rendering engine provided by the first embodiment is further described below.

[0098] In an optional implementation manner, the light source component control method of the rendering engine provided in the first embodiment of the present application further includes step S105:

[0099] S105. According to the lighting parameter range of each lighting component in the rendering engine, the light source setting instruction corresponding to the light source requirement text is adjusted so that the numerical value corresponding to each light source attribute in the adjusted lighting setting instruction conforms to the numerical value range of each light source attribute in the rendering engine.

[0100] By adjusting the light setting instructions to ensure that the values ​​of the light source properties are within a reasonable range, you can avoid unreasonable or physically unrealistic lighting effects. For example, too high an intensity value may cause overexposure, while too low an intensity value may cause the scene to be too dark.

[0101] Second embodiment

[0102] Next, combine Figure 2 , a method for controlling a light source component of a rendering engine provided in the second embodiment of the present application is described. Figure 2 A schematic flow chart of a method for controlling a light source component of a rendering engine provided in the second embodiment of the present application.

[0103] like Figure 2 As shown, the light source component control method of the rendering engine includes steps S201-S210:

[0104] S201. Create a custom plug-in, where the custom plug-in provides a graphical user interface.

[0105] S202, integrating the custom plug-in into the rendering engine, wherein the integrated custom plug-in includes a control interface of the light source Actor; wherein the light source instance is an instance of the light source Actor.

[0106] S203. The custom plug-in obtains light source information of all light source instances in the virtual scene from the rendering engine through the control interface of the light source Actor.

[0107] S204: The custom plug-in receives the input light source requirement text through the graphical user interface.

[0108] As described above, through the graphical user interface, the light source requirement text manually input by the user or automatically input by other plug-ins can be received.

[0109] S205. The custom plug-in splices the light source information and the light source requirement text of the light source instance based on the preset prompt template, obtains the target prompt and sends the target prompt to the Python server.

[0110] S206. Integrate the light source setting prediction model into the Python server.

[0111] S207. The Python server inputs the target prompt into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text.

[0112] S208. The Python server sends a light source setting instruction corresponding to the light source requirement text to the custom plug-in.

[0113] S209. The custom plug-in adjusts the light source setting instructions corresponding to the light source requirement text according to the lighting parameter range of each lighting component in the rendering engine, so that the numerical values ​​corresponding to each light source attribute in the adjusted lighting setting instructions conform to the numerical value range of each light source attribute in the rendering engine.

[0114] S210. The custom plug-in controls the light source component of the light source instance in the virtual scene through the rendering engine according to the light source setting instruction.

[0115] As mentioned above, a custom plug-in developed for a rendering engine is a custom module that can be integrated into the rendering engine. It can contain code, resource files (such as textures, models, etc.), scripts, and other necessary files. The purpose of custom plug-ins is to enhance or supplement the default functions of the rendering engine, making it easier for developers to achieve specific goals or functions.

[0116] The embodiment of the present application provides a method for obtaining light source information of all light source instances in a virtual scene, creating a custom plug-in, and providing a graphical user interface. The light source requirement text input by the user is received through the graphical user interface, which can simplify the user's input process. The user does not need to remember complex commands or parameter formats, but only needs to enter the light source requirement text on the graphical user interface to characterize the user's requirements for the light source effect of the light source component in the virtual scene created by the rendering engine.

[0117] Integrate the custom plug-in into the rendering engine. The integrated custom plug-in includes the control interface of the light source Actor. In this way, the custom plug-in can directly obtain the light source information of all light source instances in the virtual scene from the rendering engine through the control interface of the light source Actor, without manually writing code or going deep into the internal settings of the engine to obtain it. This method lowers the user's operation threshold, allowing non-professional technicians to easily obtain the light source information of all light source instances in the virtual scene.

[0118] Based on the preset prompt template, the custom plug-in splices the light source information and light source requirement text of the light source instance to obtain the target prompt. According to the target prompt, the light source setting prediction model can more easily grasp the light source information of all light source instances in the current virtual scene and understand the user's light source effect requirements, thereby prompting the light source setting prediction model to output light source setting instructions that better meet the user's light source effect requirements. Subsequently, the custom plug-in sends the target prompt to the Python server.

[0119] The Python server inputs the target prompt into the light source setting prediction model, obtains the light source setting instruction corresponding to the light source requirement text, and then sends the light source setting instruction corresponding to the light source requirement text to the custom plug-in. The custom plug-in adjusts the light source setting instruction corresponding to the light source requirement text according to the lighting parameter range of each lighting component in the rendering engine, so that the value corresponding to each light source attribute in the adjusted lighting setting instruction conforms to the value range of each light source attribute in the rendering engine.

[0120] The custom plug-in controls the light source components of the light source instances in the virtual scene through the rendering engine according to the light source setting instructions corresponding to the light source requirement text. In the light source component control method of the entire rendering engine, the user only needs to enter the light source requirement text, without manually calculating the light source component setting parameters and manually controlling the light source components in the virtual scene one by one through the rendering engine, which greatly reduces the user's operating threshold and workload and improves the efficiency of light source control.

[0121] Third embodiment

[0122] Next, combine Figure 3 , the training method of the light source setting prediction model provided in the third embodiment of the present application is described, Figure 3 A schematic diagram of the training process of the light source setting prediction model provided in the third embodiment of the present application.

[0123] like Figure 3 As shown, the training method of the light source setting prediction model includes steps S301-S304:

[0124] S301. Obtain a training data set, wherein the training data set includes a plurality of sample pairs, and the sample pairs are composed of a light source setting description, a light source attribute, and an attribute value.

[0125] S302: Obtain a preset multimodal model, where the preset multimodal model has general text comprehension and visual perception capabilities.

[0126] As mentioned above, the preset multimodal models refer to machine learning models that can process and fuse data from different modalities. Modality refers to different forms of data, such as text, images, audio, video, etc. The goal of the multimodal model is to extract complementary information from these different modalities of data when processing them, and use this information to complete specific tasks, such as image description generation, sentiment analysis, question-answering systems, etc. The preset multimodal model can be the LLaVA-3 model, which is only an example, and the embodiments of the present application do not impose any restrictions on this.

[0127] S303, loading a preset multimodal model and freezing its weight matrix, introducing a first low-rank matrix and a second low-rank matrix, and obtaining a light source setting prediction model to be trained, wherein a matrix obtained by performing a dot multiplication of the first low-rank matrix and the transposed matrix of the second low-rank matrix has the same dimension as the weight matrix.

[0128] In an embodiment of the present application, a preset multimodal model is loaded and its weight matrix is ​​frozen. In order to realize matrix decomposition, a first low-rank matrix and a second low-rank matrix are introduced to reduce the computational complexity and storage requirements. The ranks of the first low-rank matrix and the second low-rank matrix are very low. The dimensions of the two low-rank matrices are usually much smaller than the dimensions of the weight matrix. Among them, a low-rank matrix refers to a matrix in mathematics, in which the rank (rank) refers to the number of the largest linearly independent groups in its row vector (or column vector). Low-rank matrices refer to matrices with relatively small ranks. In machine learning, low-rank matrices are often used to approximate high-dimensional matrices in order to reduce computational complexity and storage requirements. In LoRA (Low-Rank Adaptation, LoRA), low-rank matrices are used to fine-tune pre-trained large models to adapt to new tasks or data sets.

[0129] Exemplarily, assuming that the dimension of the weight matrix of the preset multimodal model is N*M, the dimension of the first low-rank matrix is ​​N*K, and the dimension of the second low-rank matrix is ​​M*K. The matrix obtained by performing dot multiplication of the transposed matrix of the first low-rank matrix and the second low-rank matrix has the same dimension as the weight matrix, namely N*M.

[0130] S304: input the training data set into the light source setting prediction model to be trained, update the first low-rank matrix and the second low-rank matrix, and obtain the light source setting prediction model after the training is completed.

[0131] The training method of the light source setting prediction model provided in the embodiment of the present application obtains a training data set, wherein the training data set includes multiple groups of sample pairs, and the sample pairs are composed of a light source setting description, a light source attribute, and an attribute value. A preset multimodal model is obtained, and the preset multimodal model has a general text comprehension ability and visual perception ability. The preset multimodal model is loaded and its weight matrix is ​​frozen, and a first low-rank matrix and a second low-rank matrix are introduced to obtain a light source setting prediction model to be trained. Among them, the matrix obtained by cross-multiplying the first low-rank matrix by the second low-rank matrix has the same dimension as the weight matrix. The training data set is input into the light source setting prediction model to be trained, and the first low-rank matrix and the second low-rank matrix are updated. After the training is completed, the light source setting prediction model is obtained.

[0132] In an embodiment of the present application, in the process of fine-tuning the preset multimodal model using a training data set, the weight matrix of the preset multimodal model is frozen and remains unchanged, and the first low-rank matrix and the second low-rank matrix are updated using a standard optimization algorithm (such as gradient descent), so that the fine-tuned preset multimodal model, on the basis of having general text comprehension and visual perception capabilities, gradually has the ability to output corresponding light source attributes and attribute values ​​according to the light source setting description.

[0133] In the embodiment of the present application, the preset multimodal model after introducing the first low-rank matrix and the second low-rank matrix is ​​called the light source setting prediction model to be trained. The light source setting prediction model to be trained is fine-tuned by LoRA using the training data set, and the first low-rank matrix and the second low-rank matrix are updated. After the training is completed, the light source setting prediction model is obtained. This allows the preset multimodal model to maintain good generalization ability while having the ability to output corresponding light source attributes and attribute values ​​according to the light source setting description, and also significantly reduces the demand for computing resources.

[0134] Based on the above-mentioned training method of the light source setting prediction model, optionally, step S3031 is further included after step S303, and step S3041 is further included after step S304. Step S3031 and step S3041 are respectively as follows:

[0135] S3031. Partition and store the weight matrix, the first low-rank matrix, and the second low-rank matrix.

[0136] S3041. Perform weight merging on the weight matrix, the first low-rank matrix, and the second low-rank matrix to obtain a target weight matrix.

[0137] In an embodiment of the present application, in the process of using the training data set to perform LoRA fine-tuning on the light source setting prediction model to be trained, the weight matrix, the first low-rank matrix, and the second low-rank matrix are partitioned and stored using deepspeed zero2, which means that each training node only saves a part of the state parameters of the model, thereby reducing memory usage.

[0138] In an embodiment of the present application, in step S304 "the training data set is input into the light source setting prediction model to be trained, the first low-rank matrix and the second low-rank matrix are updated, and the light source setting prediction model is obtained after the training is completed." After that, the weight matrix, the first low-rank matrix and the second low-rank matrix are weighted and merged to obtain the target weight matrix. Specifically, first, the intermediate matrix obtained by multiplying the transposed matrix of the second low-rank matrix by the first low-rank matrix is ​​obtained, and the sum of the intermediate matrix and the weight matrix of the original preset multimodal model is determined as the target weight matrix. The light source component control device of the rendering engine provided by the present application is described below, and the light source component control device of the rendering engine described below and the light source component control method of the rendering engine described above can be referenced to each other.

[0139] Figure 4 This is a schematic diagram of the structure of the light source component control device of the Unreal Engine provided in the fourth embodiment of the present application. Figure 4 As shown, the light source component control device 400 of the rendering engine includes: a first acquisition module 401, a second acquisition module 402, a prediction module 403 and a control module 404.

[0140] A first acquisition module is used to acquire light source information of all light source instances in a virtual scene; wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information;

[0141] A second acquisition module is used to acquire a light source requirement text input by a user in a graphical user interface, wherein the light source requirement text is used to represent the user's requirement for a light source effect of a light source component in a virtual scene created by a rendering engine;

[0142] A prediction module, used for inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model, and obtaining a light source setting instruction corresponding to the light source requirement text; wherein the light source setting prediction model is used for outputting a light source setting instruction that can be recognized by the rendering engine and realizes the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values;

[0143] A control module is used to control the light source components of the light source instance in the virtual scene through the rendering engine according to the light source setting instruction corresponding to the light source requirement text.

[0144] In an optional implementation manner, the device further includes a correction module, wherein the correction module is used to:

[0145] According to the lighting parameter range of each lighting component in the rendering engine, the light source setting instruction corresponding to the light source requirement text is adjusted so that the numerical value corresponding to each light source attribute in the adjusted lighting setting instruction conforms to the numerical value range of each light source attribute in the rendering engine.

[0146] In an optional implementation manner, the prediction module is further used for:

[0147] Based on a preset prompt template, the light source information of the light source instance and the light source requirement text are spliced ​​to obtain a target prompt, and the target prompt is input into a light source setting prediction model.

[0148] In an optional implementation manner, the second acquisition module is further used to:

[0149] Creating a custom plug-in, the custom plug-in providing a graphical user interface;

[0150] Integrate the custom plug-in into the rendering engine, wherein the integrated custom plug-in includes a control interface of the light source Actor; wherein the light source instance is an instance of the light source Actor;

[0151] The step of obtaining light source information of all light source instances in the virtual scene includes:

[0152] The custom plug-in obtains light source information of all light source instances in the virtual scene from the rendering engine through the control interface of the light source Actor;

[0153] The step of obtaining the input light source requirement text includes:

[0154] The custom plug-in receives the input light source requirement text through the graphical user interface;

[0155] The custom plug-in receives a light source setting instruction corresponding to the light source requirement text.

[0156] In an optional implementation manner, the prediction module is used to:

[0157] Building a Python server, and integrating the light source setting prediction model into the Python server;

[0158] The step of inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model to obtain a light source setting instruction corresponding to the light source requirement text includes:

[0159] The Python server inputs the light source information of the light source instance and the light source requirement text into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text;

[0160] The Python server sends a light source setting instruction corresponding to the light source requirement text to the custom plug-in.

[0161] In an optional implementation manner, the device further includes a training module, wherein the training module is specifically configured to:

[0162] Acquire a training data set, wherein the training data set includes a plurality of sample pairs, wherein the sample pairs are composed of a light source setting description, a light source attribute, and an attribute value;

[0163] Acquire a preset multimodal model, wherein the preset multimodal model has general text comprehension ability and visual perception ability;

[0164] Loading the preset multimodal model and freezing its weight matrix, introducing a first low-rank matrix and a second low-rank matrix, and obtaining a light source setting prediction model to be trained; wherein a matrix obtained by performing a dot multiplication of the first low-rank matrix and the transposed matrix of the second low-rank matrix has the same dimension as the weight matrix;

[0165] The training data set is input into the light source setting prediction model to be trained, the first low-rank matrix and the second low-rank matrix are updated, and the light source setting prediction model is obtained after the training is completed.

[0166] In an optional implementation manner, the training module is further used for:

[0167] Partitioning and storing the weight matrix, the first low-rank matrix, and the second low-rank matrix;

[0168] After inputting the training data set into the light source setting prediction model to be trained and updating the first low-rank matrix and the second low-rank matrix, the method further includes:

[0169] The weight matrix, the first low-rank matrix and the second low-rank matrix are weighted together to obtain a target weight matrix.

[0170] The light source component control device of the rendering engine provided in this embodiment can be used to execute the technical solution of the light source component control method embodiment of the rendering engine mentioned above. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0171] Figure 5 A hardware structure diagram of an electronic device provided in the fifth embodiment of the present application is shown in FIG. Figure 5 As shown, the electronic device 500 of this embodiment includes: a processor 501 and a memory 502;

[0172] Memory 502, used to store computer-executable instructions;

[0173] The processor 501 is used to execute the computer-executable instructions stored in the memory to implement the various steps executed by the light source component control method of the rendering engine in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0174] Optionally, the memory 502 may be independent or integrated with the processor 501 .

[0175] When the memory 502 is independently provided, the electronic device further includes a bus 503 for connecting the memory 502 and the processor 501 .

[0176] The sixth embodiment of the present application further provides a computer-readable storage medium, in which computer execution instructions are stored. When the processor executes the computer execution instructions, the technical solution corresponding to the light source component control method of the rendering engine in any of the above embodiments executed by the above electronic device is implemented.

[0177] The seventh embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the technical solution corresponding to the light source component control method of the rendering engine in any of the above embodiments.

[0178] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0180] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application.

[0181] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0182] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0183] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0184] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0185] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for controlling a light source component of a rendering engine, characterized in that: The method comprises: Acquire light source information of a light source instance in a virtual scene; wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information; Acquire input light source requirement text, where the light source requirement text is used to represent the light source effect requirement in a virtual scene created by a rendering engine; Inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model to obtain a light source setting instruction corresponding to the light source requirement text; wherein the light source setting prediction model is used to output a light source setting instruction that can be recognized by the rendering engine and realizes the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values; According to the light source setting instruction, the light source component of the light source instance in the virtual scene is controlled by the rendering engine.

2. The method according to claim 1, characterized in that The method further comprises: According to the lighting parameter range of each lighting component in the rendering engine, the light source setting instruction corresponding to the light source requirement text is adjusted so that the numerical value corresponding to each light source attribute in the adjusted lighting setting instruction conforms to the numerical value range of each light source attribute in the rendering engine.

3. The method according to claim 1, characterized in that The step of inputting the light source information of the light source instance and the light source requirement text into the light source setting prediction model comprises: Based on a preset prompt template, the light source information of the light source instance and the light source requirement text are spliced ​​to obtain a target prompt, and the target prompt is input into a light source setting prediction model.

4. The method according to claim 1, characterized in that: The method further comprises: Creating a custom plug-in, the custom plug-in providing a graphical user interface; Integrate the custom plug-in into the rendering engine, wherein the integrated custom plug-in includes a control interface of the light source Actor; wherein the light source instance is an instance of the light source Actor; The step of obtaining light source information of a light source instance in a virtual scene includes: The custom plug-in obtains light source information of the light source instance in the virtual scene from the rendering engine through the control interface of the light source Actor; The step of obtaining the input light source requirement text includes: The custom plug-in receives the input light source requirement text through the graphical user interface; The custom plug-in receives a light source setting instruction corresponding to the light source requirement text.

5. The method according to claim 4, characterized in that The method further comprises: Building a Python server, and integrating the light source setting prediction model into the Python server; The step of inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model to obtain a light source setting instruction corresponding to the light source requirement text includes: The Python server inputs the light source information of the light source instance and the light source requirement text into the light source setting prediction model to obtain the light source setting instruction corresponding to the light source requirement text; The Python server sends a light source setting instruction corresponding to the light source requirement text to the custom plug-in.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire a training data set, wherein the training data set includes a plurality of sample pairs, wherein the sample pairs are composed of a light source setting description, a light source attribute, and an attribute value; Acquire a preset multimodal model, wherein the preset multimodal model has general text comprehension ability and visual perception ability; Loading the preset multimodal model and freezing its weight matrix, introducing a first low-rank matrix and a second low-rank matrix, and obtaining a light source setting prediction model to be trained; wherein a matrix obtained by performing a dot multiplication of the first low-rank matrix and the transposed matrix of the second low-rank matrix has the same dimension as the weight matrix; The training data set is input into the light source setting prediction model to be trained, the first low-rank matrix and the second low-rank matrix are updated, and the light source setting prediction model is obtained after the training is completed.

7. The method according to claim 6, characterized in that After loading the preset multimodal model and freezing its weight matrix, and introducing the first low-rank matrix and the second low-rank matrix, the method further includes: Partitioning and storing the weight matrix, the first low-rank matrix, and the second low-rank matrix; After inputting the training data set into the light source setting prediction model to be trained and updating the first low-rank matrix and the second low-rank matrix, the method further includes: The weight matrix, the first low-rank matrix and the second low-rank matrix are weighted together to obtain a target weight matrix.

8. A light source component control device for a rendering engine, characterized in that: The device comprises: A first acquisition module is used to acquire light source information of a light source instance in a virtual scene; wherein the virtual scene is created by a rendering engine, and the light source information includes light source attribute information and light source position information; A second acquisition module is used to acquire an input light source requirement text, where the light source requirement text is used to represent the light source effect requirement in the virtual scene created by the rendering engine; A prediction module, used for inputting the light source information of the light source instance and the light source requirement text into a light source setting prediction model, and obtaining a light source setting instruction corresponding to the light source requirement text; wherein the light source setting prediction model is used for outputting a light source setting instruction that can be recognized by the rendering engine and realizes the light source effect corresponding to the light source requirement text according to the light source requirement text, and the light source setting instruction includes light source attributes and their corresponding values; A control module is used to control the light source components of the light source instance in the virtual scene through the rendering engine according to the light source setting instruction.

9. An electronic device, characterized in that: The electronic device comprises: Processor; and The memory is used to store a data processing program. After the electronic device is powered on and runs the program through the processor, the light source component control method of the rendering engine described in any one of claims 1-7 is executed.

10. A computer-readable storage medium, characterized in that: A data processing program is stored, and the program is run by a processor to execute the light source component control method of the rendering engine as described in any one of claims 1-7.

Images