Rendering method and device, chip, terminal, server and storage medium

Abstract

The invention provides a rendering method and device, a chip, a terminal, a server and a storage medium, and relates to the field of image processing.The method comprises the steps that in response to input operation, illumination requirement description information for a to-be-processed image is acquired; obtaining first illumination distribution information and physical information obtained by performing reverse rendering prediction on the to-be-processed image; performing illumination adjustment on the first illumination distribution information according to the illumination demand description information to obtain second illumination distribution information; and performing illumination rendering on the to-be-processed image based on the physical information and the second illumination distribution information to obtain and display a target image. Therefore, a user can flexibly express the specific expectation of the user on the image illumination effect in a natural language form, and corresponding illumination rendering processing is performed on the to-be-processed image according to the illumination demand description information input by the user, so that the technical threshold of manually adjusting illumination parameters by the user can be reduced, and the user experience is improved. And the diversified requirements of different scenes and user groups on the image illumination effect are met.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a rendering method, apparatus, chip, terminal, server, and storage medium. Background Technology

[0002] Image lighting editing technology is used to adjust, modify, and optimize the lighting effects in images to improve or enhance their visual performance. The demand for image lighting editing technology is particularly urgent in many application fields. For example, in film and television production, lighting can be adjusted according to different scenes and plots to create suitable atmosphere and visual effects, better serving the expression of the story and artistic presentation. In advertising design, optimizing the lighting of product images can highlight the characteristics and advantages of the product, enhancing the attractiveness and persuasiveness of the advertisement. In facial recognition, lighting processing of facial images helps improve the accuracy and robustness of recognition algorithms. Furthermore, with the rapid development of social media platforms and online content creation, the demand for image lighting editing from ordinary users is constantly growing. Users hope to easily beautify and adjust the lighting of their photos to obtain higher-quality visual effects and enhance the experience and dissemination of content sharing. Summary of the Invention

[0003] This application proposes a rendering method, apparatus, chip, terminal, server, and storage medium to at least partially solve one of the technical problems in the related art.

[0004] One embodiment of this application proposes a rendering method, comprising: in response to an input operation, acquiring illumination requirement description information for an image to be processed; acquiring first illumination distribution information and physical information obtained by inverse rendering prediction of the image to be processed; adjusting the first illumination distribution information according to the illumination requirement description information to obtain second illumination distribution information; and performing illumination rendering on the image to be processed based on the physical information and the second illumination distribution information to obtain and display a target image.

[0005] Another embodiment of this application proposes a different rendering method, including: receiving an image to be processed sent by a terminal, and performing reverse rendering prediction on the image to be processed to obtain physical information and first illumination distribution information required for rendering the image to be processed; sending the physical information and the first illumination distribution information to the terminal; wherein, the physical information and the first illumination distribution information are used by the terminal to integrate illumination requirement description information, perform illumination rendering on the image to be processed, and obtain and display a target image.

[0006] Another embodiment of this application proposes a chip, comprising: a central processing unit (CPU) configured to acquire physical information and first illumination distribution information obtained by reverse rendering an image to be processed, and to send the physical information to a graphics processing unit (GPU), and to send the first illumination distribution information to a neural network processing unit (NPU); the NPU configured to adjust the first illumination distribution information according to input illumination requirement description information to obtain second illumination distribution information, and to send the second illumination distribution information to the GPU; the GPU configured to perform illumination rendering on the image to be processed according to the physical information and the second illumination distribution information to obtain a target image.

[0007] In another aspect, this application proposes a terminal including the chip and image sensor proposed in the foregoing embodiments, wherein the image sensor is used to capture an input image in response to a shooting operation.

[0008] In another aspect, this application proposes a server, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the rendering method described in the other aspect above.

[0009] Another embodiment of this application proposes a rendering apparatus, comprising: a first acquisition module, configured to acquire illumination requirement description information for an image to be processed in response to an input operation; a second acquisition module, configured to acquire first illumination distribution information and physical information obtained by inverse rendering prediction of the image to be processed; an adjustment module, configured to adjust the first illumination distribution information according to the illumination requirement description information to obtain second illumination distribution information; and a rendering module, configured to perform illumination rendering on the image to be processed based on the physical information and the second illumination distribution information to obtain and display a target image.

[0010] Another embodiment of this application proposes a rendering apparatus, comprising: a prediction module, configured to receive an image to be processed sent by a terminal, and perform reverse rendering prediction on the image to be processed to obtain physical information and first illumination distribution information required for rendering the image to be processed; and a sending module, configured to send the physical information and the first illumination distribution information to the terminal; wherein the physical information and the first illumination distribution information are used by the terminal to integrate illumination requirement description information to perform illumination rendering on the image to be processed, thereby obtaining and displaying a target image.

[0011] In another aspect of this application, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the rendering method as described in the foregoing aspect, or, when executed, implement the rendering method as described in the foregoing other aspect.

[0012] Another aspect of this application provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the rendering method as described in the foregoing aspect, or, when executed, implements the rendering method as described in the foregoing aspect.

[0013] The rendering method, apparatus, chip, terminal, server, and storage medium proposed in this application allow users to flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, and overall atmosphere. In this application, based on the lighting requirement description information input by the user in natural language, the corresponding lighting rendering processing is performed on the image to be processed. This not only reduces the technical threshold for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances the user's freedom and personalized expression ability in image creation, and can better meet the diverse needs of different scenarios and user groups for image lighting effects.

[0014] This application, by introducing the physical and lighting distribution information required for rendering the image to be processed, enables lighting modeling based on real-world optical laws, avoiding unreasonable lighting changes or shadow distortions, thereby significantly improving the realism and visual credibility of the rendered image. Furthermore, by combining user-input lighting requirement descriptions, it is possible to accurately match the user's subjective lighting intentions with the physical lighting model, achieving efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for rapid image enhancement but also supports professional users in fine-grained control of lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0017] Figure 1 A flowchart illustrating a rendering method provided for an exemplary embodiment of this application;

[0018] Figure 2A schematic flowchart of yet another rendering method provided for an exemplary embodiment of this application;

[0019] Figure 3 A schematic flowchart illustrating another rendering method provided for an exemplary embodiment of this application;

[0020] Figure 4 A flowchart illustrating another rendering method provided for an exemplary embodiment of this application;

[0021] Figure 5 A flowchart illustrating another rendering method provided for an exemplary embodiment of this application;

[0022] Figure 6 A schematic diagram illustrating a method for predicting physical information provided in an exemplary embodiment of this application;

[0023] Figure 7 A schematic diagram of the lighting processing flow provided for an exemplary embodiment of this application;

[0024] Figure 8 A schematic diagram of a chip structure provided for an exemplary embodiment of this application;

[0025] Figure 9 A schematic diagram of another chip structure provided for an exemplary embodiment of this application;

[0026] Figure 10 A schematic diagram of the structure of a rendering apparatus provided for an exemplary embodiment of this application;

[0027] Figure 11 A schematic diagram of another rendering apparatus provided for an exemplary embodiment of this application;

[0028] Figure 12 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] The development of computer graphics has provided a solid theoretical foundation and technical support for image lighting editing technology. By constructing physically based lighting models, the interaction between light sources and objects, as well as their reflection behavior, can be simulated relatively accurately, thereby achieving high-quality rendering and adjustment of image lighting effects. For example, in the process of 3D modeling and rendering, by precisely setting parameters such as the position, color, and intensity of light sources, virtual scenes and objects with different lighting characteristics can be generated, providing important methodological theories and technical references for image lighting editing technology.

[0031] In related technologies, lighting editing typically involves extracting feature point information of objects in an image and matching it with a preset model. If the two match successfully, the physical information of the preset model is mapped to the image space, thereby editing the brightness of the image and generating shadow information corresponding to the object.

[0032] However, this approach relies on the existence of preset models; lighting editing can only be effectively performed when objects in the image match the preset models. When objects not included in the preset model library appear in the image, the system will be unable to complete the match, resulting in lighting editing failure. Furthermore, even if an object matches a preset model, modeling errors often exist between the preset model and the real scene. These errors may further affect the accuracy of lighting editing, causing image distortion and reducing the quality and realism of the final image.

[0033] Therefore, in view of at least one of the problems existing in the above-mentioned related technologies, this application proposes a rendering method, apparatus, chip, terminal, server and storage medium.

[0034] The rendering method, apparatus, chip, terminal, server, and storage medium of this application are described below with reference to the accompanying drawings.

[0035] Figure 1 This is a flowchart illustrating a rendering method provided for an exemplary embodiment of this application.

[0036] It should be noted that the rendering method of this application embodiment can be applied to a rendering device. In some possible embodiments, the rendering device can be configured in a terminal or chip so that the terminal or chip can perform rendering functions. In addition, in some possible embodiments, the rendering device can also be software in the terminal, etc.

[0037] In any embodiment of this application, the chip can be integrated into a terminal. The chip includes a Central Processing Unit (CPU), an Image Signal Processor (ISP), a Graphics Processing Unit (GPU), a Neural Processing Unit (NPU), an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a System-on-Chip (SOC), a Reduced Instruction Set Computer (RISC), etc., which will not be listed individually here.

[0038] In this context, a terminal is a user-side entity used to receive or transmit signals, such as a mobile phone. A terminal can also be called a terminal device, user equipment (UE), mobile station (MS), or mobile terminal (MT). Terminals can be communication-enabled vehicles, smart cars, mobile phones, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminals, augmented reality (AR) terminals, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments in this application do not limit the specific technology or device form used in the terminal.

[0039] For ease of explanation, the following description will use the terminal as the execution subject of this rendering method as an example.

[0040] like Figure 1 As shown, the rendering method may include the following steps S101 to S104:

[0041] Step S101: In response to the input operation, obtain the illumination requirement description information for the image to be processed.

[0042] The image to be processed refers to the image that needs to be rendered with lighting. It should be noted that the method of acquiring the image to be processed in this application embodiment is not limited. For example, the image to be processed can be an image captured by an image sensor, that is, the captured image to be processed is acquired in response to the shooting operation; or, the image to be processed can be an image currently edited by the user; or, the image to be processed can be an image to be shared by the user; or, the image to be processed can be an image collected online by the user; or, the image to be processed can be an image artificially synthesized by the user, etc.

[0043] The shooting operations include previewing, taking photos, and recording videos.

[0044] The triggering methods for shooting operations include, but are not limited to: physical button triggering, gesture recognition triggering, voice command triggering, and biometric triggering. Physical button triggering refers to triggering the camera to take a picture by pressing a physical button (such as the side button or home button on the phone). Gesture recognition triggering refers to triggering the camera to take a picture by recognizing user gestures through the camera or sensors. Voice command triggering refers to triggering the camera to take a picture by using a voice command (such as "open the camera"). Biometric triggering refers to triggering the camera to take a picture by recognizing biometric features (such as facial features, voice features, fingerprint features, etc.).

[0045] The image to be processed can be an original image captured or photographed by an image sensor (or camera), or it can be a new image obtained by image processing of the original image. This application embodiment does not limit this.

[0046] The images to be processed include, but are not limited to: preview images, captured images, and recorded video images. The cameras include, but are not limited to: front-facing cameras and rear-facing cameras.

[0047] The lighting requirement description information refers to the specific requirements of the user, expressed in natural language, for the lighting effects in the image to be processed. This lighting requirement description information is used to convey the user's specific expectations for the lighting effects in the image to be processed or the shooting scene.

[0048] For example, lighting requirement description information can be used to indicate at least one of the following lighting-related edits:

[0049] The first item is brightness adjustment information. For example, a user might describe it as, "Please brighten the image."

[0050] The second item is local lighting enhancement information. For example, a user description might be: "Highlight foreground figures and darken background objects."

[0051] The third item is information on shadow and highlight control. For example, a user description reads: "Reduce strong shadows in images, especially on faces."

[0052] The fourth item is specific light source simulation information. For example, a user description: "Add a sidelight effect to this image, like sunlight at dusk."

[0053] The fifth item is color temperature adjustment information. For example, a user might describe: "This night scene image looks too cold. Can you make it look warmer?"

[0054] The sixth item is information on ambient light reflection and scattering. For example, a user might describe: "I want the water in the image to reflect the color of the sky."

[0055] The seventh item is information on lighting and shadow effects. For example, a user description might be: "To add some mystery to the shooting scene, such as edge lighting or soft halo."

[0056] The eighth item is dynamic lighting change information. For example, a user might describe: "I want this image to show a gradual lighting effect from day to dusk."

[0057] The ninth item is conveying information through emotional atmosphere. For example, a user described it as: "Making images look more tense, such as using cool-toned bright lights and dark shadows to create a suspenseful atmosphere."

[0058] It should be noted that the above examples of lighting requirement description information are merely illustrative, but this application is not limited thereto. Users can input the required lighting requirement description information according to their actual lighting editing or adjustment needs, and this application embodiment does not impose any restrictions on this.

[0059] In this embodiment of the application, upon detecting a user-triggered input operation, the user's illumination requirement description information for the image to be processed can be obtained. The input methods for the illumination requirement description information include, but are not limited to: touch input (such as swiping, clicking, etc.), keyboard input, and voice input.

[0060] Step S102: Obtain the first illumination distribution information and physical information obtained by reverse rendering prediction of the image to be processed.

[0061] Inverse rendering refers to inferring the physical properties and lighting conditions of the shooting scene from the image to be processed.

[0062] The first illumination distribution information refers to the illumination distribution information required for rendering the image to be processed. For example, the first illumination distribution information may include the distribution information of various types of illumination components; wherein, the illumination components include, but are not limited to, ambient light, diffuse light, and specular light, etc., and the distribution information is used to indicate the light source position and / or light source color of the corresponding illumination component.

[0063] The presentation formats of distribution information include, but are not limited to, text and image formats. Taking image format as an example, the distribution information of each illumination component can include two RGB format images (R (Red): representing the red channel, controlling the intensity of red in the image, with an intensity value range of 0-255; G (Green): representing the green channel, controlling the intensity of green in the image, with an intensity value range of 0-255; B (Blue): representing the blue channel, controlling the intensity of blue in the image, with an intensity value range of 0-255). The three color channels (R, G, B channels) in one image are used to describe the position of the light source, and the three color channels in the other image are used to describe the color of the light source.

[0064] Physical information refers to the physical properties required for rendering the image to be processed. For example, physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction.

[0065] Scene elements include, but are not limited to, objects, people, animals, and scenery.

[0066] In other words, physical information is a set of data that accurately simulates the spatial position, material properties, surface orientation, and lighting interaction effects of scene elements in the image capture and rendering scene. It includes data such as depth, albedo (used to indicate the albedo of each pixel in the image to be processed, where albedo is used to indicate the texture and color of the corresponding pixel, where the color of each pixel refers to the color of the scene element to which the pixel belongs, that is, the original color before image post-processing (such as lighting processing, adding shadows and special effects, etc.), normal, and specular, which can reflect the objective physical properties of scene elements.

[0067] Among them, depth belongs to the spatial position dimension, which reflects the spatial relationship of scene elements in the shooting scene, such as the front and back layers and the distance from the camera; albedo belongs to the surface material dimension, which reflects the texture and color of scene elements and is closely related to material properties; normal and specular reflection are related to the lighting interaction dimension, which determine the direction and effect of light reflection on the surface of scene elements.

[0068] In the embodiments of this application, the first illumination distribution information may be obtained by the terminal through reverse rendering prediction of the image to be processed, or it may be obtained by the server through reverse rendering prediction of the image to be processed and sent to the terminal. The embodiments of this application do not limit this.

[0069] Similarly, the physical information can be obtained by the terminal through reverse rendering prediction of the image to be processed, or it can be obtained by the server through reverse rendering prediction of the image to be processed and sent to the terminal. This application embodiment does not limit this. For example, deep learning technology can be used to perform reverse rendering prediction of the image to be processed to obtain the physical properties in each dimension or modality, such as depth, albedo, normal, and specular reflection.

[0070] Step S103: Adjust the first illumination distribution information according to the illumination demand description information to obtain the second illumination distribution information.

[0071] The second illumination distribution information is used to indicate the illumination distribution information required by the user. For example, the second illumination distribution information may also include the distribution information of various types of illumination components, wherein the distribution information is used to indicate the light source position and / or light source color of the corresponding illumination component.

[0072] The presentation format of the second illumination distribution information is the same as that of the first illumination distribution information.

[0073] In this embodiment, the terminal can adjust the lighting based on the first lighting distribution information required for rendering the image to be processed, according to the lighting requirement description information input by the user, to obtain the second lighting distribution information.

[0074] As an example, deep learning techniques, such as large language models (LLMs), can be used to adjust the first lighting distribution information required for rendering the image to be processed based on the user's input description of lighting requirements, thereby obtaining the second lighting distribution information.

[0075] For example, a large model or LLM can perform intent recognition on the lighting requirement description information to obtain the user's lighting intent, and based on the lighting intent, update the first lighting distribution information required for rendering the image to be processed to obtain the second lighting distribution information.

[0076] Step S104: Based on physical information and second illumination distribution information, perform illumination rendering on the image to be processed to obtain and display the target image.

[0077] In this embodiment of the application, the image to be processed can be rendered with illumination based on the physical information required for rendering the image to be processed and the second illumination distribution information to obtain the target image and display the target image.

[0078] For example, deep learning techniques, such as lighting rendering models, can be used to perform lighting rendering on the image to be processed based on physical information and second lighting distribution information to obtain the target image.

[0079] The rendering method of this application embodiment allows users to flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, overall atmosphere, and other visual characteristics. In this application, based on the lighting requirement description information input by the user in natural language, the corresponding lighting rendering processing is performed on the image to be processed. This not only reduces the technical threshold for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances the user's freedom and personalized expression ability in image creation, and can better meet the diverse needs of different scenarios and user groups for image lighting effects.

[0080] This application, by introducing the physical and lighting distribution information required for rendering the image to be processed, enables lighting modeling based on real-world optical laws, avoiding unreasonable lighting changes or shadow distortions, thereby significantly improving the realism and visual credibility of the rendered image. Furthermore, by combining user-input lighting requirement descriptions, it is possible to accurately match the user's subjective lighting intentions with the physical lighting model, achieving efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for rapid image enhancement but also supports professional users in fine-grained control of lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0081] As one possible implementation method, Figure 2 A flowchart illustrating yet another rendering method provided for an exemplary embodiment of this application.

[0082] It should be noted that the rendering method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this application do not limit this.

[0083] like Figure 2As shown, the rendering method may include the following steps S201 to S206:

[0084] Step S201: In response to the input operation, obtain the illumination requirement description information for the image to be processed.

[0085] Step S202: Obtain the first illumination distribution information and physical information obtained by reverse rendering prediction of the image to be processed.

[0086] It should be noted that the explanations of steps S201 to S202 can be found in the relevant descriptions in any embodiment of this application, and will not be repeated here.

[0087] In any embodiment of this application, the illumination requirement description information can be obtained using any of the following methods:

[0088] The first step involves responding to user input via the voice input control to obtain the user's description of the lighting requirements for the image to be processed. The voice input control is used to invoke the large model and receive voice input commands.

[0089] The second step involves responding to user input via a text input control to obtain the lighting requirements manually entered by the user for the image to be processed. The text input control is used to invoke the large model and receive text input instructions.

[0090] The third step involves responding to user input to the interactive controls and obtaining the user's description of lighting requirements for the image to be processed within the interactive controls. These interactive controls are used to interact with the larger model.

[0091] For example, interactive controls include, but are not limited to: dialog windows, text fields, rich text editors, drop-down lists (allowing users to select one or more options from a predefined list of options as input lighting requirement description information), etc.

[0092] In summary, users can use different methods to input lighting requirement descriptions, which can improve the flexibility and applicability of this method.

[0093] Step S203: Obtain the prompt template; wherein, the prompt template is used to indicate the lighting adjustment task to be performed on the large model.

[0094] The prompt template can be pre-configured, or it can be dynamically updated according to actual application needs; this embodiment of the application does not impose any limitations on this. The prompt template is used to indicate the task information to be performed on the large model, namely, the lighting adjustment task.

[0095] Step S204: Update the prompt template based on the lighting requirement description information and the first lighting distribution information to obtain the prompt information.

[0096] It should be noted that the explanation of the first illumination distribution information in the foregoing embodiments also applies to this embodiment, and will not be repeated here.

[0097] For example, the lighting requirement description information and the first lighting distribution information can be filled into the corresponding positions in the prompt template to obtain the prompt information.

[0098] Step S205: Call the large model to perform a lighting adjustment task on the prompt information to obtain the second lighting distribution information output by the large model.

[0099] The large model can be a local model on the terminal or a large model on the server side; this application embodiment does not limit this.

[0100] In this embodiment, the terminal can call a large model to process the prompt information and obtain the second illumination distribution information output by the large model.

[0101] Step S206: Based on physical information and second illumination distribution information, perform illumination rendering on the image to be processed to obtain and display the target image.

[0102] It should be noted that the explanation of step S206 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0103] The rendering method of this application embodiment uses prompt information as prior information to indicate the task information to be executed by the large model, which can improve the prediction accuracy of the large model, that is, improve the generation quality of the second illumination distribution information.

[0104] As one possible implementation method, Figure 3 A flowchart illustrating another rendering method provided for an exemplary embodiment of this application.

[0105] It should be noted that the rendering method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this application do not limit this.

[0106] like Figure 3 As shown, the rendering method may include the following steps S301 to S306:

[0107] Step S301: In response to the input operation, obtain the illumination requirement description information for the image to be processed.

[0108] It should be noted that the explanation of step S301 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0109] Step S302: Send the image to be processed to the server.

[0110] As an example, the terminal can send the image to be processed to the server in plaintext.

[0111] As another example, to enhance data transmission security, the terminal can also send the image to be processed to the server in encrypted form. That is, the terminal can use an encryption algorithm to encrypt the image to be processed, obtain encrypted data, and send this encrypted data to the server. Correspondingly, after receiving the encrypted data, the server can decrypt it to recover the image to be processed.

[0112] The encryption algorithm can be an encryption algorithm negotiated between the terminal and the server.

[0113] Step S303: Receive compressed data sent by the server; wherein, the compressed data is obtained by the server performing reverse rendering prediction on the image to be processed to obtain physical information and first illumination distribution information, and compressing the physical information and first illumination distribution information.

[0114] In this embodiment of the application, after the server obtains the image to be processed, it can use deep learning technology to perform reverse rendering prediction on the image to be processed, obtain the physical information and the first illumination distribution information required for rendering the image to be processed, compress the physical information and the first illumination distribution information to obtain compressed data, and send the compressed data to the terminal.

[0115] Step S304: Decompress the compressed data to obtain physical information and first illumination distribution information.

[0116] In this embodiment of the application, after receiving the compressed data sent by the server, the terminal can decompress the compressed data to obtain the physical information and the first illumination distribution information required for rendering the image to be processed.

[0117] Step S305: Adjust the first illumination distribution information according to the illumination demand description information to obtain the second illumination distribution information.

[0118] Step S306: Based on physical information and second illumination distribution information, perform illumination rendering on the image to be processed to obtain and display the target image.

[0119] It should be noted that the explanations of steps S305 to S306 can be found in the relevant descriptions in any embodiment of this application, and will not be repeated here.

[0120] The rendering method of this application utilizes a server with integrated powerful computing resources to perform inverse rendering prediction on the image to be processed, obtaining the physical information and first illumination distribution information required for rendering the image. This reduces the computational burden on the terminal, improves the terminal's operating efficiency and response speed, and provides users with a smoother user experience. Furthermore, compressing the data to be transmitted (i.e., the physical information and the first illumination distribution information) before data transmission can reduce network resource overhead and improve transmission efficiency.

[0121] As one possible implementation method, Figure 4 A flowchart illustrating another rendering method provided for an exemplary embodiment of this application.

[0122] It should be noted that the rendering method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this application do not limit this.

[0123] like Figure 4 As shown, the rendering method may include the following steps S401 to S407:

[0124] Step S401: In response to the input operation, obtain the illumination requirement description information for the image to be processed.

[0125] Step S402: Obtain the first illumination distribution information and physical information obtained by reverse rendering prediction of the image to be processed.

[0126] Step S403: Based on the illumination demand description information, adjust the first illumination distribution information to obtain the second illumination distribution information.

[0127] Step S404: Based on physical information and second illumination distribution information, perform illumination rendering on the image to be processed to obtain and display the target image.

[0128] It should be noted that the explanations of steps S401 to S404 can be found in the relevant descriptions in any embodiment of this application, and will not be repeated here.

[0129] Step S405: In response to the first user operation, save the target image; wherein the first user operation is used to indicate that the lighting effect of the target image meets the lighting adjustment requirements of the target object, and the target object includes the trigger object of the input operation.

[0130] In this embodiment of the application, when the target object (i.e., the user) triggers the first user operation, the target image can be saved.

[0131] For example, the display interface may show "Accept", "Adopt", "Satisfied" or "Save" buttons, as well as "Reject", "Dissatisfied", "Update" or "Delete" buttons. When the user clicks the "Accept", "Adopt", "Satisfied" or "Save" button, it can be determined that the user has triggered the first user operation. At this time, the target image can be saved.

[0132] Step S406: In response to the second user operation, reacquire the illumination requirement description information; wherein the second user operation is used to indicate that the illumination effect of the target image does not meet the illumination adjustment requirements.

[0133] In this embodiment of the application, if the target object (i.e., the user) triggers a second user operation, the target image can be discarded and the user's re-entered description of the lighting requirements can be obtained.

[0134] Step S407: Based on the newly acquired lighting requirement description information, re-render the image to be processed until the rendered image meets the lighting adjustment requirements, and then save the rendered image.

[0135] In this embodiment of the application, the image to be processed can be re-rendered based on the re-acquired lighting requirement description information (the implementation principle is similar to that of steps S103 to S104, and will not be described in detail here) until the rendered image to be processed meets the lighting adjustment requirements of the target object, and then the rendered image to be processed is saved.

[0136] Taking the above example again, when the user clicks the "Reject", "Dissatisfied", "Update" or "Delete" button, it can be determined that the user has triggered a second user operation. At this time, the user can re-enter the lighting requirement description information, and according to the re-entered lighting requirement description information, the image to be processed is re-rendered until the rendered image to be processed meets the lighting adjustment requirements, and then the rendered image to be processed is saved.

[0137] The rendering method of this application embodiment can re-acquire the user's input lighting requirement description information when the target image obtained by lighting rendering does not meet the user's lighting adjustment requirements, and re-render the image to be processed based on the lighting requirement description information to meet the user's personalized lighting adjustment requirements and improve the user's shooting experience.

[0138] The above are various method embodiments executed by the terminal. This application also provides a rendering method executed by the server.

[0139] Figure 5 A flowchart illustrating another rendering method provided for an exemplary embodiment of this application.

[0140] like Figure 5 As shown, the rendering method may include the following steps S501 to S503:

[0141] Step S501: Receive the image to be processed sent by the terminal.

[0142] The image to be processed refers to the image that needs to be rendered with lighting.

[0143] It should be noted that the explanation of the image to be processed in the foregoing embodiments also applies to this embodiment, and will not be repeated here.

[0144] Step S502: Perform reverse rendering prediction on the image to be processed to obtain the physical information and first illumination distribution information required for rendering the image to be processed.

[0145] It should be noted that the explanations of physical information and first illumination distribution information in the foregoing embodiments also apply to this embodiment, and will not be repeated here.

[0146] In this embodiment of the application, the server may use deep learning technology to perform reverse rendering prediction on the image to be processed, and obtain the physical information and first illumination distribution information required for rendering the image to be processed.

[0147] In any embodiment of this application, in order to improve the accuracy of reverse rendering prediction, the server can perform at least one round of iterative reverse rendering prediction process based on the image to be processed, and use the physical information and illumination distribution information output by the last round of iterative reverse rendering prediction process as the physical information and first illumination distribution information required for rendering the image to be processed.

[0148] As an example, the server can use steps A through E to perform the first iteration of the reverse rendering prediction process in at least one iteration of the reverse rendering prediction process:

[0149] Step A: Perform reverse rendering prediction on the image to be processed to obtain the physical information and illumination distribution information output by the first round of reverse rendering prediction process.

[0150] In this application, the server can use deep learning technology to perform reverse rendering prediction on the image to be processed, and obtain the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process.

[0151] For example, physical information includes physical information of multiple dimensions or modalities, such as normal, albedo, depth, specular, etc. In this application, the server can use a neural network corresponding to each modality to perform inverse rendering prediction on the image to be processed, thereby obtaining the physical information of each modality. Similarly, the server can use a separate neural network to predict the illumination distribution information required for rendering the image to be processed.

[0152] Step B: Based on the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process, perform illumination rendering on the image to be processed to obtain the rendered image output by the first round of iterative reverse rendering prediction process.

[0153] Step C: Calculate the difference (including but not limited to the mean square error) between the rendered image output by the first round of reverse rendering prediction process and the image to be processed, and determine whether the difference is less than the set difference threshold. If yes, proceed to step D; otherwise, proceed to step E.

[0154] Step D: If the difference between the rendered image output by the first round of reverse rendering prediction process and the image to be processed is less than the set difference threshold, the iterative reverse rendering prediction process ends.

[0155] Step E: If the difference between the rendered image output by the first round of reverse rendering prediction process and the image to be processed is greater than or equal to the set difference threshold, execute the next round of reverse rendering prediction process (i.e., the second round of reverse rendering prediction process).

[0156] As another example, the server can use steps a to f to execute at least one round of iterative reverse rendering prediction process, such as the i-th (i is a positive integer greater than 1) round of iterative reverse rendering prediction process:

[0157] Step a: Determine if i is less than or equal to the set number of iterations. If yes, proceed to step b; otherwise, proceed to step c and subsequent steps.

[0158] The number of iterations is set to the maximum number of iterations preset in advance.

[0159] Step b: If i is greater than the set number of iterations, or if the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed is less than the set difference threshold, the iterative reverse rendering prediction process ends.

[0160] Step c: When i is less than or equal to the set number of iterations, the image to be processed can be reverse rendered based on the physical information and illumination distribution information output by the reverse rendering prediction process of the first i-1 iterations, so as to obtain the physical information and illumination distribution information output by the reverse rendering prediction process of the i-th iteration.

[0161] In this application, the server can employ deep learning technology to perform reverse rendering prediction on the image to be processed based on the physical information and illumination distribution information output from the first i-1 iterations of the reverse rendering prediction process, thereby obtaining the physical information and illumination distribution information output from the i-th iteration of the reverse rendering prediction process. That is, the image to be processed, the physical information and illumination distribution information output from the first i-1 iterations of the reverse rendering prediction process, can be input into a neural network for reverse rendering prediction to obtain the physical information and illumination distribution information output from the i-th iteration of the reverse rendering prediction process.

[0162] As an example, taking the server-side use of a neural network (such as the U-Shaped Neural Network) corresponding to each modality to perform inverse rendering prediction on the image to be processed, and obtain the physical information of each modality, the server can use, for example... Figure 6 The method shown predicts the physical information of the corresponding mode in the current iteration based on the physical information of the corresponding mode output from the previous iteration and the image to be processed.

[0163] in, Figure 6 The image to be processed is in RGB format, as an example.

[0164] Step d: Based on the physical information and illumination distribution information output by the i-th iteration of the reverse rendering prediction process, perform illumination rendering on the image to be processed to obtain the rendered image output by the i-th iteration of the reverse rendering prediction process.

[0165] Step e: Calculate the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed, and determine whether the difference is less than the set difference threshold. If yes, proceed to step b; otherwise, proceed to step f.

[0166] Step f: If the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed is greater than or equal to the set difference threshold, execute the next iteration of the reverse rendering prediction process (i.e., the i+1-th iteration of the reverse rendering prediction process).

[0167] In summary, employing at least one round of iterative reverse rendering prediction to update the physical and illumination distribution information required for rendering the image to be processed can improve the accuracy and reliability of the prediction of physical and illumination distribution information.

[0168] Step S503: Send physical information and first illumination distribution information to the terminal; wherein, the physical information and first illumination distribution information are used by the terminal to integrate illumination demand description information, perform illumination rendering on the image to be processed, and obtain and display the target image.

[0169] In this embodiment, the server can send the physical information and first illumination distribution information required for rendering the image to be processed to the terminal. Correspondingly, after receiving the physical information and the first illumination distribution information, the terminal can adjust the illumination based on the illumination requirement description information input by the user, adjust the first illumination distribution information required for rendering the image to be processed to obtain the second illumination distribution information, and perform illumination rendering on the image to be processed based on the physical information and the second illumination distribution information to obtain and display the target image.

[0170] In any embodiment of this application, the server can compress the physical information and the first illumination distribution information to obtain compressed data, and send the compressed data to the terminal. Correspondingly, after receiving the compressed data, the terminal can decompress the compressed data to obtain the physical information and the first illumination distribution information, and adjust the illumination according to the illumination requirement description information input by the user, based on the first illumination distribution information required for rendering the image to be processed, to obtain the second illumination distribution information. Based on the physical information and the second illumination distribution information required for rendering the image to be processed, the terminal performs illumination rendering on the image to be processed to obtain and display the target image.

[0171] Therefore, compressing the data to be transmitted (i.e., physical information and first illumination distribution information) by the server before data transmission can reduce network resource overhead and improve transmission efficiency.

[0172] It should be noted that the explanations and descriptions of the various method embodiments executed on the terminal described above also apply to this embodiment, and their implementation principles are similar, so they will not be repeated here.

[0173] The rendering method of this application allows users to flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, and overall atmosphere. Based on the user's lighting requirement description in natural language, the application performs corresponding lighting rendering processing on the image to be processed. This not only lowers the technical barrier for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances users' freedom and personalized expression in image creation, better meeting the diverse needs of different scenarios and user groups for image lighting effects. Furthermore, by introducing the physical information and lighting distribution information required for rendering the image to be processed, this application enables lighting modeling based on real-world optical laws, avoiding unreasonable lighting changes or light and shadow distortion, thereby significantly improving the realism and visual credibility of the rendered image. Based on this, by combining the user's input description of lighting requirements, it is possible to accurately match the user's subjective lighting intentions on the basis of the physical lighting model, and achieve efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for quick image enhancement, but also supports professional users to finely control lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0174] In any embodiment of this application, the lighting rendering scheme is not affected by a preset model. Through deep learning technology, it calculates the physical information required for lighting. Based on this physical information, lighting distribution information, and the user-input lighting requirement description, it performs lighting editing on the image to be processed, improving the lighting editing effect. For example, the terminal can upload the image to be processed to the server, where the server performs inverse rendering calculations. That is, it uses the GPU to calculate the rendering information required for rendering the image to be processed (including physical information and lighting distribution information) and sends it to the terminal. The terminal analyzes the user-input lighting requirement description information using a local large model to determine the user's lighting intention. Based on this lighting intention, it updates the lighting distribution information sent by the server to obtain updated lighting distribution information (including the light source positions and colors corresponding to various lighting components). Then, it calls the GPU rendering pipeline to perform lighting processing on the image to be processed according to the updated lighting distribution information and physical information, obtaining a rendered image under the new lighting.

[0175] As an example, taking a captured image as the image to be processed, the illumination processing of the captured image can be achieved through a collaborative processing method between the terminal and the server. The illumination processing flow can be as follows: Figure 7 As shown, it mainly includes the following steps:

[0176] 1. The terminal's ISP processes the input image to obtain the captured image and sends the captured image to the server.

[0177] The input image is the image captured in response to the shooting operation.

[0178] 2. The server uses a neural network to generate the physical information required for rendering (including information on multiple modalities such as depth, albedo, normals, and specular reflection) and the lighting distribution information of the scene being captured, based on the captured images. The neural network takes RGB format images as input and outputs all the rendering information needed for rendering.

[0179] For example, the server can use a separate neural network to process the physical information of each modality, as implemented as follows: Figure 6 As shown. For each iteration, the captured RGB image and the physical information of the corresponding modality from the previous iteration (e.g., blank input for the first iteration) are input into the UNet. The network output is the physical information of the corresponding modality for the current iteration. For each specific modality, an independent neural network is used for processing. For example, for the four modalities of depth, albedo, normal, and specular reflection, four independent UNet networks are needed for prediction.

[0180] 3. The server calls the rendering pipeline to process the rendering image using the rendering information (including physical information and lighting distribution information) generated in step 2, obtains the rendered image, and calculates the difference between the rendered image and the real RGB format captured image.

[0181] For example, the server can use a lighting rendering model (such as the Blinn-Phong lighting model) to calculate the lighting effects on the object's surface based on the rendering information, and then perform lighting processing on the rendered image based on these lighting effects. The lighting distribution information can include the distribution information of three lighting components: ambient light, diffuse light, and specular light. The distribution information of each lighting component can be described by two RGB images: one image describes the light source position using RGB three channels, and the other image describes the light source color using RGB three channels.

[0182] 4. If the difference in step 3 is small enough (e.g., the mean square error is less than the threshold M), or the number of iterations is too large (e.g., the number of iterations exceeds the threshold N), then the lighting distribution information and the physical information required for rendering are compressed and transmitted to the terminal, proceeding to step 6. Otherwise, proceed to step 5.

[0183] 5. Using the rendering information output from this iteration as input, the neural network is invoked again to generate the rendering information for the next iteration, and the process is repeated iteratively. After the update, proceed to step 3, and the rendering pipeline is invoked again to perform lighting processing on the captured image.

[0184] 6. After receiving the compressed data, the terminal's CPU decompresses the compressed data and hands over the rendering-related physical information to the GPU rendering pipeline.

[0185] 7. The user describes the required lighting for the captured image using natural language. The terminal's NPU calls the local large model to analyze the user's input lighting requirement description, obtain the user's lighting intention, and adjust the lighting distribution information sent by the server based on the lighting intention.

[0186] 8. The terminal's GPU invokes the rendering pipeline, using the physical information from step 6 and the lighting distribution information from step 7 to perform lighting rendering on the captured image, generating a new rendered image. The implementation principle is similar to step 3 and will not be elaborated upon here.

[0187] 9. The user views the rendered image and determines whether the rendered image meets the user's lighting adjustment requirements. If yes, proceed to step 10; otherwise, return to step 7, regenerate the lighting distribution information using the large model, and render the captured image using the GPU rendering pipeline.

[0188] 10. Output and save the current rendered image.

[0189] To implement the above embodiments, this application also proposes a chip.

[0190] Figure 8 This is a schematic diagram of the structure of a chip provided for an exemplary embodiment of this application.

[0191] like Figure 8 As shown, the chip 800 may include: CPU 810, NPU 820 and GPU 830.

[0192] The central processing unit (CPU) 810 is used to acquire physical information and first illumination distribution information obtained by reverse rendering the image to be processed, and send the physical information to the graphics processing unit (GPU) 830, and send the first illumination distribution information to the neural network processing unit (NPU) 820.

[0193] The NPU 820 is used to adjust the first illumination distribution information according to the input illumination demand description information to obtain the second illumination distribution information, and then send the second illumination distribution information to the GPU 830.

[0194] The GPU 830 is used to perform lighting rendering on the image to be processed based on physical information and second lighting distribution information to obtain the target image.

[0195] In one implementation of the embodiments of this application, such as Figure 9 As shown, the chip 800 may also include:

[0196] The ISP 840 is used to process an input image to obtain a processed image; where the input image is an image captured in response to a shooting operation.

[0197] The processing operations performed by the ISP 840 on the input image include, but are not limited to: black level correction, lens shading correction, bad pixel correction, demosaic, white balance, automatic exposure control, automatic focus control, automatic white balance, gamma correction, color correction, noise reduction, color enhancement, and lens distortion correction.

[0198] In one implementation of this application, physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction;

[0199] The first illumination distribution information includes the distribution information of various types of illumination components. The distribution information is used to indicate the position of the light source and / or the color of the light source for the corresponding illumination component.

[0200] In one implementation of this application, the NPU 820 is used to: obtain a prompt template; wherein the prompt template is used to indicate the lighting adjustment task to be performed by the large model; update the prompt template according to the lighting requirement description information and the first lighting distribution information to obtain prompt information; and call the large model to perform the lighting adjustment task on the prompt information to obtain the second lighting distribution information output by the large model.

[0201] In one implementation of this application, the NPU 820 is further configured to perform any of the following:

[0202] In response to an input operation triggered by a voice input control, the system obtains a description of the lighting requirements for the image to be processed; the voice input control is used to invoke a large model and receive voice input commands.

[0203] In response to an input operation triggered by a text input control, the system obtains a description of the lighting requirements for the image to be processed; the text input control is used to call up a large model and receive text input instructions.

[0204] In response to input operations triggered by interactive controls, obtain lighting requirement description information for the input image to be processed; wherein, interactive controls are used to interact with the large model.

[0205] In one implementation of this application, the ISP 840 is further configured to: send the image to be processed to the server; and the CPU 810 is configured to: receive compressed data sent by the server; wherein the compressed data is obtained by the server performing reverse rendering prediction on the image to be processed to obtain physical information and first illumination distribution information, and compressing the physical information and first illumination distribution information; decompressing the compressed data to obtain physical information and first illumination distribution information; sending the physical information to the GPU 830, and sending the first illumination distribution information to the NPU 820.

[0206] It should be noted that the explanation of the rendering method embodiment executed on the terminal described above also applies to the chip in this embodiment, and will not be repeated here.

[0207] In the chip of this application embodiment, users can flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, overall atmosphere, and other visual characteristics. In this application, based on the lighting requirement description information input by the user in natural language, corresponding lighting rendering processing is performed on the image to be processed. This not only reduces the technical barrier for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances the user's freedom and personalized expression in image creation, better meeting the diverse needs of different scenarios and user groups for image lighting effects. In this application, by introducing the physical information and lighting distribution information required for rendering the image to be processed, lighting modeling can be performed based on the optical laws of the real world, avoiding unreasonable lighting changes or light and shadow distortion, thereby significantly improving the realism and visual credibility of the rendered image. Based on this, by combining the user's input description of lighting requirements, it is possible to accurately match the user's subjective lighting intentions on the basis of the physical lighting model, and achieve efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for quick image enhancement, but also supports professional users to finely control lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0208] To implement the above embodiments, this application also proposes a terminal, wherein the terminal includes as follows: Figure 8 or Figure 9 The embodiment includes a chip 800 and an image sensor, wherein the image sensor is used to capture an input image in response to a shooting operation.

[0209] To implement the above embodiments, this application also proposes a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned... Figure 5 The rendering method of the embodiment.

[0210] To implement the above embodiments, this application also proposes a rendering apparatus.

[0211] Figure 10 This is a schematic diagram of the structure of a rendering apparatus provided for an exemplary embodiment of this application.

[0212] like Figure 10 As shown, the rendering device 1000 may include: a first acquisition module 1010, a second acquisition module 1020, an adjustment module 1030, and a rendering module 1040.

[0213] The first acquisition module 1010 is used to acquire illumination requirement description information for the image to be processed in response to the input operation.

[0214] The second acquisition module 1020 is used to acquire the first illumination distribution information and physical information obtained by reverse rendering prediction of the image to be processed;

[0215] The adjustment module 1030 is used to adjust the first illumination distribution information according to the illumination demand description information to obtain the second illumination distribution information;

[0216] The rendering module 1040 is used to perform lighting rendering on the image to be processed based on physical information and second lighting distribution information, so as to obtain and display the target image.

[0217] In one implementation of this application, physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction;

[0218] The first illumination distribution information includes the distribution information of various types of illumination components. The distribution information is used to indicate the position of the light source and / or the color of the light source for the corresponding illumination component.

[0219] In one implementation of this application, the adjustment module 1030 is used to: obtain a prompt template; wherein the prompt template is used to indicate the lighting adjustment task to be performed by the large model; update the prompt template according to the lighting requirement description information and the first lighting distribution information to obtain prompt information; and call the large model to perform the lighting adjustment task on the prompt information to obtain the second lighting distribution information output by the large model.

[0220] In one implementation of this application embodiment, the first acquisition module 1010 is configured to perform any of the following:

[0221] In response to input operations to the voice input control, the system obtains lighting requirement description information for the image to be processed; wherein, the voice input control is used to invoke the large model and receive voice input commands;

[0222] In response to input operations on the text input control, obtain lighting requirement description information for the image to be processed; wherein, the text input control is used to call the large model and receive text input instructions;

[0223] In response to input operations to the interactive controls, the system obtains lighting requirement description information for the input image to be processed; the interactive controls are used to interact with the large model.

[0224] In one implementation of this application, the second acquisition module 1020 is used to: send an image to be processed to the server; receive compressed data sent by the server; wherein the compressed data is obtained by the server performing reverse rendering prediction on the image to be processed to obtain physical information and first illumination distribution information, and compressing the physical information and first illumination distribution information; and decompress the compressed data to obtain physical information and first illumination distribution information.

[0225] In one implementation of this application embodiment, the rendering apparatus 1000 may further include:

[0226] A save module is used to save the target image in response to a first user operation; wherein the first user operation is used to indicate that the lighting effect of the target image meets the lighting adjustment requirements of the target object, and the target object includes the trigger object of the input operation;

[0227] The third acquisition module is further configured to: in response to the second user operation, reacquire the lighting requirement description information; wherein the second user operation is used to indicate that the lighting effect of the target image does not meet the lighting adjustment requirements;

[0228] The rendering module 1040 is also used to: re-render the image to be processed based on the re-acquired lighting requirement description information, until the rendered image to be processed meets the lighting adjustment requirements, and then save the rendered image to be processed.

[0229] It should be noted that the explanation of the rendering method embodiment executed on the terminal described above also applies to the rendering device of this embodiment, and will not be repeated here.

[0230] In the rendering apparatus of this application embodiment, users can flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, overall atmosphere, and other visual characteristics. In this application, based on the lighting requirement description information input by the user in natural language, corresponding lighting rendering processing is performed on the image to be processed. This not only lowers the technical barrier for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances the user's freedom and personalized expression in image creation, better meeting the diverse needs of different scenarios and user groups for image lighting effects. In this application, by introducing the physical information and lighting distribution information required for rendering the image to be processed, lighting modeling can be performed based on the optical laws of the real world, avoiding unreasonable lighting changes or light and shadow distortion, thereby significantly improving the realism and visual credibility of the rendered image. Based on this, by combining the user's input description of lighting requirements, it is possible to accurately match the user's subjective lighting intentions on the basis of the physical lighting model, and achieve efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for quick image enhancement, but also supports professional users to finely control lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0231] To implement the above embodiments, this application also proposes a rendering apparatus.

[0232] Figure 11 This is a schematic diagram of the structure of another rendering apparatus provided for an exemplary embodiment of this application.

[0233] like Figure 11 As shown, the rendering device 1100 may include a prediction module 1110 and a sending module 1120.

[0234] The prediction module 1110 is used to receive the image to be processed sent by the terminal and perform reverse rendering prediction on the image to be processed to obtain the physical information and the first illumination distribution information required for rendering the image to be processed.

[0235] The transmitting module 1120 is used to transmit physical information and first illumination distribution information to the terminal;

[0236] Among them, physical information and first illumination distribution information are used to describe the terminal's comprehensive illumination requirements, perform illumination rendering on the image to be processed, and obtain and display the target image.

[0237] In one implementation of this application, the prediction module 1110 is configured to: perform at least one round of iterative reverse rendering prediction process based on the image to be processed; and use the physical information and illumination distribution information output by the last round of iterative reverse rendering prediction process as the physical information and first illumination distribution information required for rendering the image to be processed.

[0238] In one implementation of this application embodiment, the prediction module 1110 executes the first iteration of the reverse rendering prediction process in at least one iteration of the reverse rendering prediction process. For example, it performs reverse rendering prediction on the image to be processed to obtain the physical information and illumination distribution information output by the first iteration of the reverse rendering prediction process; performs illumination rendering on the image to be processed based on the physical information and illumination distribution information output by the first iteration of the reverse rendering prediction process to obtain the rendered image output by the first iteration of the reverse rendering prediction process; and terminates the iterative reverse rendering prediction process when the difference between the rendered image output by the first iteration of the reverse rendering prediction process and the image to be processed is less than a set difference threshold.

[0239] In one implementation of this application embodiment, the prediction module 1110 executes the i-th iteration of the reverse rendering prediction process in at least one iteration. For example, in response to i being less than or equal to a set number of iterations, the image to be processed is reverse rendered based on the physical information and illumination distribution information output by the previous i-1 iterations of the reverse rendering prediction process to obtain the physical information and illumination distribution information output by the i-th iteration of the reverse rendering prediction process; where i is a positive integer greater than 1; the image to be processed is illuminated based on the physical information and illumination distribution information output by the i-th iteration of the reverse rendering prediction process to obtain the rendered image output by the i-th iteration of the reverse rendering prediction process; in response to the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed being less than a set difference threshold, the iterative reverse rendering prediction process ends.

[0240] In one implementation of this application, the prediction module 1110 executes the i-th iteration of the reverse rendering prediction process in at least one iteration. For example, in response to i being greater than a set number of iterations, the reverse rendering prediction process ends; or, in response to the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed being greater than or equal to a set difference threshold, the (i+1)-th iteration of the reverse rendering prediction process is executed.

[0241] In one implementation of this application, the sending module 1120 is used to: compress physical information and first illumination distribution information to obtain compressed data; send the compressed data to the terminal; wherein the compressed data is used by the terminal to decompress to obtain physical information and first illumination distribution information, and to adjust the first illumination distribution information according to illumination requirement description information to obtain second illumination distribution information, and to perform illumination rendering on the image to be processed based on the physical information and second illumination distribution information to obtain and display the target image.

[0242] It should be noted that the foregoing explanation of the rendering method embodiment executed on the server also applies to the rendering device of this embodiment, and will not be repeated here.

[0243] In the rendering apparatus of this application embodiment, users can flexibly express their specific expectations for image lighting effects in natural language, such as brightness, shadows, light source direction, color temperature, overall atmosphere, and other visual characteristics. In this application, based on the lighting requirement description information input by the user in natural language, corresponding lighting rendering processing is performed on the image to be processed. This not only lowers the technical barrier for users to manually adjust lighting parameters and improves the automation and interactive intelligence of the image lighting editing process, but also enhances the user's freedom and personalized expression in image creation, better meeting the diverse needs of different scenarios and user groups for image lighting effects. In this application, by introducing the physical information and lighting distribution information required for rendering the image to be processed, lighting modeling can be performed based on the optical laws of the real world, avoiding unreasonable lighting changes or light and shadow distortion, thereby significantly improving the realism and visual credibility of the rendered image. Based on this, by combining the user's input description of lighting requirements, it is possible to accurately match the user's subjective lighting intentions on the basis of the physical lighting model, and achieve efficient mapping from semantics to image generation. This not only meets the needs of ordinary users for quick image enhancement, but also supports professional users to finely control lighting details, balancing ease of use and professionalism, and improving the overall image editing experience.

[0244] Figure 12 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. The electronic device 1200 includes the terminal or server described in the above embodiments. For example, the electronic device 1200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0245] Reference Figure 12The electronic device 1200 may include one or more of the following components: a processing component 1202, a memory 1204, a power component 1206, a multimedia component 1208, an audio component 1210, an input / output (I / O) interface 1212, a sensor component 1214, and a communication component 1216.

[0246] Processing component 1202 typically controls the overall operation of electronic device 1200, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1202 may include one or more processors 1220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1202 may include one or more modules to facilitate interaction between processing component 1202 and other components. For example, processing component 1202 may include a multimedia module to facilitate interaction between multimedia component 1208 and processing component 1202.

[0247] Memory 1204 is configured to store various types of data to support the operation of electronic device 1200. Examples of such data include instructions for any application or method operating on electronic device 1200, contact data, phonebook data, messages, pictures, videos, etc. Memory 1204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0248] Power component 1206 provides power to various components of electronic device 1200. Power component 1206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1200.

[0249] Multimedia component 1208 includes a screen that provides an output interface between the electronic device 1200 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1208 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0250] Audio component 1210 is configured to output and / or input audio signals. For example, audio component 1210 includes a microphone (MIC) configured to receive external audio signals when electronic device 1200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1204 or transmitted via communication component 1216. In some embodiments, audio component 1210 also includes a speaker for outputting audio signals.

[0251] I / O interface 1212 provides an interface between processing component 1202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0252] Sensor assembly 1214 includes one or more sensors for providing state assessment of various aspects of electronic device 1200. For example, sensor assembly 1214 may detect the on / off state of electronic device 1200, the relative positioning of components such as the display and keypad of electronic device 1200, changes in position of electronic device 1200 or a component of electronic device 1200, the presence or absence of user contact with electronic device 1200, the orientation or acceleration / deceleration of electronic device 1200, and temperature changes of electronic device 1200. Sensor assembly 1214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1214 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 1214 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0253] Communication component 1216 is configured to facilitate wired or wireless communication between electronic device 1200 and other devices. Electronic device 1200 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 1216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1216 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0254] In an exemplary embodiment, the electronic device 1200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0255] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1204 including instructions, which can be executed by a processor 1220 of an electronic device 1200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0256] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the rendering method as described in any of the foregoing method embodiments.

[0257] To implement the above embodiments, this application also proposes a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the rendering method as described in any of the foregoing method embodiments.

[0258] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0259] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0260] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0261] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and compact disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0262] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0263] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0264] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0265] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A rendering method, characterized in that, include: In response to input operations, obtain illumination requirement description information for the image to be processed; Obtain the first illumination distribution information and physical information obtained by inverse rendering prediction of the image to be processed; Based on the described illumination demand information, the first illumination distribution information is adjusted to obtain the second illumination distribution information. Based on the physical information and the second illumination distribution information, the image to be processed is rendered with illumination to obtain and display the target image.

2. The method according to claim 1, characterized in that, The physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction; The first illumination distribution information includes the distribution information of multiple types of illumination components, and the distribution information is used to indicate the light source position and / or light source color of the corresponding illumination component.

3. The method according to claim 1, characterized in that, The step of adjusting the first illumination distribution information based on the illumination demand description information to obtain the second illumination distribution information includes: Obtain the prompt template; the prompt template is used to indicate the lighting adjustment task to be performed on the large model; The prompt template is updated based on the lighting requirement description information and the first lighting distribution information to obtain the prompt information; The large model is invoked to perform the illumination adjustment task on the prompt information, and the second illumination distribution information output by the large model is obtained.

4. The method according to claim 3, characterized in that, In response to the input operation, the acquisition of illumination requirement description information for the image to be processed includes any one of the following: In response to an input operation to the voice input control, the system obtains illumination requirement description information for the image to be processed; wherein the voice input control is used to invoke the large model and receive voice input commands. In response to an input operation on the text input control, the system obtains illumination requirement description information for the image to be processed; wherein the text input control is used to invoke the large model and receive text input instructions; In response to input operations to the interactive control, the system obtains illumination requirement description information for the image to be processed; wherein the interactive control is used to interact with the large model.

5. The method according to claim 1, characterized in that, The step of obtaining the first illumination distribution information and physical information obtained by inverse rendering prediction of the image to be processed includes: Send the image to be processed to the server; The server receives compressed data sent by the server; wherein the compressed data is obtained by the server performing reverse rendering prediction on the image to be processed to obtain the physical information and the first illumination distribution information, and then compressing the physical information and the first illumination distribution information. The compressed data is decompressed to obtain the physical information and the first illumination distribution information.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: In response to a first user operation, the target image is saved; wherein the first user operation is used to indicate that the lighting effect of the target image meets the lighting adjustment requirements of the target object, and the target object includes the trigger object of the input operation; In response to a second user operation, the illumination requirement description information is retrieved again; wherein the second user operation is used to indicate that the illumination effect of the target image does not meet the illumination adjustment requirement; Based on the reacquired lighting requirement description information, the image to be processed is re-rendered until the rendered image meets the lighting adjustment requirements, and then the rendered image is saved.

7. A rendering method, characterized in that, include: The receiving terminal sends an image to be processed, and performs reverse rendering prediction on the image to be processed to obtain the physical information and first illumination distribution information required for rendering the image to be processed. Send the physical information and the first illumination distribution information to the terminal; The physical information and the first illumination distribution information are used by the terminal to comprehensively describe the illumination requirements, perform illumination rendering on the image to be processed, and obtain and display the target image.

8. The method according to claim 7, characterized in that, Perform inverse rendering prediction on the image to be processed to obtain the physical information and first illumination distribution information required for rendering the image to be processed, including: Perform at least one round of iterative reverse rendering prediction process based on the image to be processed; The physical information and illumination distribution information output from the last iteration of the reverse rendering prediction process are used as the physical information and first illumination distribution information required for rendering the image to be processed.

9. The method according to claim 8, characterized in that, The first iteration of the reverse rendering prediction process in at least one iteration includes: The image to be processed is subjected to reverse rendering prediction to obtain the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process; Based on the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process, the image to be processed is rendered with illumination to obtain the rendered image output by the first round of iterative reverse rendering prediction process. The iterative reverse rendering prediction process ends when the difference between the rendered image output by the first round of iterative reverse rendering prediction process and the image to be processed is less than a set difference threshold.

10. The method according to claim 9, characterized in that, The i-th iteration of the reverse rendering prediction process in at least one iteration includes: In response to i being less than or equal to a set number of iterations, the image to be processed is subjected to reverse rendering prediction based on the physical information and illumination distribution information output by the reverse rendering prediction process of the first i-1 iterations, thereby obtaining the physical information and illumination distribution information output by the reverse rendering prediction process of the i-th iteration; where i is a positive integer greater than 1. Based on the physical information and illumination distribution information output by the i-th iteration reverse rendering prediction process, the image to be processed is rendered with illumination to obtain the rendered image output by the i-th iteration reverse rendering prediction process. The iterative reverse rendering prediction process ends when the difference between the rendered image output by the i-th round of the reverse rendering prediction process and the image to be processed is less than the set difference threshold.

11. The method according to claim 10, characterized in that, The i-th iteration of reverse rendering prediction process also includes any one of the following: In response to i being greater than the set number of iterations, the iterative reverse rendering prediction process ends; In response to the difference between the rendered image output by the i-th iteration of the reverse rendering prediction process and the image to be processed being greater than or equal to the set difference threshold, the (i+1)-th iteration of the reverse rendering prediction process is executed.

12. The method according to any one of claims 7-11, characterized in that, Sending the physical information and the first illumination distribution information to the terminal includes: The physical information and the first illumination distribution information are compressed to obtain compressed data; The compressed data is sent to the terminal; wherein the compressed data is used by the terminal to decompress the data to obtain the physical information and the first illumination distribution information, and to adjust the first illumination distribution information according to the illumination requirement description information to obtain the second illumination distribution information, and to perform illumination rendering on the image to be processed based on the physical information and the second illumination distribution information to obtain and display the target image.

13. A chip, characterized in that, include: The central processing unit (CPU) is used to acquire physical information and first illumination distribution information obtained by reverse rendering the image to be processed, and send the physical information to the graphics processing unit (GPU) and the first illumination distribution information to the neural network processing unit (NPU). The NPU is used to adjust the first illumination distribution information according to the input illumination demand description information to obtain the second illumination distribution information, and send the second illumination distribution information to the GPU. The GPU is used to perform lighting rendering on the image to be processed based on the physical information and the second lighting distribution information to obtain the target image.

14. The chip according to claim 13, characterized in that, Also includes: An image signal processor (ISP) is used to process an input image to obtain the image to be processed; wherein the input image is an image captured in response to a shooting operation.

15. The chip according to claim 13, characterized in that, The physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction; The first illumination distribution information includes the distribution information of multiple types of illumination components, and the distribution information is used to indicate the light source position and / or light source color of the corresponding illumination component.

16. The chip according to claim 13, characterized in that, The NPU is used for: Obtain the prompt template; the prompt template is used to indicate the lighting adjustment task to be performed on the large model; The prompt template is updated based on the lighting requirement description information and the first lighting distribution information to obtain the prompt information; The large model is invoked to perform the illumination adjustment task on the prompt information, and the second illumination distribution information output by the large model is obtained.

17. The chip according to claim 16, characterized in that, The NPU is also used to perform any of the following: In response to an input operation triggered by a voice input control, the system acquires illumination requirement description information for the image to be processed; wherein the voice input control is used to invoke the large model and receive voice input commands. In response to an input operation triggered by a text input control, the system obtains illumination requirement description information for the image to be processed; wherein the text input control is used to invoke the large model and receive text input instructions. In response to an input operation triggered by an interactive control, the system acquires illumination requirement description information for the image to be processed; wherein the interactive control is used to interact with the large model.

18. The chip according to claim 14, characterized in that, The ISP is further configured to: send the image to be processed to the server. The CPU is used for: The server receives compressed data sent by the server; wherein the compressed data is obtained by the server performing reverse rendering prediction on the image to be processed to obtain the physical information and the first illumination distribution information, and then compressing the physical information and the first illumination distribution information. The compressed data is decompressed to obtain the physical information and the first illumination distribution information; The physical information is sent to the GPU, and the first illumination distribution information is sent to the NPU.

19. A terminal, characterized in that, Includes the chip and image sensor as described in any one of claims 13-18, wherein the image sensor is used to capture an input image in response to a capture operation.

20. A server, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as described in any one of claims 7 to 12.

21. A rendering apparatus, characterized in that, include: The first acquisition module is used to acquire illumination requirement description information for the image to be processed in response to input operations; The second acquisition module is used to acquire the first illumination distribution information and physical information obtained by reverse rendering prediction of the image to be processed; The adjustment module is used to adjust the first illumination distribution information according to the illumination demand description information to obtain the second illumination distribution information; The rendering module is used to perform lighting rendering on the image to be processed based on the physical information and the second lighting distribution information, so as to obtain and display the target image.

22. The apparatus according to claim 21, characterized in that, The physical information is used to indicate the physical properties of scene elements in the shooting scene to which the image to be processed belongs in at least one dimension; wherein, the dimension includes at least one of the following: spatial location, surface material, and lighting interaction; The first illumination distribution information includes the distribution information of multiple types of illumination components, and the distribution information is used to indicate the light source position and / or light source color of the corresponding illumination component.

23. A rendering apparatus, characterized in that, include: The prediction module is used to receive the image to be processed sent by the terminal and perform reverse rendering prediction on the image to be processed to obtain the physical information and the first illumination distribution information required for rendering the image to be processed. The sending module is used to send the physical information and the first illumination distribution information to the terminal; The physical information and the first illumination distribution information are used by the terminal to comprehensively describe the illumination requirements, perform illumination rendering on the image to be processed, and obtain and display the target image.

24. The apparatus according to claim 23, characterized in that, The prediction module is used for: Perform at least one round of iterative reverse rendering prediction process based on the image to be processed; The physical information and illumination distribution information output from the last iteration of the reverse rendering prediction process are used as the physical information and first illumination distribution information required for rendering the image to be processed. Among them, the first iteration of the reverse rendering prediction process in at least one iteration of the reverse rendering prediction process includes: The image to be processed is subjected to reverse rendering prediction to obtain the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process; Based on the physical information and illumination distribution information output by the first round of iterative reverse rendering prediction process, the image to be processed is rendered with illumination to obtain the rendered image output by the first round of iterative reverse rendering prediction process. The iterative reverse rendering prediction process ends when the difference between the rendered image output by the first round of iterative reverse rendering prediction process and the image to be processed is less than a set difference threshold. Among them, the i-th iteration of the reverse rendering prediction process in at least one iteration of the reverse rendering prediction process includes: In response to i being less than or equal to a set number of iterations, the image to be processed is subjected to reverse rendering prediction based on the physical information and illumination distribution information output by the reverse rendering prediction process of the first i-1 iterations, thereby obtaining the physical information and illumination distribution information output by the reverse rendering prediction process of the i-th iteration; where i is a positive integer greater than 1. Based on the physical information and illumination distribution information output by the i-th iteration reverse rendering prediction process, the image to be processed is rendered with illumination to obtain the rendered image output by the i-th iteration reverse rendering prediction process. The iterative reverse rendering prediction process ends when the difference between the rendered image output by the i-th round of the reverse rendering prediction process and the image to be processed is less than the set difference threshold.

25. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method according to any one of claims 1 to 6, and / or implement the steps of the method according to any one of claims 7 to 12.

26. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6, and / or implements the steps of the method according to any one of claims 7 to 12.

Images