Image rendering method, electronic device and computer program product

Abstract

The application is suitable for the field of image technology, and provides an image rendering method, an electronic device and a computer program product. The image rendering method comprises the following steps: inputting a natural language rendering instruction into a large language model to obtain a rendering task by analysis, wherein the rendering task type comprises image rendering or animation rendering; encoding target information of the rendering task into a query vector, and matching a target function in a function library based on the query vector; extracting parameters of the target function according to the rendering task type; and calling a function interface through the target function to output a rendering result based on the function interface and the parameters of the target function. The embodiment of the application can reduce the operation complexity of the user in the image rendering process, so that the user can obtain the expected rendering result without manually configuring parameters or switching the setting mode for different tasks.

Description

Technical Field

[0001] This application belongs to the field of image technology, and in particular relates to an image rendering method, electronic device and computer program product. Background Technology

[0002] Image rendering aims to provide users with a visual representation of data. Current image rendering methods typically require users to manually adjust a large number of parameters or rely on preset templates for generation. These methods are complex to operate, require users to have extensive medical knowledge and image processing skills, and require users to adjust rendering parameters one by one. Users often need to modify parameters back and forth multiple times to obtain a visual representation that meets their expectations. Summary of the Invention

[0003] This application provides an image rendering method, electronic device, and computer program product that can reduce the operational complexity for users during the image rendering process, allowing users to obtain the expected rendering results without manually configuring parameters or switching settings for different tasks.

[0004] The first aspect of this application provides an image rendering method, comprising: inputting natural language rendering instructions into a large language model, parsing to obtain a rendering task, wherein the rendering task type includes image rendering or animation rendering; encoding the target information of the rendering task into a query vector, and matching a target function based on the query vector in a function library; extracting the parameters of the target function according to the rendering task type; and calling a function interface through the target function to output a rendering result based on the function interface and the parameters of the target function.

[0005] In some implementations of the first aspect, the parameters of the objective function are extracted according to the rendering task type, including: if the rendering task type is image rendering, the parameters of the objective function are extracted from the target information of the rendering task; if the rendering task type is animation rendering, the timing information of the animation is extracted from the target information of the rendering task, and the parameters of the key frame are generated based on the timing information, wherein the key frame is associated with the natural language rendering instruction, and the parameters of the current key frame are obtained based on the parameter transformation of the previous key frame, or based on the accumulation of the parameters of all historical key frames.

[0006] In some implementations of the first aspect, when the rendering task type is animation rendering, the image rendering method further includes: decomposing the animation rendering task into sub-tasks corresponding to each key frame; calling a function interface through a target function to output rendering results based on the parameters of the function interface and the target function, including: calling a function interface through a target function to render the image corresponding to each key frame respectively based on the parameters of the function interface and the target function; and combining the images of each key frame according to the timing information to generate rendering results.

[0007] In some embodiments of the first aspect, combining the images of each key frame according to temporal information to generate a rendering result includes: generating one or more transition frames between two adjacent key frames according to a preset inter-frame interpolation strategy; and combining the images corresponding to each key frame and transition frame according to temporal information to generate a rendering result.

[0008] In some embodiments of the first aspect, if the parameters of the objective function are incomplete or do not match the objective function, the image rendering method further includes: outputting prompt information to obtain supplementary information about the objective function and / or the parameters of the objective function.

[0009] In some embodiments of the first aspect, if the parameters of the objective function are still incomplete after the number of output rounds of the prompt message reaches a preset threshold, the image rendering method further includes: configuring default values ​​and / or default ranges for the missing parameters of the objective function.

[0010] In some embodiments of the first aspect, the image rendering method further includes: displaying the rendering result; receiving modification information input by the user; adjusting the rendering result according to the modification information, or regenerating a new rendering result based on the modification information.

[0011] In some embodiments of the first aspect, encoding the target information of the rendering task into a query vector and matching the target function within a function library based on the query vector includes: determining a target function library from at least two function libraries according to the rendering task type, the at least two function libraries including a first function library for image rendering and a second function library for animation rendering; and matching the target function within the target function library based on the query vector.

[0012] A second aspect of this application provides an image rendering apparatus, comprising: a large model processing unit, configured to input natural language rendering instructions into a large language model and parse them to obtain rendering tasks, wherein the rendering task type includes image rendering or animation rendering; a function query unit, configured to encode the target information of the rendering task into a query vector and match the target function based on the query vector in a function library; a parameter extraction unit, configured to extract the parameters of the target function according to the rendering task type; and an image rendering unit, configured to call a function interface through the target function and output the rendering result based on the function interface and the parameters of the target function.

[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image rendering method as described in any of the first aspects.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described image rendering method.

[0015] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the steps of the above-described image rendering method.

[0016] In the embodiments of this application, natural language rendering instructions are input into a large language model, which parses the rendering task to obtain whether the rendering task type is image rendering or animation rendering. The target information of the rendering task is encoded into a query vector. The target function is matched based on the query vector in the function library. The parameters of the target function are extracted according to the rendering task type. The function interface is called through the target function, and the rendering result is output based on the function interface and the parameters of the target function. This allows the system to automatically understand the user's intent and match the appropriate target function and its parameters when the user describes the rendering requirements in natural language, without the need for repeated manual adjustments. At the same time, the system can automatically identify the rendering task type and extract the corresponding parameters. Users do not need to configure them separately for different rendering task types, thereby reducing operational complexity and improving rendering efficiency and accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the implementation flow of the image rendering method provided in the embodiments of this application; Figure 2 This is a schematic diagram illustrating the specific implementation process of extracting parameters of the target function provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the specific implementation flow of the image rendering method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the specific structure of the image rendering apparatus provided in the embodiments of this application; Figure 5 This is a schematic diagram of the specific structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] Current image rendering methods typically require users to manually adjust numerous parameters or rely on preset templates for generation. These methods are complex, demanding extensive medical knowledge and image processing skills from users, and necessitate adjusting rendering parameters one by one. Users often need to modify parameters repeatedly to obtain the desired visualization. Therefore, this application proposes an image rendering method that reduces the operational complexity for users during the image rendering process, allowing them to obtain the expected rendering results without manually configuring parameters or switching settings for different tasks.

[0024] To illustrate the technical solution of this application, specific embodiments are described below.

[0025] Please refer to Figure 1 , Figure 1The illustration shows a schematic diagram of the implementation flow of the image rendering method provided in an embodiment of this application. This method can be applied to electronic devices. The electronic devices can be smart devices such as medical devices, computers, and tablets; this application does not limit the scope of the application.

[0026] Specifically, the above image rendering method may include the following steps S101 to S104.

[0027] Step S101: Input the natural language rendering instructions into the large language model and parse them to obtain the rendering task.

[0028] In the embodiments of this application, natural language rendering instructions are text information input by the user in everyday natural language, describing the desired rendering effect of an image or animation, such as "reduce the volume data to half its original size" or "generate a 10-second video, first cropping one-third from left to right, then rotating it once starting from the third second." The large language model is a pre-trained deep learning model capable of understanding and generating natural language, and can extract semantic information from the unstructured text input of natural language rendering instructions. Specifically, when natural language rendering instructions are input into the large language model, the model performs semantic parsing on the input text, extracting keywords, action intentions, and timing information, transforming them into a structured rendering task, and outputting it.

[0029] A rendering task is used to characterize the rendering operations that need to be performed. Each rendering task has a corresponding rendering task type. The rendering task type is a classification identifier for the rendering task, and can include image rendering or animation rendering. Image rendering is a rendering task type without temporal information, referring to real-time, static rendering operations on a single frame of an image, such as scaling, cropping, and rotation. Animation rendering is a rendering task type with temporal information, referring to a set of continuous rendering operations that include changes over time.

[0030] Step S102: Encode the target information of the rendering task into a query vector, and match the target function based on the query vector in the function library.

[0031] The target information of the rendering task is semantic information extracted from natural language rendering instructions, describing the specific operation content and the object of the operation. Examples include "shrink to half the original size" in "shrink the volume data to half its original size," or "rotate 360 ​​degrees" in "rotate 360 ​​degrees at the 10th second." The semantic text in the target information is converted into a fixed-dimensional numerical vector representation using a large language model or a dedicated encoding model, resulting in a query vector. Query vectors are used to group semantically similar texts closer together in the vector space, facilitating similarity calculation and retrieval.

[0032] A function library is a pre-built database that stores multiple callable rendering functions and their metadata. Each rendering function is stored in the function library in a structured format, which may include the function name, function description, related operation keywords, and command examples for semantic-based retrieval and matching.

[0033] By calculating the similarity (e.g., cosine similarity) between the query vector and the function vectors of each rendering function in the function library, the rendering function with the highest similarity is selected as the target function. The target function is the rendering function in the function library that best matches the semantics of the target information. In this way, an automatic mapping between the user's natural language expression of operation intent and the executable functions inside the software can be achieved, avoiding the need for the user to know the function name in advance or manually search for the rendering function. Even if the target information does not completely match the function description stored in the function library, accurate matching can still be achieved based on semantic similarity.

[0034] Step S103: Extract the parameters of the objective function according to the rendering task type.

[0035] The parameters of the objective function are the specific input values ​​required for its execution, controlling details such as the degree, direction, and timing of the rendering operation. Since different rendering task types may have different parameter structures and meanings, extracting the parameters of the objective function based on the rendering task type ensures that the extracted parameters match the task type, avoiding rendering failures or unexpected results due to incorrect parameter formats.

[0036] Step S104: Call the function interface through the target function to output the rendering result based on the parameters of the function interface and the target function.

[0037] The function interface is the standardized entry point provided by the target function, which can specify the input format, data type, and calling method of the parameters. After determining the target function and its parameters, the corresponding function interface can be called to pass the parameters of the target function to the underlying image editing software or rendering engine according to the format agreed upon by the interface. The image editing software or rendering engine then returns the calculation results, forming a rendering result that is perceptible to the user.

[0038] The rendering results can be static images, animated GIFs, or videos. The specific output format can be set according to actual needs, thereby meeting the diverse display needs of the medical field and lowering the threshold for playing and using the rendering results on different devices.

[0039] In the embodiments of this application, natural language rendering instructions are input into a large language model, which parses the rendering task to obtain whether the rendering task type is image rendering or animation rendering. The target information of the rendering task is encoded into a query vector. The target function is matched based on the query vector in the function library. The parameters of the target function are extracted according to the rendering task type. The function interface is called through the target function, and the rendering result is output based on the function interface and the parameters of the target function. This allows the system to automatically understand the user's intent and match the appropriate target function and its parameters when the user describes the rendering requirements in natural language, without the need for repeated manual adjustments. At the same time, the system can automatically identify the rendering task type and extract the corresponding parameters. Users do not need to configure them separately for different rendering task types, thereby reducing operational complexity and improving rendering efficiency and accuracy.

[0040] In some embodiments of this application, users can input natural language rendering instructions via voice or text. When voice input is used, the voice can be converted into text to obtain the natural language rendering instructions.

[0041] Natural language rendering instructions describe the target information required by the user, including specific processing requirements for certain medical data, such as adjusting certain visual effects, generating 3D rendered images or videos from a certain angle, etc. The large language model can vectorize the lexical units in the user-input natural language rendering instructions, and through vector computation, obtain the specific semantic content contained in the user input, thereby determining whether the rendering task is animation rendering or image rendering.

[0042] The animation rendering task can be broken down into multiple sub-tasks corresponding to keyframes. Keyframes can be associated with natural language rendering instructions, specifically referring to frames containing target information mentioned in those instructions. The sub-tasks also have temporal sequence information; each keyframe's sub-task contains the requirements for that keyframe. For example, if the natural language rendering instructions are: "Crop 1 / 2 from left to right at 0 seconds," "Crop 1 / 3 from left to right at 3 seconds," and "Rotate 360 ​​degrees at 10 seconds," then seconds 0, 3, and 10 are the three keyframes. The sub-task corresponding to the keyframe at 0 seconds is "Crop 1 / 2 from left to right," the sub-task corresponding to the keyframe at 3 seconds is "Crop 1 / 3 from left to right," and the sub-task corresponding to the keyframe at 10 seconds is "Rotate 360 ​​degrees."

[0043] In some embodiments of this application, the target function for matching query vectors within the function library can be implemented based on Retrieval-augmented Generation (RAG).

[0044] Specifically, encoding the target information of the rendering task into a query vector and matching the target function within the function library based on the query vector may include: determining the target function library from at least two function libraries according to the rendering task type, wherein the at least two function libraries include a first function library for image rendering and a second function library for animation rendering; and matching the target function within the target function library based on the query vector.

[0045] Specifically, the first function library is dedicated to storing rendering functions related to image rendering. Each rendering function in the first function library corresponds to a time-independent, immediate rendering operation, such as scaling, cropping, and rotation. The second function library is dedicated to storing rendering functions related to animation rendering. Each rendering function in the second function library can correspond to either a rendering operation with timing information or a time-independent rendering operation, such as keyframe rotation and keyframe cropping. The target function library is a specific function library used for subsequent function retrieval.

[0046] Since rendering functions for image rendering do not require temporal information, while rendering functions for animation rendering often need to handle continuous changes along the timeline, mixing all rendering functions in the same function library may lead to mismatches due to semantic similarity bias during retrieval. Therefore, this application pre-divides function libraries according to rendering task types, and then selects the corresponding target function library (either the first or second function library) from at least two function libraries based on the rendering task type, limiting the retrieval scope to function libraries matching the rendering task type. Then, within the selected target function library, a similarity search is performed using query vectors to find the semantically most matching target function, avoiding retrieval errors or parameter incompatibility issues caused by mixed function storage.

[0047] Specifically, the target function library covers various rendering functions that can be called in image editing software, and each function is stored using a rendering function and its description.

[0048] For example, the scaling function is stored in the function library as follows: - Function name: scaling; - Function description: Changes the size of an object, either by enlarging or shrinking it; -Related operations: zoom in, zoom out, scale, shrink, enlarge; - Command examples: "Double the volume data", "Shrink to half the original size", "Restore the original size", "Shrink the volume data to 1 / 3 of the original size".

[0049] The target information of the rendering task is encoded into a query vector using a large language model, and then matched within a target function library to find the target function and its description that best fits the rendering task. For example, if the target information of the rendering task is "to reduce the volume data to half its original size," the search will find the target function and its description as the aforementioned scaling function.

[0050] In some embodiments of this application, such as Figure 2 As shown, extracting the parameters of the objective function according to the rendering task type may include steps S201 to S202.

[0051] Step S201: If the rendering task type is image rendering, then extract the parameters of the objective function from the target information of the rendering task.

[0052] Image rendering requires only static parameters; the parameters of the objective function can be directly obtained from the target information of the rendering task. The parameters of the objective function may include specific image data, rendering effect options, viewpoint settings, etc. For the objective function, parameters can be parsed and filled in to form specific rendering instructions.

[0053] For example, if the rendering task is "to shrink the volume data to half its original size", after matching the aforementioned scaling function, the initial state of the volume data (e.g., the initial scaling ratio is 2) can be combined to extract the parameter of the target function as ScaleTo(1).

[0054] Step S202: If the rendering task type is animation rendering, extract the timing information of the animation from the target information of the rendering task, and generate the parameters of the key frame based on the timing information.

[0055] As mentioned earlier, keyframes are associated with natural language rendering instructions. Furthermore, the parameters of the current keyframe are obtained by transforming the parameters of the previous keyframe, or by accumulating the parameters of all historical keyframes. Here, "all historical frames" refers to all keyframes that precede the current keyframe in time. For example, if the current keyframe is the fifth frame, then the previous keyframe is the fourth frame, and all historical keyframes are frames one through four.

[0056] Since animation rendering often requires breaking down the user-described temporal actions into a series of keyframes, and since the state of subsequent frames in an animation often depends on the previous frame, parameter values ​​need to be calculated using accumulation or transformation methods. Specifically, temporal information can be extracted from the target information to sort the keyframes. The processing method for the first keyframe can refer to the single-frame image processing corresponding to image rendering. Starting from the second keyframe, parameter values ​​can be calculated through accumulation or transformation. Taking the third frame as an example, specific increments or differences can be superimposed on the parameters of the second frame, or the increments or differences generated by the parameters of all historical keyframes (the first and second frames) can be superimposed on the initial values ​​before the processing of the starting keyframe. For example, after matching the rendering function of the rotation function, the initial state of the combined data (e.g., the initial rotation angle is 0), according to the temporal information, has accumulated a rotation of 180° by the previous keyframe, so it can be rotated another 180° on top of the 180°.

[0057] Similarly, for the objective function of each keyframe, the parameters of the keyframe can be filled in to form specific rendering instructions.

[0058] For the parameters of the objective function, the initial state and target information of the volume data can be input into a large language model for extraction. Volume data, in the fields of medicine and computer graphics, refers to a collection of data with three-dimensional spatial coordinates (x, y, z) as position functions; essentially, it is a data set formed after discretizing and sampling real three-dimensional objects.

[0059] By using parameter extraction branches driven by rendering task type, it is possible to ensure that the structure of the parameters matches the nature of the rendering task type, avoiding the incorrect extraction of static parameters for animation rendering or the forced generation of timing parameters for image rendering. In particular, the accumulation / transformation mechanism in animation rendering can accurately reproduce the continuous action logic implicit in the user description, thereby generating smooth and physically intuitive animation effects.

[0060] When the rendering task type is animation rendering, the image rendering method further includes: decomposing the animation rendering task into sub-tasks corresponding to multiple keyframes. In this case, calling the function interface through the target function to output the rendering result based on the parameters of the function interface and the target function can include: calling the function interface through the target function to render the image corresponding to each keyframe separately based on the parameters of the function interface and the target function; and combining the images of each keyframe according to timing information to generate the rendering result.

[0061] Specifically, the animation rendering task is broken down into multiple keyframe sub-tasks. Each keyframe represents the rendering state at a specific moment in the animation, but the actual pixel image has not yet been generated. By calling the function interface of the target function, the parameters of each keyframe (such as time point, rotation angle, scaling ratio, etc.) are passed as parameters to the underlying rendering engine. The engine independently renders a corresponding image for each keyframe. These images are discontinuous on the timeline and only contain the visual state of the key moments explicitly specified by the user. Subsequently, these discrete keyframe images are arranged and combined according to the temporal information (the order of time points from smallest to largest) to form an animation sequence with temporal order, which is output as the final rendering result. In this way, the combined rendering result can be played on various devices, reducing the hardware dependence of real-time rendering and enabling high-fidelity medical animations to be used and shared in more scenarios.

[0062] In some embodiments of this application, combining the images of each key frame according to timing information to generate a rendering result may include: generating one or more transition frames between two adjacent key frames according to a preset inter-frame interpolation strategy; and combining the images corresponding to each key frame and transition frame according to timing information to generate a rendering result.

[0063] The preset inter-frame interpolation strategy is a predefined algorithm rule used to generate intermediate frames between two adjacent keyframes. It can include linear interpolation (attribute values ​​change uniformly over time), nonlinear interpolation (such as ease-in / ease-out, acceleration / deceleration curves), etc., to control the motion speed and smoothness of the animation, and can realize inter-frame transitions, scene switching, dynamic effects, etc.

[0064] Understandably, playing keyframe images directly in sequence would create visual jumps between adjacent keyframes, resulting in a stiff and choppy animation. To address this, between two adjacent keyframes, a preset inter-frame interpolation strategy is used to calculate the attribute values ​​of one or more transition frames (e.g., interpolating intermediate angles like 120° and 150° between 90° and 180° rotation angles). The rendering engine then generates the corresponding images for these transition frames. The keyframe images and transition frame images are then sorted and combined according to temporal information (from smallest to largest time point) to form a complete and smooth animation sequence, which serves as the final rendering output. This allows for rich motion effects and meets the personalized animation rhythm requirements of different medical demonstration scenarios.

[0065] In some embodiments of this application, if the parameters of the objective function are incomplete or do not match the objective function, the image rendering method further includes: outputting prompt information to obtain supplementary information about the objective function and / or the parameters of the objective function.

[0066] Outputting prompts refers to the process of proactively initiating interactive queries to the user in the form of natural language. The content of the prompts can be generated based on missing information in the currently parsed rendering task, as well as the user's historical input.

[0067] For example, if the rendering task is "rotational volume data" but no specific rotation angle is provided, the large model will output prompts through multiple rounds of dialogue, asking the user for the specific rotation angle, such as "What is the rotation angle?"

[0068] The user's initial natural language rendering commands may be incomplete or ambiguous, causing the system to be unable to uniquely determine the target function (e.g., RAG retrieval similarity is below a threshold) or to extract complete parameters (e.g., "rotation data" does not provide angles). When such missing information is detected, a dialogue mechanism is triggered, generating a targeted natural language question based on the type of missing information and outputting it to the user. After the user responds, the supplementary information can be merged with the historical input, and the RAG retrieval or parameter extraction process can be re-executed, avoiding rendering failures or incorrect results due to missing commands. In this way, by leveraging the reasoning capabilities of a large language model and retrieval-enhanced generation technology, complex rendering commands can be processed and generated quickly. Real-time support is provided to users through multi-turn intelligent dialogue, effectively optimizing transitions and detail adjustments in animations and significantly shortening the rendering cycle for keyframe animations.

[0069] In some embodiments of this application, the above-mentioned prompting information can be output in multiple rounds to achieve polling dialogue. If the parameters of the objective function are still incomplete after the number of output rounds of the prompting information reaches a preset threshold, the image rendering method may further include: configuring default values ​​or default ranges for the missing parameters of the objective function.

[0070] The preset threshold can be set according to actual needs, such as 3 times or 5 times.

[0071] In other words, theoretically, with multi-turn dialogue to supplement parameters enabled, there can be an unlimited number of interactions until the parameters are complete. However, in practical applications, user experience and processing efficiency need to be considered. Therefore, a preset threshold can be set as the upper limit of the number of output turns. After each round of interaction with the user, the completeness of the parameters of the objective function can be checked. If the parameters of the objective function are incomplete and the current output turn has not reached the threshold, a prompt message continues to be output. If the parameters of the objective function are still incomplete after reaching the threshold, the prompt message will no longer be output. Instead, the default processing mechanism will be activated: the missing parameter will be identified, the default value or default range of the parameter will be found, and it will be used as a temporary value to fill the objective function.

[0072] Default values ​​or default ranges can be preset global default values ​​or dynamic default values ​​inferred from the rendering task type or context.

[0073] Therefore, by asking users for missing function or parameter information through multiple rounds of dialogue, the prompts output during the dialogue will guide users to provide specific functions, or help users select the parameters of the target function through options, ensuring that the workflow can always move towards the completion state and output the rendering result, thus improving the robustness and user-friendliness of the system.

[0074] In some embodiments of this application, the image rendering method may further include: displaying the rendering result; receiving modification information input by the user; adjusting the rendering result according to the modification information, or regenerating a new rendering result based on the modification information.

[0075] Specifically, image editing software generates rendered images or videos and displays the results to the user. Users can also download and save the rendered images or videos locally. Users can provide further feedback on the generated images or videos and choose whether to adjust the previous rendering or generate entirely new content. If adjustments are chosen, the rendering result is updated based on the received user input, such as further rotation or scaling. If new content is generated, the objective function and its parameters are redefined based on the user's modifications to generate a new rendering result.

[0076] In addition, a rendering template can be pre-set for each of the various scenarios. The rendering template is configured with a corresponding target function. Users can call the rendering template corresponding to the scenario, provide the parameters of the target function, and realize image rendering, thereby further improving rendering efficiency.

[0077] Figure 3 The flowchart illustrating the specific implementation of the image rendering method provided in this application is shown. After the user inputs a natural language rendering command, the large language model can parse and classify it into image rendering and animation rendering. At this point, the target function can be searched in the function library. If no target function is found, a prompt message is output to obtain supplementary information. Otherwise, based on the initial state of the target function and volume data, the parameters of the target function are extracted using the large language model. If no parameters of the target function are extracted, it indicates that the parameters of the target function are incomplete, and a prompt message is output. If the parameters of the target function are extracted, the function interface is called to perform rendering.

[0078] like Figure 4 The diagram shown is a structural schematic of an image rendering device 400 provided in an embodiment of this application. The image rendering device 400 is configured on an electronic device.

[0079] Specifically, the image rendering device 400 may include: The large model processing unit 401 is used to input natural language rendering instructions into the large language model and parse them to obtain rendering tasks. The types of rendering tasks include image rendering or animation rendering. The function query unit 402 is used to encode the target information of the rendering task into a query vector, and to match the target function based on the query vector in the function library. The parameter extraction unit 403 is used to extract the parameters of the target function according to the rendering task type; The image rendering unit 404 is used to call the function interface through the target function to output the rendering result based on the parameters of the function interface and the target function.

[0080] In some embodiments of this application, the parameter extraction unit 403 may be specifically used to: if the rendering task type is image rendering, extract the parameters of the objective function from the target information of the rendering task; if the rendering task type is animation rendering, extract the timing information of the animation from the target information of the rendering task, and generate the parameters of the key frame based on the timing information, wherein the key frame is associated with the natural language rendering instruction, and the parameters of the current key frame are obtained based on the parameter transformation of the previous key frame, or based on the accumulation of parameters of all historical key frames.

[0081] In some embodiments of this application, the image rendering unit 404 may be specifically used to: decompose the animation rendering task into sub-tasks corresponding to each key frame; call the function interface through the target function to render the image corresponding to each key frame based on the parameters of the function interface and the target function; and combine the images of each key frame according to the timing information to generate the rendering result.

[0082] In some embodiments of this application, the image rendering unit 404 may be specifically used to: generate one or more transition frames between two adjacent key frames according to a preset inter-frame interpolation strategy; and combine the images corresponding to each key frame and transition frame according to the timing information to generate a rendering result.

[0083] In some embodiments of this application, the image rendering apparatus 400 further includes an interaction unit, which can be specifically used to: output prompt information to obtain supplementary information about the target function and / or the parameters of the target function if the parameters of the target function are incomplete or do not match the target function.

[0084] In some embodiments of this application, the interaction unit may be specifically used to: if the parameters of the objective function are still incomplete after the number of output rounds of the prompt information reaches a preset threshold, then configure default values ​​and / or default ranges for the missing parameters of the objective function.

[0085] In some embodiments of this application, the image rendering apparatus 400 further includes a modification unit, which can be specifically used for: displaying rendering results; receiving modification information input by the user; adjusting the rendering results according to the modification information, or regenerating new rendering results based on the modification information.

[0086] In some embodiments of this application, the function query unit 402 may be specifically used to: determine a target function library from at least two function libraries according to the rendering task type, wherein the at least two function libraries include a first function library for image rendering and a second function library for animation rendering; and match a target function in the target function library based on a query vector.

[0087] It should be noted that, for the sake of convenience and brevity, the specific working process of the image rendering device 400 described above can be found in the following reference: Figures 1 to 3 The corresponding process of the methods will not be elaborated here.

[0088] like Figure 5 The diagram shown is a schematic representation of an electronic device according to an embodiment of this application. The electronic device 5 may include: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as an image rendering program. When the processor 50 executes the computer program 52, it implements the steps described in the various image rendering method embodiments above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 50 executes the computer program 52, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 4 The functions of the large model processing unit 401, function query unit 402, parameter extraction unit 403, and image rendering unit 404 are shown.

[0089] The computer program can be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0090] The electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0091] The processor 50 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0092] The memory 51 can be an internal storage unit of the electronic device, such as a hard drive or memory. The memory 51 can also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 51 can include both internal and external storage units. The memory 51 is used to store the computer program and other programs and data required by the electronic device. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0093] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned electronic device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0097] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0101] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image rendering method, characterized in that, include: Input natural language rendering instructions into a large language model, parse them to obtain rendering tasks, and the types of rendering tasks include image rendering or animation rendering. The target information of the rendering task is encoded into a query vector, and the target function is matched based on the query vector within the function library; Extract the parameters of the objective function based on the rendering task type; The function interface is called through the target function to output the rendering result based on the parameters of the function interface and the target function.

2. The image rendering method according to claim 1, characterized in that, The step of extracting the parameters of the objective function according to the rendering task type includes: If the rendering task type is image rendering, then the parameters of the objective function are extracted from the target information of the rendering task; If the rendering task type is animation rendering, then the timing information of the animation is extracted from the target information of the rendering task, and the parameters of the key frame are generated based on the timing information. The key frame is associated with the natural language rendering instruction, and the parameters of the current key frame are obtained based on the parameter transformation of the previous key frame, or based on the accumulation of parameters of all historical key frames.

3. The image rendering method according to claim 2, characterized in that, When the rendering task type is animation rendering, the image rendering method further includes: decomposing the animation rendering task into sub-tasks corresponding to each keyframe; The step of calling the function interface through the target function to output rendering results based on the parameters of the function interface and the target function includes: The target function calls the function interface to render the image corresponding to each keyframe based on the parameters of the function interface and the target function. The images of each keyframe are combined according to the time sequence information to generate the rendering result.

4. The image rendering method according to claim 3, characterized in that, The step of combining the images of each key frame according to the temporal information to generate the rendering result includes: Between two adjacent keyframes, one or more transition frames are generated according to a preset inter-frame interpolation strategy; The images corresponding to each key frame and the transition frame are combined according to the time sequence information to generate the rendering result.

5. The image rendering method according to claim 1, characterized in that, If the parameters of the objective function are incomplete or do not match the objective function, the image rendering method further includes: Output a prompt message to obtain supplementary information about the objective function and / or its parameters.

6. The image rendering method according to claim 5, characterized in that, If the parameters of the objective function are still incomplete after the number of output rounds of the prompt information reaches a preset threshold, the image rendering method further includes: Configure default values ​​and / or default ranges for missing parameters of the objective function.

7. The image rendering method according to any one of claims 1-6, characterized in that, Also includes: Display the rendering result; Receive modification information input by the user; The rendering result may be adjusted based on the modified information, or a new rendering result may be generated based on the modified information.

8. The image rendering method according to any one of claims 1-6, characterized in that, The step of encoding the target information of the rendering task into a query vector and matching the target function based on the query vector within the function library includes: The target function library is determined from at least two function libraries based on the rendering task type, the at least two function libraries including a first function library for image rendering and a second function library for animation rendering; The target function is matched based on the query vector in the target function library.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the image rendering method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, When the computer program is run, the image rendering method as described in any one of claims 1 to 8 is executed.

Images