Image rendering method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an image rendering method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of image rendering, and the method comprises the steps: carrying out the difference analysis of a material image, and determining N to-be-rendered images; respectively rendering the N images to be rendered to obtain rendering results; wherein the N to-be-rendered images comprise M target images, N and M are positive integers, and N is greater than or equal to M; under the condition that the number of the material images is greater than 1, each target image corresponds to at least two frames of material images; and / or under the condition that the number of the material images is greater than or equal to 1, each target image corresponds to at least two image areas in the material images. According to the invention, the problems of long rendering time and large occupied cloud computing power resources in the current image rendering mode can be solved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an image rendering method, device, electronic device and computer-readable storage medium. Background Art

[0002] The development of cloud computing has promoted the development of cloud rendering, giving birth to cloud rendering. Cloud computing has efficient resource utilization and dynamic characteristics through the high degree of virtualization of hardware resources and network storage, and can easily complete the high-performance computing required for offline rendering. Offline cloud rendering means that after the scene and rendering parameters are planned in advance, the computer can generate high-quality images or videos in a long time. It is often used in scenes that require high-quality rendering, such as animated films and advertisements. The current rendering processing solution basically uses the parallel capability of cloud computing power to render frame by frame and block by block according to the frame sequence. This rendering method takes a long time to render and requires a large amount of cloud computing resources. Summary of the invention

[0003] The present application provides an image rendering method, device, electronic device and computer-readable storage medium, which solve the problems of long rendering time and large occupation of cloud computing resources in current image rendering methods.

[0004] In order to achieve the above object, an embodiment of the present application provides an image rendering method, comprising:

[0005] Perform difference analysis on the material images to determine N images to be rendered;

[0006] Rendering the N images to be rendered respectively to obtain rendering results;

[0007] Wherein, the N images to be rendered include M target images, N and M are positive integers, and N≥M;

[0008] When the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material images.

[0009] Optionally, performing difference analysis on the material images to determine N images to be rendered includes:

[0010] In the case that there are multiple material images, a difference analysis is performed on different frames of material images, and one frame of material images among at least two frames of material images whose similarity is greater than or equal to a first threshold is determined as the image to be rendered.

[0011] Optionally, performing difference analysis on the material images to determine N images to be rendered includes:

[0012] Dividing the material image into a plurality of image regions;

[0013] Performing difference analysis on different image regions obtained by dividing the material image, and determining one image region among at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered.

[0014] Optionally, performing difference analysis on different image regions obtained by dividing the material image and determining one image region among at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered includes:

[0015] Performing difference analysis on different image regions belonging to the same material image, and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as an intermediate image to be rendered;

[0016] A difference analysis is performed on different intermediate images to be rendered that belong to different material images, and one intermediate image to be rendered among at least two intermediate images to be rendered whose similarity is greater than or equal to a second threshold is determined as the image to be rendered.

[0017] Optionally, dividing the material image into at least two image areas includes:

[0018] Dividing the material image into a plurality of image regions according to an image layering rule; wherein each image region corresponds to an image layer;

[0019] and / or,

[0020] The material image is divided into a plurality of image regions according to an image block rule; wherein each image region corresponds to an image block.

[0021] Optionally, dividing the material image into a plurality of image regions according to an image segmentation rule includes:

[0022] Dividing the material image into a plurality of image regions according to a rule that the image blocks have the same size;

[0023] or,

[0024] Determine the target image block size corresponding to the material image according to the first rendering duration, and divide the material image into multiple image areas according to the target image block size; wherein the first rendering duration includes: when the material image is divided according to different image block sizes, the total rendering time of the material image under each image block size.

[0025] Optionally, in the process of determining N images to be rendered, when frame sequence marking is performed for at least two frames of material images corresponding to each target image, and / or when region sequence marking is performed for at least two image regions in the same frame of material image corresponding to each target image, the method further includes:

[0026] According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence graph;

[0027] or,

[0028] According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence diagram, and performing video synthesis according to the frame sequence diagram to obtain a target video;

[0029] or,

[0030] According to the rendering result corresponding to the image to be rendered, a frame sequence of at least two frames of material images corresponding to the rendering result is subjected to video synthesis to obtain a target video.

[0031] In order to achieve the above object, the embodiment of the present application provides an image rendering device, including:

[0032] An analysis module, used for performing difference analysis on the material images and determining N images to be rendered;

[0033] A rendering module, used to render the N images to be rendered respectively to obtain rendering results;

[0034] Wherein, the N images to be rendered include M target images, N and M are positive integers, and N≥M;

[0035] When the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material images.

[0036] In order to achieve the above-mentioned purpose, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image rendering method described above when executing the computer program.

[0037] In order to achieve the above-mentioned purpose, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image rendering method described above are implemented.

[0038] In order to achieve the above-mentioned objective, an embodiment of the present application provides a computer program product, including computer instructions, which implement the steps of the image rendering method described above when executed by a processor.

[0039] In an embodiment of the present application, for material images, difference analysis is performed using frames and / or image areas in a single-frame material image as units to determine N images to be rendered, that is, the N images to be rendered include M target images, N and M are positive integers, N≥M; when the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material image, that is, the rendering results of M target images among the N images to be rendered determined through difference analysis during rendering can be reused for at least two frames of material images or at least two image areas in a single-frame material image, so that when rendering at least one frame of material image, the number of rendered frames and / or image areas can be reduced, thereby reducing the rendering time and reducing the occupation of cloud computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 One of the flowcharts of the image rendering method according to an embodiment of the present application;

[0041] Figure 2 A second flowchart showing the image rendering method according to an embodiment of the present application;

[0042] Figure 3 A schematic diagram showing a frame image in a 3D model according to an embodiment of the present application;

[0043] Figure 4 A schematic diagram showing the frame image segmentation of an embodiment of the present application;

[0044] Figure 5 A block diagram showing an image rendering device according to an embodiment of the present application;

[0045] Figure 6 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

[0047] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0048] In various embodiments of the present application, it should be understood that the order of execution of the following processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the terms "system" and "network" are often used interchangeably in this article.

[0049] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0050] The following is an introduction to the workflow of offline cloud rendering, which includes the following processes:

[0051] Upload process: The user-side device uploads 3D model files, materials, textures and other rendering materials to cloud storage;

[0052] Task scheduling: The received rendering tasks are assigned to the server cluster through the scheduling system to ensure that the rendering tasks can get reasonable computing resource allocation;

[0053] Rendering processing: On the cloud side, the rendering server is equipped with professional rendering software and starts rendering work according to the settings specified by the user-side device.

[0054] Result acquisition: Users can preview the completed rendering work in the cloud or download the completed rendering results to local operation.

[0055] The following is an introduction to some technical terms in cloud rendering technology:

[0056] Frame: The smallest unit of video editing. A sequence of frames is a series of pictures, and a quick display is an animation.

[0057] Rendering: refers to the process of converting a three-dimensional model or scene into a two-dimensional image.

[0058] Block rendering: A rendering method in which the area to be rendered of the model frame is divided into several blocks, and the blocks are rendered in parallel.

[0059] Cloud rendering: Similar to cloud computing, it uses cloud computing power for concurrent rendering. Usually each rendering task will generate a large number of sequence frame images.

[0060] Traditional offline cloud rendering renders frame by frame and block by block according to the model uploaded by the user-side device, and fully utilizes the capabilities of cloud computing through parallel rendering. This method does not optimize the rendering task in the process of completing the rendering task, and does not save resources in terms of overall resource usage. In addition, for most rendering tasks, animation sequence frames have the characteristics of continuous changes, and there are strong correlations and similarities between various regions within the frame and between frames. Traditional rendering methods do not consider the shape correlation of rendering tasks. After rendering, each frame of the picture has only slight differences. After block division, only a few blocks of adjacent frame images have differences, which causes the phenomenon of repeated rendering of block images, which increases the rendering time and wastes cloud computing resources.

[0061] like Figure 1 As shown, the embodiment of the present application provides an image rendering method, comprising the following steps:

[0062] Step 11: Perform difference analysis on the material images to determine N images to be rendered; wherein the N images to be rendered include M target images, N and M are positive integers, and N≥M; when the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material image.

[0063] Optionally, before step 11, the method may further include: obtaining a material image. For example, the image rendering method may be applied to a cloud server, and the material image may be uploaded by a user-side device. For example, a user-side device may upload a video material, and a video material is usually composed of multiple frames of material images, that is, the material image is obtained, and the number of the material images may be multiple frames. Alternatively, the user-side device may upload one or more frames of picture material, that is, the material image is obtained, and the number of the material images may be one or more frames.

[0064] Optionally, when performing difference analysis, difference analysis may be performed on one frame of material image, such as performing difference analysis on multiple image areas in one frame of material image; or difference analysis may be performed on multiple needle material images, etc. The embodiments of the present application are not limited thereto.

[0065] Example 1: If the difference analysis is performed in frames, the number of images to be rendered N can be less than or equal to the number of material images. For example, when the number of images to be rendered N is equal to the number of material images, one image to be rendered corresponds to one frame of material; for another example, when the number of images to be rendered N is less than the number of material images, the N images to be rendered contain at least one target image, and each target image corresponds to at least two frames of material images, that is, the rendering result of the target image can be used as the rendering result of the corresponding at least two frames of material images.

[0066] Example 2: If the difference analysis is performed in units of image regions, the number N of images to be rendered can be greater than, less than, or equal to the number of source images. For example, each source image frame can include one or more image regions, and a target image can correspond to at least two image regions in the same source image frame, that is, the rendering result of the target image can be used as the rendering result of at least two image regions in the same source image frame corresponding to it; and / or, a target image can correspond to at least two image regions in different source images, that is, the rendering result of the target image can be used as the rendering result of at least two image regions in different source images corresponding to it.

[0067] Example 3: If the difference analysis is performed in units of frames and image areas, for example, the difference analysis is first performed in units of frames and then in units of image areas, a combination of the above examples 1 and 2 can be used, which will not be repeated here.

[0068] Step 12: Render the N images to be rendered respectively to obtain rendering results.

[0069] In an embodiment of the present application, for material images, difference analysis is performed using frames and / or image areas in a single-frame material image as units to determine N images to be rendered, that is, the N images to be rendered contain M target images, N and M are positive integers, and N≥M; when the number of material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material image, that is, when rendering, the rendering results of M target images in the N images to be rendered determined through difference analysis can be reused for at least two frames of material images or at least two image areas in a single-frame material image. In this way, when rendering at least one frame of material image, the number of rendered frames and / or image areas can be reduced, thereby reducing the rendering time and reducing the occupation of cloud computing resources.

[0070] Optionally, performing difference analysis on the material images to determine N images to be rendered includes:

[0071] In the case that there are multiple material images, a difference analysis is performed on different frames of material images, and one frame of material images among at least two frames of material images whose similarity is greater than or equal to a first threshold is determined as the image to be rendered.

[0072] Among them, determining one frame of material image among at least two frames of material images whose similarity is greater than or equal to a first threshold as the image to be rendered may specifically include: determining one frame of material image with an earlier frame number (or a smaller frame number) among at least two frames of material images whose similarity is greater than or equal to the first threshold as the image to be rendered; or, determining any one frame of material image among at least two frames of material images whose similarity is greater than or equal to the first threshold as the image to be rendered; or, determining one frame of material image with a later frame number (or a larger frame number) among at least two frames of material images whose similarity is greater than or equal to the first threshold as the image to be rendered, etc. The embodiments of the present application are not limited to this.

[0073] For example, an artificial intelligence (AI) model can be used to perform difference analysis on different frames of material images. For example, taking the case of determining a frame of material image with an earlier frame number (or a smaller frame number) among at least two frames of material images whose similarity is greater than or equal to a first threshold as the image to be rendered, first use AI training to analyze the material images of each frame within the frame range as a whole. If it is found that the similarity of the material images of adjacent frames reaches the first threshold (for example, the first threshold can be set to 99% or other values, etc.), the next frame of material image is marked as not requiring rendering and the rendering effect of the previous frame is fully reused, thereby reducing the rendering of one frame of image. In this way, by inputting multiple frames of material images into the AI ​​model, N images to be rendered can be screened, that is, the number of rendered image frames can be achieved.

[0074] Specifically, we can collect a large number of different three-dimensional (3D) model files, extract the basic rendering feature data such as geometric data, primitive information, fragment model, texture, etc. involved in each frame of the material image, and then use the training algorithm to train the AI ​​model to detect the camera parameters, light source position and intensity, color, texture and other parameters in the material image. The parameters can be used to identify complex factors such as light changes, shadows, reflections, and people in the material image. At the same time, based on these parameters, the pixel data of the material image can be calculated, and the changes between different material images can be judged based on the differences in pixel data. The greater the difference, the faster the change.

[0075] Optionally, performing difference analysis on the material images to determine N images to be rendered includes:

[0076] Dividing the material image into a plurality of image regions;

[0077] Performing difference analysis on different image regions obtained by dividing the material image, and determining one image region among at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered.

[0078] For example, the material image is divided into a plurality of image regions by dividing a frame of the material image into a plurality of image regions. Alternatively, the material image is divided into a plurality of image regions by dividing each frame of the material image in the plurality of frames of the material image into a plurality of image regions, for example, each frame of the material image in the acquired N1 frames of the material image is divided into a plurality of image regions; or a difference analysis is performed on the acquired N1 frames of the material image to determine the N2 frames of the material image, and then each frame of the material image in the N2 frames of the material image is divided into a plurality of image regions; wherein N1 and N2 are positive integers, and N1 is greater than N2.

[0079] That is to say, dividing the material image into multiple image areas may specifically include: for at least one first image, dividing each of the first images into at least two image areas; wherein the first image is the material image, or the first image corresponds to at least two frames of material images.

[0080] Example B1: Divide the obtained N1 frames of material images into multiple image areas, that is, when at least one first image is a material image, specifically for at least one frame of material images, each frame of the material image is divided into at least two image areas; perform difference analysis on the different image areas obtained by dividing the at least one frame of material images, and determine one image area among the at least two image areas whose similarity is greater than or equal to a second threshold as the image to be rendered.

[0081] Example B2: N2 material images determined based on the obtained N1 frame material image are divided into multiple image areas. That is, when at least one first image corresponds to at least two frames of material images (for example, the first image is a target image determined by performing difference analysis in frames), specifically for at least one target image, each target image is divided into at least two image areas; a difference analysis is performed on different image areas obtained by dividing the at least one target image, and one image area among the at least two image areas whose similarity is greater than or equal to a second threshold is determined as the image to be rendered.

[0082] Example B3: When at least one first image includes both a material image and a target image, the process of determining N images to be rendered may adopt a combination of the above-mentioned Example B1 and Example B2, which will not be described in detail here.

[0083] For example, determining one of at least two image regions whose similarity is greater than or equal to the second threshold as the image to be rendered may specifically include: for at least two image regions belonging to the same material image, determining one of at least two image regions whose similarity is greater than or equal to the second threshold (for example, the value of the second threshold may be 99% or other values, etc.) as the image to be rendered. For example: for at least two image regions belonging to the same material image, any one of at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered; or when numbering each image region when dividing the image region, an image region with a forward number (or a smaller number) in at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered; or an image region with a backward number (or a larger number) in at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered, etc. The embodiments of the present application are not limited thereto.

[0084] For another example, determining one of at least two image regions whose similarity is greater than or equal to the second threshold as the image to be rendered may specifically include: for at least two image regions belonging to different material images, determining one of at least two image regions whose similarity is greater than or equal to the second threshold (for example, the value of the second threshold may be 99% or other values, etc.) as the image to be rendered. For example: for at least two image regions belonging to different material images, any one of at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered; or when numbering each image region when dividing the image region, an image region whose frame sequence number and number are at the front (or whose frame sequence number and number are smaller) among at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered; or an image region whose frame sequence number and number are at the back (or whose frame sequence number and number are larger) among at least two image regions whose similarity is greater than or equal to the second threshold may be determined as the image to be rendered, etc. The embodiments of the present application are not limited thereto.

[0085] For another example, performing difference analysis on different image regions obtained by dividing the material image and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered specifically includes:

[0086] Performing difference analysis on different image regions belonging to the same material image, and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as an intermediate image to be rendered;

[0087] A difference analysis is performed on different intermediate images to be rendered that belong to different material images, and one intermediate image to be rendered among at least two intermediate images to be rendered whose similarity is greater than or equal to a second threshold is determined as the image to be rendered.

[0088] For example, when each of the multiple material images is divided into multiple image regions, a difference analysis can be first performed according to different image regions belonging to the same material image, and one of the at least two image regions whose similarity is greater than or equal to the second threshold is determined as the intermediate image to be rendered, and then a difference analysis can be performed according to different intermediate images to be rendered belonging to different material images, and one of the at least two intermediate images to be rendered whose similarity is greater than or equal to the second threshold is determined as the image to be rendered. The specific process of determining the intermediate image to be rendered and the image to be rendered can be referred to the above embodiment, and will not be repeated here.

[0089] Optionally, dividing the material image into at least two image areas includes:

[0090] Divide the material image into a plurality of image regions according to an image layering rule; wherein each image region corresponds to an image layer; for example, when there are a plurality of material images, each of the material images may be divided into a plurality of image regions according to the image layering rule;

[0091] and / or,

[0092] The material image is divided into a plurality of image regions according to an image block rule; wherein each image region corresponds to an image block; for example, when there are a plurality of material images, each of the material images may be divided into a plurality of image regions according to the image block rule.

[0093] For example, the rule of image layering is to divide each image layer in the material image into different image areas, such as image layering including but not limited to: background layer, foreground layer, intermediate layer between background layer and foreground layer, etc., and the embodiments of the present application are not limited to this. In this embodiment, by adopting the image layering rule to divide each of the multiple material images into multiple image areas, the image to be rendered is determined based on the difference analysis between the divided multiple image areas, and a single rendering can be performed for similar background information or foreground information between different frame images, thereby reducing the rendering time and improving the rendering efficiency.

[0094] For another example, the image segmentation rule is to divide the material image into different blocks. For example, the image segmentation rule includes but is not limited to: the number of blocks remains unchanged, the block size remains unchanged, the block size is adaptive, the number of blocks is adaptive, etc., and the embodiments of the present application are not limited thereto. In this embodiment, the material image is divided into multiple image areas by adopting the image segmentation rule, so that the image to be rendered is determined based on the difference analysis between the multiple divided image areas, and a single rendering can be performed for similar partial image areas in the same frame or between different frame images, thereby reducing the rendering time and improving the rendering efficiency.

[0095] Optionally, dividing the material image into a plurality of image regions according to an image segmentation rule includes:

[0096] Divide the material image into a plurality of image regions according to a rule that the image blocks have the same size; for example, when there are a plurality of material images, each of the material images may be divided into a plurality of image regions according to a rule that the image blocks have the same size (or the same number of image blocks);

[0097] or,

[0098] Determine the target image block size corresponding to the material image according to the first rendering duration, and divide the material image into multiple image areas according to the target image block size; wherein the first rendering duration includes: when the material image is divided according to different image block sizes, the total rendering duration of the material image under each image block size; for example: in the case where there are multiple material images, it can be that for each of the material images, the target image block size corresponding to the material image is determined according to the first rendering duration, and the material image is divided into multiple image areas according to the target image block size.

[0099] For example, taking each frame as 4*4, each of the material images is divided according to the rule of the same image block size, then each material image corresponds to 16 image areas, and the 16 image areas of each frame are numbered. First, the image areas in the same frame are compared. If the similarity between two image areas reaches a second threshold (for example, the second threshold is 99% or other values), the image area with a later number is marked as not requiring rendering and completely reusing the rendering result of the previous image area; then, the image areas between adjacent frames are compared. If the similarity between some image areas between the two frames reaches a second threshold (for example, the second threshold is 99% or other values), the similar image areas in the next frame of the frame sequence are marked as not requiring rendering and completely reusing the corresponding image areas in the previous frame.

[0100] For example, the rule of image block size adaptation (or image block number adaptation) can be that when the material image is divided into different image block sizes, the total rendering time of the material image at each image block size is used to determine the image block size (or image block number). For example, an AI model can be used to dynamically adjust the image block size (or image block number) of each frame. Specifically, parameters such as resolution and rendering quality settings can be detected. Based on these parameters, the AI ​​model is trained to simulate the rendering calculation process to output the rendering time of the same material image at different image block sizes (or image block numbers). For example, the relationship between the rendering time is:

[0101] F(res)=(X*α+Y)*res

[0102] Among them, F(res) is the rendering time, X is the scene complexity, α is the compensation factor, Y is the rendering quality parameter, and res is the block resolution.

[0103] For example: when the scene in the video involves fewer complex factors or changes slowly, the entire frame sequence can use a fixed image block size (or number of image blocks); or when the scene in the video involves more complex factors or changes quickly, and the frame rates vary greatly, if the reuse rate is low when a unified image block size (or number of image blocks) is used, the rendering groups can be divided according to the frame rate (each rendering group can include one or more frames of material images), and then the material images in each rendering group use a custom image block size (or number of image blocks). For example, a video includes 2400 frames and a frame rate of 24, which can be divided into 100 rendering groups. The image block size (or number of image blocks) of the material images in each rendering group is determined based on factors such as rendering performance.

[0104] Optionally, in the process of determining N images to be rendered, when frame sequence marking is performed for at least two frames of material images corresponding to each target image, and / or when region sequence marking is performed for at least two image regions in the same frame of material image corresponding to each target image, the method further includes:

[0105] According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence graph;

[0106] or,

[0107] According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence diagram, and performing video synthesis according to the frame sequence diagram to obtain a target video;

[0108] or,

[0109] According to the rendering result corresponding to the image to be rendered, a frame sequence of at least two frames of material images corresponding to the rendering result is subjected to video synthesis to obtain a target video.

[0110] For example: when region sequence marking is performed on at least two image areas in the same frame of material image corresponding to each target image, the region sequence of the image area corresponding to each rendering result can be obtained, so that at least one frame sequence diagram can be synthesized based on the corresponding rendering results in the order of the region sequence of the image area.

[0111] For another example: when frame sequence marking is performed for at least two frames of material images corresponding to each target image, and area sequence marking is performed for at least two image areas in the same frame of material image corresponding to each target image, the area sequence and frame sequence of the image area corresponding to each rendering result can be obtained. In this way, at least one frame sequence diagram can be synthesized based on the corresponding rendering results in the order of the area sequences of each image area of ​​the same frame sequence, and then the target video can be synthesized in the order of the frame sequence of the at least one frame sequence diagram.

[0112] For another example: when frame sequence marking is performed on at least two frames of material images corresponding to each target image, the frame sequence of the material image corresponding to each rendering result can be obtained, so that the target video can be synthesized based on the corresponding rendering results in the order of the frame sequence of the material image.

[0113] The image rendering method of the present application is introduced below in conjunction with specific embodiments. Figure 2 As shown, the specific process includes:

[0114] Step 21: User model and material submission.

[0115] Combined with the capabilities of 3D modeling software, a large number of 3D model files are used to train the AI ​​model to analyze factors such as scene content, parameters, lighting, texture, etc. in 3D models, with the goal of accurately outputting the differences in rendering content between each frame sequence. Users upload a rendering material, analyze it, and set parameters before submitting it for rendering.

[0116] Step 22: Number the frames in the order in which the model will be played.

[0117] Generally, rendering tasks are to output sequence frame images and then synthesize them into video for playback. A sequence frame image is usually a segment of animation decomposed according to the frame rate, so the difference between adjacent frames is small. Figure 3 As shown, the 3D model represents the movement trajectory of a piece of cloth blowing in the wind. We extract three adjacent frames of rendered images to illustrate the analysis process, marked as frame 1, frame 2, and frame 3 respectively.

[0118] Step 23: Perform similarity analysis based on artificial intelligence generated content (AIGC). If the similarity exceeds a threshold (e.g., 99%), skip rendering and save the mapping relationship between frames and blocks for rendering result synthesis.

[0119] Step 24: Task scheduling and rendering.

[0120] Perform a preliminary analysis of the material to obtain some basic parameters, such as the frame range and frame rate that need to be rendered, which are the basis for subsequent difference analysis and rendering. First, use AI training to analyze the images in the frame range as a whole. If the similarity between adjacent frames exceeds the first threshold (such as 99%), the next frame will be marked as not requiring rendering and the rendering effect of the previous frame will be fully reused, thus reducing the rendering of one frame of the image. For example, if frames 1 and 2 in 3 are almost completely similar, then only frame 1 needs to be rendered, which is equivalent to completing the rendering of frames 1 and 2.

[0121] Based on the above steps, the frames with differences (i.e., frame 1 and frame 3) have been marked, and these frames with differences can be further divided into blocks. Figure 4 As shown, each frame is divided into 4*4, a total of 16 blocks, and they are numbered separately, and then analyzed by block. First, compare the image blocks in the same frame. If the similarity between two image blocks exceeds the second threshold (for example, 99%), the latter image block is marked as not needing to be rendered and the previous image block is fully reused. For example: the similarity of image blocks 1, 5, and 9 to 16 in frame 1 reaches 99%. In this way, only image block 1 needs to be rendered, which is equivalent to completing the rendering of nine image blocks 1, 5, and 9 to 16. Then the rendering task of this frame of the picture is reduced from the original 16 image blocks to 7 image blocks, reducing resource usage; then compare the image blocks between adjacent frames. If the similarity of some picture blocks between two frames exceeds the second threshold (for example, 99%), these similar picture blocks of the latter frame are marked as not needing to be rendered and the similar picture blocks of the previous frame are completely reused. For example, if the similarity of picture blocks 1 and 9 to 16 of frame 3 and picture blocks 1, 5 and 9 to 16 of frame 1 reaches 99%, these picture blocks of frame 3 can be marked as the rendering results of picture block 1 in frame 1 that are reused without repeated rendering. In this way, the rendering task of frame 3 can be reduced from 16 picture blocks to 7 picture blocks, further reducing resource utilization.

[0122] Based on the above process, it can be seen that only image blocks 1 to 4 and 6 to 8 of frame 1 and image blocks 2 to 8 of frame 3 need to be rendered to complete the rendering of three frames. In this way, the number of rendering tasks is reduced from 48 to 14, which not only reduces the resource occupancy rate, but also improves the rendering efficiency, ultimately achieving the purpose of cost saving.

[0123] Optionally, considering that the complexity of 3D models in actual situations varies, the block strategy in the above process can be dynamically adjusted through AI training, such as fixed block size or adaptive block size. For example: by detecting parameters such as resolution and rendering quality settings, AI is trained based on these parameters to simulate the rendering calculation process to output the rendering time of the same frame image under different block sizes, and the rendering time is used as the basis for subsequent block division. If the scene in the video involves fewer complex factors or changes slowly, a fixed block size can be used for the entire frame sequence; if the scene in the video involves more complex factors or changes quickly, and the frame rate difference is large, considering that the reuse rate is low when using a unified block size, the rendering group can be divided according to the frame rate, and then a custom block size is used in each rendering group. For example: a video with 2400 frames and a frame rate of 24 can be divided into 100 rendering groups, and the block size in each rendering group is determined based on factors such as rendering performance.

[0124] In addition, before segmentation, frames can also be layered based on AI training. Block segmentation is to first segment each frame, and then perform local analysis and local reuse. The advantage of layering is that each frame is compared as a whole. The unchanged parts of the entire frame sequence or a frame sequence can be separated for separate rendering, and then reused based on annotations during image synthesis. This can also improve some rendering efficiency. For example, for common video backgrounds, the entire video generally does not change. In this way, the background can be separated and rendered separately, and then the background can be ignored when comparing the differences in block rendering, which reduces the complexity of the scene and improves rendering performance.

[0125] Step 25: Record frame-by-frame synthesis based on AIGC analysis and save the rendering results.

[0126] After rendering is completed, the frame sequence diagram is synthesized according to the AI ​​analysis and annotation results, and the rendering results are saved. Finally, the video is synthesized according to the sequence frames and bit rate for playback.

[0127] In an embodiment of the present application, when rendering a single image, by dividing the image into blocks and analyzing the similarities between the blocks (including global comparison and nearby comparison), similar image blocks are obtained. When rendering continuous images, by dividing adjacent images into blocks, the trained AI model is used to analyze and compare the similarities of the corresponding blocks of adjacent images to obtain similar image block information. In this way, by analyzing the similar areas of the rendering task, similar areas are only rendered once during the rendering process. Through this discovery and marking mechanism, the rendering workload is reduced, thereby optimizing the rendering process.

[0128] And in the embodiment of the present application, combined with the 3D modeling software capabilities, a large number of 3D model files are used to train the AI ​​large model to analyze the scene content, parameters, lighting, texture and other factors in the 3D model, and the goal is to accurately output the rendering content differences between each frame sequence. By discovering the similarity between adjacent blocks in a single frame image and the corresponding blocks between adjacent images, and marking highly similar image blocks, highly similar image blocks are only rendered once, and the rendering results are reused, and the rendering efficiency is improved by reducing the number of renderings, saving resources and reducing costs. And by collecting a large number of different 3D model files, extracting the basic rendering feature data such as geometric data, primitive information, fragment models, textures, etc. involved in the frame image, and then using the training algorithm for AI training, the complex factors such as light, shadow, reflection, and characters in the picture and the changes between different frame images can be identified, and the rendering performance of different block sizes can be further evaluated based on these factors. According to these analysis results, it can be determined how to divide the blocks appropriately to improve the reuse rate.

[0129] like Figure 5 As shown, the embodiment of the present application provides an image rendering device 500, including:

[0130] An analysis module 510 is used to perform difference analysis on the material images and determine N images to be rendered;

[0131] A rendering module 520, used to render the N images to be rendered respectively to obtain rendering results;

[0132] Wherein, the N images to be rendered include M target images, N and M are positive integers, and N≥M;

[0133] When the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material images.

[0134] Optionally, the analysis module 510 includes:

[0135] The first analysis unit is used to perform difference analysis on different frames of material images when there are multiple material images, and determine one frame of material images among at least two frames of material images whose similarity is greater than or equal to a first threshold as the image to be rendered.

[0136] Optionally, the analysis module 510 includes:

[0137] A division unit, used for dividing the material image into a plurality of image regions;

[0138] The second analysis unit is used to perform difference analysis on different image regions obtained by dividing the material image, and determine one image region among at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered.

[0139] Optionally, the second analysis unit is further used for:

[0140] Performing difference analysis on different image regions belonging to the same material image, and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as an intermediate image to be rendered;

[0141] A difference analysis is performed on different intermediate images to be rendered that belong to different material images, and one intermediate image to be rendered among at least two intermediate images to be rendered whose similarity is greater than or equal to a second threshold is determined as the image to be rendered.

[0142] Optionally, the dividing unit is further used for:

[0143] Dividing the material image into a plurality of image regions according to an image layering rule; wherein each image region corresponds to an image layer;

[0144] and / or,

[0145] The material image is divided into a plurality of image regions according to an image block rule; wherein each image region corresponds to an image block.

[0146] Optionally, when the dividing unit is used to divide the material image into a plurality of image regions according to an image block rule, the method includes:

[0147] Dividing the material image into a plurality of image regions according to a rule that the image blocks have the same size;

[0148] or,

[0149] Determine the target image block size corresponding to the material image according to the first rendering duration, and divide the material image into multiple image areas according to the target image block size; wherein the first rendering duration includes: when the material image is divided according to different image block sizes, the total rendering time of the material image under each image block size.

[0150] Optionally, in the process of determining N images to be rendered, when frame sequence marking is performed for at least two frames of material images corresponding to each target image, and / or when region sequence marking is performed for at least two image regions in the same frame of material image corresponding to each target image, the image rendering device 500 further includes:

[0151] A first synthesis module, configured to perform frame synthesis on a region sequence of at least two image regions corresponding to the rendering result according to the rendering result corresponding to the image to be rendered, so as to obtain at least one frame sequence image;

[0152] or,

[0153] A second synthesis module is used to perform frame synthesis on a region sequence of at least two image regions corresponding to the rendering result according to the rendering result corresponding to the image to be rendered, to obtain at least one frame sequence diagram, and to perform video synthesis according to the frame sequence diagram to obtain a target video;

[0154] or,

[0155] The third synthesis module is used to perform video synthesis on a frame sequence of at least two frames of material images corresponding to the rendering result according to the rendering result corresponding to the image to be rendered, so as to obtain a target video.

[0156] It should be noted that the above-mentioned image rendering device and the above-mentioned image rendering method of the embodiment of the present application are based on the same inventive concept, which can implement the various embodiment processes of the above-mentioned image rendering method and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0157] like Figure 6 As shown, an embodiment of the present application further provides an electronic device. Optionally, the electronic device can be a cloud rendering server, including a transceiver 63, a processor 61, a memory 62, and a computer program stored in the memory 62 and executable on the processor 61. When the processor 61 executes the computer program, the steps of the above-mentioned image rendering method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0158] The bus architecture may include any number of interconnected buses and bridges, specifically one or more processors 61 represented by processor 61 and various circuits of memory 62 represented by memory 62 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 63 may be a plurality of components, i.e., a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. The processor 61 is responsible for managing the bus architecture and general processing, and the memory 62 may store data used by the processor when performing operations.

[0159] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing relevant hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.

[0160] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing relevant hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.

[0161] In addition, a specific embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps in the above-mentioned image rendering method are implemented. And the same technical effect can be achieved, so it will not be repeated here to avoid repetition.

[0162] The embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the various processes of the embodiment of the above-mentioned image rendering method and can achieve the same technical effect. To avoid repetition, they are not described here.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0165] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0166] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications are also within the scope of protection of the present application.

Claims

1. An image rendering method, characterized in that: include: Perform difference analysis on the material images to determine N images to be rendered; Rendering the N images to be rendered respectively to obtain rendering results; Wherein, the N images to be rendered include M target images, N and M are positive integers, and N≥M; When the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material images.

2. The image rendering method according to claim 1, characterized in that: The performing difference analysis on the material images to determine N images to be rendered includes: In the case that there are multiple material images, a difference analysis is performed on different frames of material images, and one frame of material images among at least two frames of material images whose similarity is greater than or equal to a first threshold is determined as the image to be rendered.

3. The image rendering method according to claim 1 or 2, characterized in that: The performing difference analysis on the material images to determine N images to be rendered includes: Dividing the material image into a plurality of image regions; Performing difference analysis on different image regions obtained by dividing the material image, and determining one image region among at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered.

4. The image rendering method according to claim 3, characterized in that: The performing difference analysis on different image regions obtained by dividing the material image, and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as the image to be rendered, includes: Performing difference analysis on different image regions belonging to the same material image, and determining one of at least two image regions whose similarity is greater than or equal to a second threshold as an intermediate image to be rendered; A difference analysis is performed on different intermediate images to be rendered that belong to different material images, and one intermediate image to be rendered among at least two intermediate images to be rendered whose similarity is greater than or equal to a second threshold is determined as the image to be rendered.

5. The image rendering method according to claim 3, characterized in that: The dividing the material image into at least two image areas comprises: Dividing the material image into a plurality of image regions according to an image layering rule; wherein each image region corresponds to an image layer; and / or, The material image is divided into a plurality of image regions according to an image block rule; wherein each image region corresponds to an image block.

6. The image rendering method according to claim 5, characterized in that: The step of dividing the material image into a plurality of image regions according to an image segmentation rule comprises: Dividing the material image into a plurality of image regions according to a rule that the image blocks have the same size; or, Determine the target image block size corresponding to the material image according to the first rendering duration, and divide the material image into multiple image areas according to the target image block size; wherein the first rendering duration includes: when the material image is divided according to different image block sizes, the total rendering time of the material image under each image block size.

7. The image rendering method according to claim 1, characterized in that: In the process of determining N images to be rendered, when performing frame sequence marking for at least two frames of material images corresponding to each target image, and / or performing region sequence marking for at least two image regions in the same frame of material image corresponding to each target image, the method further includes: According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence graph; or, According to the rendering result corresponding to the image to be rendered, performing frame synthesis on a region sequence of at least two image regions corresponding to the rendering result to obtain at least one frame sequence diagram, and performing video synthesis according to the frame sequence diagram to obtain a target video; or, According to the rendering result corresponding to the image to be rendered, a frame sequence of at least two frames of material images corresponding to the rendering result is subjected to video synthesis to obtain a target video.

8. An image rendering device, characterized in that: include: An analysis module, used for performing difference analysis on the material images and determining N images to be rendered; A rendering module, used to render the N images to be rendered respectively to obtain rendering results; Wherein, the N images to be rendered include M target images, N and M are positive integers, and N≥M; When the number of the material images is greater than 1, each of the target images corresponds to at least two frames of material images; and / or, when the number of the material images is greater than or equal to 1, each of the target images corresponds to at least two image areas in the material images.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image rendering method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image rendering method according to any one of claims 1 to 7 are implemented.

Images