A method and system for rendering images through a generative digital model

Abstract

The present invention relates to the field of image rendering technology, and in particular to a method and system for rendering images through a generative digital model. Based on the rendering perspective and the direction of light source light, the present invention traverses the rendering features of pixels in the superpixel block of any rendered image for analysis, obtains the texture rendering quality of each superpixel block, and screens out light and shadow affected blocks to obtain the light and shadow rendering quality of each light and shadow affected block; according to the distribution of similarities between each light and shadow affected block and the same area of ​​other different rendered images in the corresponding image cluster, and the difference between the corresponding images, obtains the rendering quality stability of each light and shadow affected block; and then obtains the overall rendering quality of each rendered image, screens out high-quality rendered images, and renders the rendered object. The present invention screens out high-quality rendered images, trains to obtain a more accurate generative model, and improves the rendering quality of the generated image.

Description

Technical Field

[0001] The present invention relates to the technical field of image rendering, and in particular to a method and system for rendering an image through a generative digital model. Background Art

[0002] Rendering is the technology that converts three-dimensional geometric data, textures, lighting information, etc. into a final two-dimensional image or video frame. It not only includes simple geometric projection, but also covers complex physical lighting models to ensure that the generated image conforms to the visual laws of the real world.

[0003] In the prior art, generative digital models are used to generate new data that is similar but not identical by learning from a large number of data samples of rendered images. This has a wide range of applications in data rendering. However, since the rendering quality of a three-dimensional scene rendering image dataset depends on multiple factors such as the rendering algorithm, lighting model, material properties, and camera position, there is a problem of inconsistent rendering quality. Directly using unscreened rendered images during the training of generative digital models may lead to the learning of image features with low rendering quality and obvious defects, which will affect the subsequent model rendering quality. Summary of the invention

[0004] In order to solve the technical problem that images with low rendering quality and obvious defects are not screened out during training, which affects the quality of subsequent model rendering, the purpose of the present invention is to provide a method and system for rendering images through a generative digital model. The technical solutions adopted are as follows:

[0005] The present invention proposes a method for rendering an image using a generative digital model, the method comprising:

[0006] Obtaining multi-dimensional rendering parameters corresponding to each rendered image of the rendering object and rendering texture parameters of each pixel point, wherein the multi-dimensional rendering parameters include: rendering viewing angle and light source direction;

[0007] According to the relative distances of multi-dimensional rendering parameters between different rendered images, all rendered images are clustered to obtain image clusters; according to the rendering perspective of each rendered image and the position distribution between the rendered objects, the perspective distance of each pixel in each rendered image is obtained;

[0008] For any rendered image, multiple superpixel blocks are obtained; the texture rendering quality of each superpixel block is obtained according to the fluctuation distribution of the rendering texture parameters and saturation characteristics of the pixels in different superpixel blocks; the light and shadow affected blocks are screened out according to the distribution of the pixels in the direction of the light source within the neighborhood of each superpixel block; the light and shadow rendering quality of each light and shadow affected block is obtained by combining the viewing angle distance and brightness characteristics of the pixels; the rendering quality stability of each light and shadow affected block is obtained according to the distribution of the similarity between each light and shadow affected block and the same area of ​​other different rendered images in the corresponding image cluster, as well as the difference between the corresponding images;

[0009] According to the texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block in each rendered image, the overall rendering quality of each rendered image is obtained; according to the overall rendering quality of all rendered images, high-quality rendering images are screened out and the rendering objects are rendered.

[0010] Furthermore, the method for obtaining the viewing angle distance includes:

[0011] For any rendered image, a plane perpendicular to the rendering view direction is constructed at the rendering view position, and the distance between each pixel point on the rendered object and the plane is obtained as the view distance of each pixel point on the corresponding rendered image.

[0012] Furthermore, the method for acquiring the texture rendering quality includes:

[0013] Obtaining fluctuation characteristics of rendering texture parameters of all pixels in each superpixel block as texture fluctuation characteristics;

[0014] Obtain the fluctuation characteristics of the component values ​​of the corresponding saturation channel of all pixel points in each superpixel block in the HSV space as the saturation fluctuation characteristics;

[0015] The texture fluctuation features and saturation fluctuation features of all superpixel blocks are arranged according to the same sorting principle. The texture rendering quality of each superpixel block is obtained according to the difference in arrangement order between the texture fluctuation features and saturation fluctuation features corresponding to each superpixel block. The difference in arrangement order is negatively correlated with the texture rendering quality.

[0016] Furthermore, the method for obtaining the light and shadow impact blocks includes:

[0017] The neighborhood range of each superpixel block is traversed along the direction of the light source to obtain multiple pixel point sequences; if there are other pixel points in the neighborhood range of the pixel point sequence that are distributed before the pixel points of the superpixel block, the corresponding pixel point sequence is used as the target pixel point sequence, and the corresponding superpixel block is used as the light and shadow affected block.

[0018] Furthermore, the method for obtaining the light and shadow rendering quality includes:

[0019] For any light and shadow affected block, obtain the ratio of the average value of the viewing angle distance between other pixels before the light and shadow affected block pixel in the target pixel sequence and the light and shadow affected block pixel, and normalize it as the first light and shadow feature;

[0020] The component value of the brightness channel corresponding to each pixel in the HSV space is obtained as the brightness feature, and the ratio of the brightness feature mean between the light and shadow affected block pixel and the corresponding other pixels in the target pixel sequence is obtained and normalized as the second light and shadow feature;

[0021] According to the difference between the first light and shadow features and the second light and shadow features of different target pixel point sequences in each light and shadow impact block, the light and shadow rendering quality of each light and shadow impact block is obtained, and the difference between the first light and shadow features and the second light and shadow features is negatively correlated with the light and shadow rendering quality.

[0022] Furthermore, the method for obtaining the rendering quality stability includes:

[0023] The SSIM function is used to obtain the structural similarity between each light and shadow affected block and the same region of each other rendered image in the corresponding image cluster as the structural similarity of each light and shadow affected block relative to each other rendered image;

[0024] Obtaining a relative distance of multi-dimensional rendering parameters between a rendered image and each other rendered image, and performing a negative correlation mapping as a rendering similarity of the rendered image relative to each other rendered image;

[0025] The structural similarity and rendering similarity corresponding to each other rendered image are arranged according to the same sorting principle, and the mean square error of the arrangement order between each light and shadow affected block and the structural similarity and rendering similarity corresponding to all other rendered images is calculated to obtain the rendering quality stability corresponding to each light and shadow affected block. The mean square error is negatively correlated with the rendering quality stability.

[0026] Furthermore, the method for obtaining the overall rendering quality includes:

[0027] The texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block are integrated and normalized and mapped to obtain the local rendering quality of each light and shadow affected block;

[0028] The average of the local rendering qualities of all light and shadow affected tiles in each rendered image is calculated as the overall rendering quality of each rendering quality.

[0029] Furthermore, the method for obtaining the high-quality rendered image includes:

[0030] If the overall corrected rendering quality of the rendered image is greater than or equal to the preset rendering threshold, the corresponding rendered image is used as a high-quality rendered image.

[0031] Furthermore, the method for obtaining the image clustering clusters includes:

[0032] According to the relative distances of multi-dimensional rendering parameters between different rendered images, K-means clustering is performed on all rendered images to obtain multiple image clusters.

[0033] The present invention also proposes a system for rendering images through a generative digital model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of the steps of the method for rendering images through a generative digital model.

[0034] The present invention has the following beneficial effects:

[0035] The present invention clusters all rendered images according to the relative distances of multi-dimensional rendering parameters between different rendered images to obtain image clustering clusters. Images with similar rendering parameters are classified into one category through clustering, which helps to reduce the complexity of data processing; according to the rendering perspective of each rendered image and the position distribution between the rendered object, the viewing angle distance of each pixel in each rendered image is obtained, reflecting the actual position of the pixel in the rendered image relative to the rendered object; for any rendered image, multiple super-pixel blocks are obtained to analyze the image more accurately; considering that saturation and texture changes in the rendered image are closely related, the texture rendering quality of each super-pixel block is obtained according to the rendering texture parameters of the pixels in different super-pixel blocks and the fluctuation distribution of saturation characteristics, which helps to more intuitively identify super-pixel blocks with clear texture and moderate saturation; according to the neighboring distribution of each super-pixel block The distribution of pixels in the direction of the light source within the domain range, screening out the light and shadow impact blocks, helps to specifically evaluate the light and shadow rendering quality of the corresponding area, so as to more accurately judge the overall quality of the image; combined with the viewing angle distance and brightness characteristics of the pixel points, the light and shadow rendering quality of each light and shadow impact block is obtained, and the quality of light and shadow rendering is more accurately evaluated; according to the distribution of similarities between each light and shadow impact block and the same area of ​​other different rendered images in the corresponding image cluster, and the differences between the corresponding images, the rendering quality stability of each light and shadow impact block is obtained, which helps to more accurately understand which parameters of light and shadow impact blocks can produce stable and high-quality rendering effects; and then the overall rendering quality of each rendered image is obtained, high-quality rendered images are screened out, and the rendered objects are rendered, providing high-quality data support for subsequent model training, thereby improving the rendering effect. The present invention screens out high-quality rendered images, trains to obtain a more accurate generative model, and improves the rendering quality of the generated image. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flowchart of a method for rendering an image through a generative digital model provided by an embodiment of the present invention;

[0038] Figure 2 A flow chart of a method for obtaining texture rendering quality provided by an embodiment of the present invention;

[0039] Figure 3A flow chart of a method for obtaining light and shadow rendering quality provided by an embodiment of the present invention;

[0040] Figure 4 A flow chart of a method for acquiring rendering quality stability provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the method and system for rendering images through a generative digital model proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The following is a detailed description of a method and system for rendering images using a generative digital model provided by the present invention in conjunction with the accompanying drawings.

[0044] See also Figure 1 , which shows a flow chart of a method for rendering an image through a generative digital model provided by an embodiment of the present invention, and the specific method includes:

[0045] Step S1: obtaining multi-dimensional rendering parameters of each rendered image and rendering texture parameters of each pixel of the rendering object, wherein the multi-dimensional rendering parameters include: rendering viewing angle and light source direction.

[0046] In an embodiment of the present invention, a rendered image is obtained through a generative digital model, but due to poor data selection when training the model, there are images with low rendering quality, which affects the quality of the generated image; in order to improve the quality of the model's rendering of the image, it is necessary to analyze the rendering situation of the rendered image and select rendering parameters with high rendering quality to train the model; first, a large number of rendering objects, rendering images, texture properties, lighting information, etc. are collected through the Internet, wherein each rendering object corresponds to a different rendering image under different rendering parameters; the multi-dimensional rendering parameters corresponding to each rendered image of the rendering object and the rendering texture parameters of each pixel are obtained, and the multi-dimensional rendering parameters include: rendering viewing angle and light source direction. It should be noted that in an embodiment of the present invention, the rendering object is three-dimensional and the rendering image is an RGB image.

[0047] It should be noted that, in an embodiment of the present invention, in order to facilitate subsequent data processing and avoid differences in units and numerical magnitudes between data, multi-dimensional rendering parameter data is standardized to eliminate the influence of dimensions in data calculations.

[0048] Step S2: clustering all rendered images according to the relative distances of multi-dimensional rendering parameters between different rendered images to obtain image clustering clusters; and obtaining the viewing angle distance of each pixel in each rendered image according to the rendering viewing angle of each rendered image and the position distribution between the rendered objects.

[0049] The rendered image reflects the two-dimensional representation of the rendered object under specific parameters. For the same rendered object, the images generated under the same rendering parameters have certain similarities. Clustering the rendered images with similar rendering parameters into one category helps to better understand the impact of rendering parameters on image quality. Therefore, according to the relative distance of multi-dimensional rendering parameters between different rendered images, all rendered images are clustered to obtain image clusters.

[0050] Preferably, in one embodiment of the present invention, the method for obtaining image clustering clusters includes:

[0051] According to the relative distances of multi-dimensional rendering parameters between different rendered images, K-means clustering is performed on all rendered images to obtain multiple image clusters.

[0052] It should be noted that, in one embodiment of the present invention, the method for obtaining the relative distance is to calculate the Euclidean distance; in other embodiments of the present invention, the relative distance can also be obtained by the Manhattan distance; the specific Euclidean distance and Manhattan distance are technical means well known to those skilled in the art and will not be described in detail here.

[0053] The K-means clustering algorithm clusters multiple clustering objects into specified K clusters based on the similarity between the clustering objects. Each clustering object belongs to and only belongs to one cluster with the smallest distance to the center of the image clustering cluster. It should be noted that when the K-means clustering algorithm is used to cluster all rendered images, the method for obtaining the K value is: determine the K value by the elbow method to obtain the corresponding number of image clusters. The specific K-means clustering algorithm and the elbow method are technical means well known to those skilled in the art and will not be described in detail here.

[0054] The rendering perspective refers to the position and orientation of the camera in space, which determines the part of the rendered object that will be projected onto the two-dimensional image. Different rendering perspectives will produce different visual effects. By calculating the perspective distance of each pixel, the interaction between light and the surface of the rendered object in reality can be simulated, thereby enhancing the visual realism of the rendered image. Therefore, according to the position distribution between the rendering perspective of each rendered image and the rendered object, the perspective distance of each pixel in each rendered image is obtained.

[0055] Preferably, in one embodiment of the present invention, the method for acquiring the viewing angle distance includes:

[0056] For any rendered image, a plane perpendicular to the rendering view direction is constructed at the rendering view position, and the distance between each pixel point on the rendered object and the plane is obtained as the view distance of each pixel point on the corresponding rendered image.

[0057] The distance between each pixel and the plane is a perpendicular line through each pixel. The distance from this pixel to the foot of the perpendicular is used to quantify the relative position characteristics of each pixel in the rendered object.

[0058] Step S3: for any rendered image, multiple superpixel blocks are obtained; the texture rendering quality of each superpixel block is obtained according to the fluctuation distribution of rendering texture parameters and saturation characteristics of pixel points in different superpixel blocks; the light and shadow affected blocks are screened out according to the distribution of pixel points in the direction of light source light within the neighborhood of each superpixel block; the light and shadow rendering quality of each light and shadow affected block is obtained by combining the viewing angle distance and brightness characteristics of the pixel points; the rendering quality stability of each light and shadow affected block is obtained according to the distribution of similarities between each light and shadow affected block and the same areas of other different rendered images in the corresponding image cluster, as well as the differences between the corresponding images.

[0059] Rendered images often contain a large number of pixels. Directly processing the pixels is not only computationally intensive but also prone to introducing noise. In order to reduce computational complexity, dividing the image into multiple superpixel blocks helps simplify the image analysis and understanding process. For any rendered image, multiple superpixel blocks are obtained.

[0060] It should be noted that, in one embodiment of the present invention, for any rendered image, multiple superpixel blocks are obtained by superpixel segmentation. Superpixel segmentation can aggregate the pixels in the image into several larger pixel blocks, i.e., superpixel blocks, according to similarity criteria such as color, texture or shape, which helps to greatly reduce the dimension of image processing while retaining important information of the image. Specific superpixel segmentation is a technical means well known to those skilled in the art and will not be elaborated here.

[0061] Since different superpixel blocks represent different texture feature areas, the image performance of different objects and material areas is different. Saturation is closely related to texture. High saturation usually means rich colors and may correspond to more detailed textures, while low saturation may indicate that the texture is blurry or monotonous, which can reflect the rendering quality to a certain extent in the image. Therefore, the texture rendering quality of each superpixel block is obtained based on the rendering texture parameters of pixels in different superpixel blocks and the fluctuation distribution of saturation characteristics.

[0062] Preferably, in one embodiment of the present invention, the method for obtaining the texture rendering quality can be found in Figure 2 , which shows a flow chart of a method for obtaining texture rendering quality, including:

[0063] Step S201: Obtain fluctuation characteristics of rendering texture parameters of all pixels of each superpixel block as texture fluctuation characteristics.

[0064] It should be noted that, in one embodiment of the present invention, the fluctuation characteristics are represented by calculating the variance of the rendering texture parameters of all pixels. The larger the variance, the larger the fluctuation characteristics, the more uneven the distribution of the rendering texture parameters, and the more complex the texture representation. In other embodiments of the present invention, the fluctuation characteristics can also be reflected by calculating the standard deviation. The specific means are technical means well known to those skilled in the art and will not be elaborated here.

[0065] Step S202: Obtain the fluctuation characteristics of the component values ​​of the corresponding saturation channel of all pixel points of each superpixel block in the HSV space as the saturation fluctuation characteristics.

[0066] The fluctuation characteristics are expressed by calculating the variance of the component values ​​of the corresponding saturation channel of all pixels in the HSV space. The larger the variance, the greater the fluctuation characteristics, the more uneven the saturation distribution, and the richer the color changes.

[0067] Step S203: Arrange the texture fluctuation characteristics and saturation fluctuation characteristics of all superpixel blocks according to the same sorting principle, and obtain the texture rendering quality of each superpixel block according to the difference in arrangement order between the texture fluctuation characteristics and saturation fluctuation characteristics corresponding to each superpixel block. The difference in arrangement order is negatively correlated with the texture rendering quality.

[0068] It should be noted that, in one embodiment of the present invention, the sorting principle is to arrange from large to small or from small to large. The difference in the arrangement order between the texture fluctuation characteristics and the saturation fluctuation characteristics corresponding to each superpixel block can be obtained by calculating the ratio or difference between the arrangement orders. The larger or smaller the ratio between the arrangement orders, the greater the difference, and the greater the difference between the arrangement orders. The specific means are technical means well known to those skilled in the art and will not be elaborated here.

[0069] Among them, the negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases, that is, the greater the difference in arrangement order, the more inconsistent the changes between rendering saturation and texture parameters, and the lower the texture rendering quality.

[0070] In one embodiment of the present invention, for any rendered image, the formula for the texture rendering quality of the superpixel block is expressed as:

[0071]

[0072] Among them, γ a Indicates the texture rendering quality of the a-th superpixel block; represents the variance of the rendering texture parameters of all pixels in the a-th superpixel block, that is, the texture fluctuation characteristics of the a-th superpixel block; It represents the variance of the component values ​​of the corresponding saturation channel of all pixels in the a-th superpixel block in the HSV space, that is, the saturation fluctuation characteristics of the a-th superpixel block; Indicates the order in which the texture fluctuation features of the ath superpixel block are arranged among all superpixel blocks; It represents the order of the saturation fluctuation characteristics of the ath superpixel block among all superpixel blocks; exp() represents an exponential function with a natural constant as the base.

[0073] In the formula for texture rendering quality, Perform negative correlation mapping, It means calculating the ratio between the order of texture fluctuation feature of the ath superpixel block in all superpixel blocks and the order of saturation fluctuation feature in all superpixel blocks. It represents the difference between the ratio of the calculated positive integer 1 and the arrangement order. The larger the difference, the larger or smaller the ratio, the greater the difference between the arrangement orders, the larger or smaller the ratio, the greater the difference between the arrangement orders, the poorer the consistency of the superpixel blocks in texture and saturation, and the worse the texture rendering quality; the smaller the difference, the closer the ratio is to 1, the closer the arrangement orders are, the better the consistency of the superpixel blocks in texture and saturation, and the better the texture rendering quality.

[0074] In addition to the image surface texture, the light and shadow effect during rendering is also an important factor reflecting the rendering quality. The direction of the light source directly affects the light and shadow effects of objects in the image. By analyzing the distribution of pixels in the direction of the light source, we can more intuitively understand which superpixel blocks are affected by the surrounding areas, and filter out the blocks affected by light and shadow; according to the distribution of pixels in the direction of the light source within the neighborhood of each superpixel block, filter out the blocks affected by light and shadow.

[0075] Preferably, in one embodiment of the present invention, the method for obtaining light and shadow impact blocks includes:

[0076] The neighborhood range of each superpixel block is traversed along the direction of the light source to obtain multiple pixel point sequences; if there are other pixel points in the neighborhood range of the pixel point sequence that are distributed before the pixel points of the superpixel block, the corresponding pixel point sequence is used as the target pixel point sequence, and the corresponding superpixel block is used as the light and shadow affected block.

[0077] It should be noted that, in one embodiment of the present invention, the neighborhood range of a superpixel block is a range consisting of other superpixel blocks adjacent to the edge based on the superpixel block; in other embodiments of the present invention, the size of the neighborhood range can be set according to specific circumstances, and is not limited or elaborated here.

[0078] It should be noted that, in one embodiment of the present invention, the method for obtaining a pixel point sequence is: obtaining edge pixel points in the neighborhood range of each superpixel block, connecting the line segment between two edge pixel points in the direction of the light source at the same position, and forming a pixel point sequence with all the pixel points on the connected line segment along the direction of the light source.

[0079] The viewing angle distance can reflect the size and shape of the object in the image, and affect the presentation of light and shadow. Closer objects may produce more obvious shadows and reflections, while farther objects are affected by factors such as atmospheric scattering, and the light and shadow effects are relatively weaker, and the blocking effect on light is weaker. The brightness feature reflects the brightness of the pixel point. The larger the brightness feature of the pixel point, the stronger the light and shadow effect. When the light direction of the light source passes through the light and shadow affected block, the actual performance may be darker. By analyzing the viewing angle distance and brightness characteristics of the pixel points, the light and shadow rendering quality of each light and shadow affected block can be more comprehensively evaluated. Combined with the viewing angle distance and brightness characteristics of the pixel points, the light and shadow rendering quality of each light and shadow affected block can be obtained.

[0080] Preferably, in one embodiment of the present invention, the method for obtaining the light and shadow rendering quality can be found in Figure 3 , which shows a flow chart of a method for obtaining light and shadow rendering quality, including:

[0081] Step S301: For any light and shadow affected block, obtain the ratio of the average viewing angle distance between other pixels before the light and shadow affected block pixel in the target pixel sequence and the light and shadow affected block pixel, and normalize it as the first light and shadow feature.

[0082] The change in viewing angle distance can reflect the severity of the light and shadow changes in the image. The larger the first light and shadow feature is, the farther away the other pixels before the light and shadow affected block pixel points are from the viewing angle position relative to the light and shadow affected block pixel points, and the smaller the light obstruction to the light and shadow affected block is.

[0083] Step S302: Obtain the component value of the brightness channel corresponding to each pixel point in the HSV space as the brightness feature, obtain the ratio of the brightness feature mean between the light and shadow affected block pixel points in the target pixel point sequence and the corresponding other pixels before, and normalize it as the second light and shadow feature.

[0084] The larger the second light and shadow feature, the greater the brightness feature of the pixel point in the light and shadow affected block compared to the brightness feature of other pixel points, the stronger the lighting effect at the light and shadow affected block compared to other areas in the neighborhood, and the more obvious the brightness change.

[0085] Step S303: Obtain the light and shadow rendering quality of each light and shadow impact block according to the difference between the first light and shadow feature and the second light and shadow feature of different target pixel point sequences in each light and shadow impact block, and the difference between the first light and shadow feature and the second light and shadow feature is negatively correlated with the light and shadow rendering quality.

[0086] Among them, the negative correlation indicates that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases, that is, the greater the difference between the first light and shadow feature and the second light and shadow feature, the more inconsistent the light and shadow changes, and the poorer the light and shadow rendering quality; the smaller the difference, the more consistent the light and shadow changes, and the higher the light and shadow rendering quality.

[0087] In one embodiment of the present invention, for any rendered image, the formula for how light and shadow affect the light and shadow rendering quality of a block is expressed as:

[0088]

[0089] Among them, τ b Indicates that the bth light and shadow affects the light and shadow rendering quality of the block; l b represents the number of target pixel sequences in the bth light and shadow affected block; j represents the sequence number of the target pixel sequence in the bth light and shadow affected block; Indicates the average viewing angle distance of other pixels before the light and shadow affect the block pixel in the target pixel sequence; Indicates the average visual distance between pixel points in the target pixel sequence affected by light and shadow; Indicates the mean value of the brightness characteristics of the pixels in the target pixel sequence affected by light and shadow; It represents the mean value of the brightness characteristics of other pixels in the target pixel sequence before the light and shadow affect the block pixel; c represents the adjustment parameter; norm() represents the normalization function.

[0090] In the formula for light and shadow rendering quality, Represents the ratio of the average viewing angle distance between other pixels and the pixel points in the target pixel sequence before the pixel points in the block affected by light and shadow, and normalizes it, that is, the first light and shadow feature of the jth target pixel sequence; It represents the ratio of the brightness feature mean between the light and shadow affected block pixel points in the target pixel point sequence and the corresponding other pixels before, and normalizes it, that is, the second light and shadow feature of the target pixel point sequence; Calculate the difference between the first light and shadow feature and the second light and shadow feature of the j-th target pixel sequence. That is, the larger the difference, the poorer the consistency of the image rendering and the poorer the quality of the light and shadow rendering. The smaller the difference, the more consistent the light and shadow changes, the more coordinated the lighting effect within the neighborhood, and the higher the quality of the light and shadow rendering.

[0091] It should be noted that, in one embodiment of the present invention, the size of the adjustment parameter is 0.01 to avoid the denominator of the formula being 0; in other embodiments of the present invention, the size of the adjustment parameter can be set according to specific circumstances, and is not limited or elaborated here.

[0092] Since images in the same image cluster are classified into one category due to their high similarity, the similarity between images is relatively high. By specifically analyzing the similarity between each superpixel block and other rendered images, we can capture the slight changes in the local area of ​​the image and evaluate the stability of the superpixel block under different rendering conditions. Based on the distribution of the similarity between each superpixel block and the same area of ​​other different rendered images in the corresponding image cluster, as well as the differences between the corresponding images, we can obtain the rendering quality stability of each light and shadow affected block in each rendered image.

[0093] Preferably, in one embodiment of the present invention, the method for obtaining the rendering quality stability is described in Figure 4 , which shows a flow chart of a method for obtaining rendering quality stability, including:

[0094] Step S401: using the SSIM function to obtain the structural similarity between each light and shadow affected block and the same region of each other rendered image in the corresponding image cluster as the structural similarity of each light and shadow affected block relative to each other rendered image.

[0095] The SSIM function is a structural similarity evaluation function that can quantify the structural similarity between two image blocks, including brightness, contrast, and structural information. By comparing the structural similarity between the same regions, the structural consistency of different images at the same regional position can be evaluated, thereby reflecting the rendering quality. The specific SSIM function is a technical means well known to those skilled in the art and will not be described in detail here.

[0096] Step S402: obtaining a relative distance of multi-dimensional rendering parameters between the rendered image and each other rendered image, and performing negative correlation mapping as rendering similarity of the rendered image relative to each other rendered image.

[0097] By calculating the relative distance of multi-dimensional rendering parameters between the rendered image and each other rendered image, analyzing the differences between the parameters can reveal the changes in the rendering process. The negative correlation mapping enables the differences in rendering parameters to be directly reflected in the rendering similarity. The larger the relative distance, the greater the difference in rendering parameters and the smaller the rendering similarity.

[0098] It should be noted that, in one embodiment of the present invention, the inverse of the relative distance is calculated to implement negative correlation mapping, and a correlation relationship is constructed in which the larger the relative distance, the smaller the rendering similarity; wherein, in order to avoid the denominator being 0 and the formula being meaningless, an artificially set parameter, such as 0.01, is added; in other embodiments of the present invention, exp(-) may also be used for negative correlation mapping, and the specific means are technical means well known to those skilled in the art and will not be limited or elaborated herein.

[0099] Step S403: Arrange the structural similarity and rendering similarity corresponding to each other rendered image according to the same sorting principle, calculate the mean square error of the arrangement order between each light and shadow affected block and the structural similarity and rendering similarity corresponding to all other rendered images, and obtain the rendering quality stability corresponding to each light and shadow affected block. The mean square error is negatively correlated with the rendering quality stability.

[0100] Among them, the negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. That is, for images with higher rendering quality in the same cluster, the structural similarity and rendering similarity between images change similarly and are positively correlated. Therefore, the larger the mean square error, the greater the difference between the structural similarity and rendering similarity of the light and shadow affected blocks relative to other rendered images, resulting in greater differences in rendering quality in the same cluster and worse rendering quality stability.

[0101] Step S4: according to the texture rendering quality, light and shadow rendering quality and rendering quality stability of each light and shadow affected block in each rendered image, the overall rendering quality of each rendered image is obtained; according to the overall rendering quality of all rendered images, high-quality rendered images are screened out to render the rendered object.

[0102] Texture rendering quality reflects the visual realism and detail richness of an image, and can reflect the material properties of an object. Light and shadow rendering quality can reflect important factors of an image's stereoscopic and depth perception. Rendering quality stability can reflect the consistency of image rendering quality in different areas and under different conditions. Comprehensive analysis helps to comprehensively evaluate the overall rendering quality of each rendered image. The overall rendering quality of each rendered image can be obtained based on the texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block in each rendered image.

[0103] Preferably, in one embodiment of the present invention, the method for obtaining the overall rendering quality includes:

[0104] The texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block are integrated and normalized and mapped to obtain the local rendering quality of each light and shadow affected block;

[0105] The average of the local rendering qualities of all light and shadow affected tiles in each rendered image is calculated as the overall rendering quality of each rendering quality.

[0106] It should be noted that, in some embodiments of the present invention, the texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block may be integrated by addition or multiplication.

[0107] In one embodiment of the present invention, the formula for overall rendering quality is expressed as:

[0108]

[0109] Among them, A i represents the overall rendering quality of the i-th rendered image; B i represents the number of light and shadow affected blocks of the i-th rendered image; τ b Indicates that the bth light and shadow affects the light and shadow rendering quality of the block; γ b Indicates that the bth light and shadow affects the texture rendering quality of the block; w b Indicates that the bth light and shadow affects the rendering quality stability of the block; tanh() represents the hyperbolic tangent function.

[0110] In the formula for overall rendering quality, the tanh function converts τ b ×γ b × b Normalization, the value is between 0 and 1; τ b ×γ b × b It means calculating the product of texture rendering quality, light and shadow rendering quality and rendering quality stability of the b-th light and shadow affected block, that is, the local rendering quality of the b-th light and shadow affected block; It means to average the local rendering quality of all light and shadow affected blocks in the rendered image, that is, the overall rendering quality of the rendered image. The higher the texture rendering quality, the higher the light and shadow rendering quality, the greater the rendering quality stability, and the higher the overall rendering quality.

[0111] The overall rendering quality can reflect the rendering technology implementation capability of the rendered image. The greater the overall rendering quality, the closer the rendered image is to the actual visual effect and retains more detail features. By screening high-quality rendered images, the noise and redundant information in the training data set can be reduced, thereby speeding up the training of the model. Therefore, high-quality rendered images are screened out based on the overall rendering quality of all rendered images.

[0112] Preferably, in one embodiment of the present invention, the method for obtaining a high-quality rendered image includes:

[0113] If the overall corrected rendering quality of the rendered image is greater than or equal to the preset rendering threshold, the corresponding rendered image is used as a high-quality rendered image.

[0114] It should be noted that, in one embodiment of the present invention, the size of the preset rendering threshold is 0.6; in other embodiments of the present invention, the size of the preset rendering threshold can be specifically set according to specific circumstances, which is not limited or elaborated herein.

[0115] Based on this, obtaining high-quality rendered images can provide richer details and more realistic lighting and shadow effects. In another embodiment of the present invention, after obtaining high-quality rendered images, the generative digital model can be trained to learn more realistic and high-quality rendering features and rules, thereby generating more realistic and delicate images in the rendering of the rendered object.

[0116] It should be noted that, in an embodiment of the present invention, before training the generative digital model based on high-quality rendered images, a generative adversarial network GAN model is first constructed: based on a DNN network, the generator uses a Unet_256 network, the discriminator uses a 70×70 patch, the number of generator filters and the number of discriminator filters are both 64, the batch size is 1, the epoch is 400, and the loss function is defined as a cross entropy function. Through continuous adversarial training, the generator maintains details when generating high-quality images, and can generate more and more realistic images to deceive the discriminator. The discriminator can improve its sensitivity to local details of the image and more and more accurately identify the authenticity of the image, which helps the trained generative model to generate high-quality rendered images. The specific model parameters can be set according to the specific situation, and are not limited or elaborated here.

[0117] In summary, the present invention traverses the rendering features of the pixels in the superpixel blocks of any rendered image based on the rendering perspective and the direction of the light source, and obtains the texture rendering quality of each superpixel block, and screens out the light and shadow affected blocks to obtain the light and shadow rendering quality of each light and shadow affected block; according to the distribution of similarities between each light and shadow affected block and the same area of ​​other different rendered images in the corresponding image cluster, and the differences between the corresponding images, the rendering quality stability of each light and shadow affected block is obtained; and then the overall rendering quality of each rendered image is obtained, and high-quality rendered images are screened out to render the rendered object. The present invention screens out high-quality rendered images, trains to obtain a more accurate generative model, and improves the rendering quality of the generated image.

[0118] The present invention also proposes a system for rendering images through a generative digital model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of the steps of a method for rendering images through a generative digital model.

[0119] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0120] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for rendering an image using a generative digital model, characterized in that: The method comprises: Obtaining multi-dimensional rendering parameters corresponding to each rendered image of the rendering object and rendering texture parameters of each pixel point, wherein the multi-dimensional rendering parameters include: rendering viewing angle and light source direction; According to the relative distances of multi-dimensional rendering parameters between different rendered images, all rendered images are clustered to obtain image clusters; according to the rendering perspective of each rendered image and the position distribution between the rendered objects, the perspective distance of each pixel in each rendered image is obtained; For any rendered image, multiple superpixel blocks are obtained; the texture rendering quality of each superpixel block is obtained according to the fluctuation distribution of the rendering texture parameters and saturation characteristics of the pixels in different superpixel blocks; the light and shadow affected blocks are screened out according to the distribution of pixels in the direction of the light source within the neighborhood of each superpixel block; the light and shadow rendering quality of each light and shadow affected block is obtained by combining the viewing angle distance and brightness characteristics of the pixels; the rendering quality stability of each light and shadow affected block is obtained according to the distribution of similarities between each light and shadow affected block and the same areas of other different rendered images in the corresponding image cluster, as well as the differences between the corresponding images; According to the texture rendering quality, light and shadow rendering quality and rendering quality stability of each light and shadow affected block in each rendered image, the overall rendering quality of each rendered image is obtained; according to the overall rendering quality of all rendered images, high-quality rendered images are screened out and the rendered objects are rendered; The method for obtaining the high-quality rendered image comprises: If the overall corrected rendering quality of the rendered image is greater than or equal to the preset rendering threshold, the corresponding rendered image is used as a high-quality rendered image.

2. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the viewing angle distance includes: For any rendered image, a plane perpendicular to the rendering view direction is constructed at the rendering view position, and the distance between each pixel point on the rendered object and the plane is obtained as the view distance of each pixel point on the corresponding rendered image.

3. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the texture rendering quality comprises: Obtaining fluctuation characteristics of rendering texture parameters of all pixels in each superpixel block as texture fluctuation characteristics; Obtain the fluctuation characteristics of the component values ​​of the corresponding saturation channel of all pixel points in each superpixel block in the HSV space as the saturation fluctuation characteristics; The texture fluctuation features and saturation fluctuation features of all superpixel blocks are arranged according to the same sorting principle. The texture rendering quality of each superpixel block is obtained according to the difference in arrangement order between the texture fluctuation features and saturation fluctuation features corresponding to each superpixel block. The difference in arrangement order is negatively correlated with the texture rendering quality.

4. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the light and shadow impact blocks includes: The neighborhood range of each superpixel block is traversed along the direction of the light source to obtain multiple pixel point sequences; if there are other pixel points in the neighborhood range of the pixel point sequence that are distributed before the pixel points of the superpixel block, the corresponding pixel point sequence is used as the target pixel point sequence, and the corresponding superpixel block is used as the light and shadow affected block.

5. The method for rendering an image by using a generative digital model according to claim 4, characterized in that: The method for obtaining the light and shadow rendering quality includes: For any light and shadow affected block, obtain the ratio of the average value of the viewing angle distance between other pixels before the light and shadow affected block pixel in the target pixel sequence and the light and shadow affected block pixel, and normalize it as the first light and shadow feature; The component value of the brightness channel corresponding to each pixel in the HSV space is obtained as the brightness feature, and the ratio of the brightness feature mean between the light and shadow affected block pixel and the corresponding other pixels in the target pixel sequence is obtained and normalized as the second light and shadow feature; According to the difference between the first light and shadow features and the second light and shadow features of different target pixel point sequences in each light and shadow impact block, the light and shadow rendering quality of each light and shadow impact block is obtained, and the difference between the first light and shadow features and the second light and shadow features is negatively correlated with the light and shadow rendering quality.

6. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the rendering quality stability includes: The SSIM function is used to obtain the structural similarity between each light and shadow affected block and the same region of each other rendered image in the corresponding image cluster as the structural similarity of each light and shadow affected block relative to each other rendered image; Obtaining a relative distance of multi-dimensional rendering parameters between a rendered image and each other rendered image, and performing a negative correlation mapping as a rendering similarity of the rendered image relative to each other rendered image; The structural similarity and rendering similarity corresponding to each other rendered image are arranged according to the same sorting principle, and the mean square error of the arrangement order between each light and shadow affected block and the structural similarity and rendering similarity corresponding to all other rendered images is calculated to obtain the rendering quality stability corresponding to each light and shadow affected block. The mean square error is negatively correlated with the rendering quality stability.

7. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the overall rendering quality includes: The texture rendering quality, light and shadow rendering quality, and rendering quality stability of each light and shadow affected block are integrated and normalized and mapped to obtain the local rendering quality of each light and shadow affected block; The average of the local rendering qualities of all light and shadow affected tiles in each rendered image is calculated as the overall rendering quality of each rendering quality.

8. The method for rendering an image by using a generative digital model according to claim 1, characterized in that: The method for obtaining the image clustering clusters comprises: According to the relative distances of multi-dimensional rendering parameters between different rendered images, K-means clustering is performed on all rendered images to obtain multiple image clusters.

9. A system for rendering images using a generative digital model, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a method for rendering an image through a generative digital model as described in any one of claims 1 to 8 are implemented.

Images