A sketch-style scene rendering method, device and storage medium

Abstract

This invention discloses a sketch-style scene rendering method, device, and storage medium. The sketch-style scene rendering method includes: acquiring the current scene rendering result; using the current scene rendering result, calculating the scene irradiance, sampling different brushstroke textures corresponding to the scene irradiance, and mixing the different brushstroke textures to obtain the base color of the sketch rendering; using the current scene rendering result, extracting silhouette contour lines based on edge detection convolution kernels, reconstructing the contour lines, sampling distortion maps, and calculating contour line distortion masks; based on the silhouette contour lines and geometric contour lines, using the current scene rendering result, reconstructing the direction of the contour lines in world coordinates, sampling frequency maps, and calculating line segment discontinuity masks; mixing the contour line distortion masks and the line segment discontinuity masks to obtain a final mask, and using the final mask and the base color of the sketch rendering to calculate the final sketch rendering color.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, in particular to a sketch style scene rendering method based on ray tracing, a device and a storage medium. BACKGROUND

[0002] The traditional sketch style scene rendering method actually only imitates the line feeling of sketch hand-drawing, does not correctly depict the elements of sketch three faces (light face, gray face, dark face) and five tones (high light, high gray part, light and dark junction line, reflection, projection), and does not show the human effects produced by hand-drawing, such as the intermittent effect and twisted line draft effect produced by pencil on sketch paper.

[0003] Therefore, how to design a sketch style scene rendering method capable of simulating sketch human effects is still a technical problem to be solved in the field. SUMMARY

[0004] Therefore, the present application provides a sketch style scene rendering method, a device and a storage medium, which renders the elements of sketch three faces and five tones based on ray tracing, and simulates the effects produced by human hand-drawing.

[0005] The first aspect of the present application provides a sketch style scene rendering method, which comprises: obtaining a current scene rendering result; using the current scene rendering result to calculate scene irradiance, sample different pen touch textures corresponding to the scene irradiance, and mix the different pen touch textures to obtain a sketch rendering base color; using the current scene rendering result to extract a silhouette contour line based on an edge detection convolution kernel, reconstruct the contour line, sample a distortion map, and calculate a contour line distortion mask; using the current scene rendering result to reconstruct the direction of the contour line in the world coordinate based on the silhouette contour line and a geometric contour line, sample a frequency map, and calculate a line segment intermittent mask; mixing the contour line distortion mask and the line segment intermittent mask to obtain a final mask, and using the final mask and the sketch rendering base color to calculate a final sketch rendering color.

[0006] Further, the current scene rendering result comprises a scene world coordinate buffer, a world normal buffer, a scene depth buffer, a scene color buffer, a scene diffuse reflection color buffer, and a screen space coordinate.

[0007] Further, the scene irradiance is calculated using the scene diffuse reflection color buffer and the scene color buffer.

[0008] Further, the step of sampling different stroke textures corresponding to the scene radiance and mixing the different stroke textures to obtain the base color of the sketch rendering comprises: dividing the scene radiance into bright parts, gray parts and dark parts; sampling stroke textures of the bright parts, the gray parts and the dark parts by using the world coordinate buffer and the world normal buffer; sampling curves of the dark parts, the gray parts and the bright parts by using the scene radiance to obtain stroke weights; performing linear difference on the stroke textures of the bright parts, the gray parts and the dark parts by using the stroke weights to obtain stroke colors of the bright parts, the gray parts and the dark parts; and mixing the stroke colors of the bright parts, the gray parts and the dark parts to obtain the base color of the sketch rendering.

[0009] Further, the step of extracting a silhouette contour line based on an edge detection convolution kernel, reconstructing the contour line, sampling a distortion map and calculating a contour line distortion mask by using the current scene rendering result comprises: extracting the silhouette contour line based on the edge detection convolution kernel; calculating a rotation radian of the contour line in a screen space by using a directional convolution kernel; calculating a direction of the contour line in a world coordinate system by using the rotation radian and the world normal buffer; reconstructing the contour line by using the direction of the contour line in the world coordinate system and the world coordinate buffer; sampling the distortion map by using the reconstructed contour line to obtain a distorted contour line, and multiplying the distorted contour line by the silhouette contour line to obtain the contour line distortion mask.

[0010] Further, the step of calculating the direction of the contour line in the world coordinate system by using the rotation radian and the world normal buffer comprises: constructing a two-dimensional rotation matrix by using the rotation radian; transforming each element in an original convolution kernel to obtain a new directional convolution kernel by using the two-dimensional rotation matrix; sampling normals of three left pixels and three right pixels from the world normal buffer by using the new directional convolution kernel, and taking averages of the normals to obtain left normals and right normals; and calculating the direction of the contour line in the world coordinate system by using cross products.

[0011] Further, the step of reconstructing a direction of a contour line in a world coordinate system by using a current scene rendering result based on a silhouette contour line and a geometric contour line, and sampling a frequency map to calculate a line segment discontinuity mask comprises: extracting the geometric contour line by using the world normal buffer in combination with a mean blur convolution kernel; calculating the direction of the contour line in the world coordinate system based on the world normal buffer and an inverse convolution kernel, and sampling a frequency map by using a component of a world coordinate buffer projected to the direction of the contour line in the world coordinate system to obtain a frequency color; and multiplying a color channel component of the frequency color by a value obtained by mixing the silhouette contour line and the geometric contour line to obtain the line segment discontinuity mask.

[0012] Further, the final sketch rendering color is calculated by multiplying the contour line distortion mask and the line segment intermittent mask to obtain a final mask, and then using the final mask and the sketch rendering base color to calculate a final sketch rendering color by using a linear interpolation formula.

[0013] The second aspect of the present application provides a sketch style scene rendering device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the sketch style scene rendering method as described above.

[0014] The third aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the sketch style scene rendering method as described above.

[0015] The sketch style scene rendering method calculates scene irradiance using a ray tracing rendering result, classifies the scene irradiance using a curve, and renders three major surface base colors in a sketch; and extracts a sketch contour line to draw an intermittent and distorted effect for the contour line. BRIEF DESCRIPTION OF DRAWINGS

[0016] For the purpose of illustration and not limitation, the present application will now be described in accordance with the preferred embodiments thereof, particularly with reference to the drawings wherein:

[0017] Figure 1 is a flow chart of the sketch style scene rendering method provided by an embodiment of the present application;

[0018] Figure 2 is a rendering result schematic diagram of the sketch style scene rendering method provided by an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of the sketch style scene rendering device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0021] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application. The described embodiments are merely some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0023] Figure 1 is a flow chart of a sketch style scene rendering method provided by an embodiment of the application. The sketch style scene rendering method calculates scene irradiance using ray tracing rendering results, classifies scene irradiance using a curve, and renders three major surface base colors in a sketch; extracts a sketch contour line, and draws a sketch contour line with a discontinuous and twisted effect.

[0024] Please refer to the accompanying drawings Figure 1 The sketch style scene rendering method comprises the following steps:

[0025] S100, obtaining a sketch style current scene rendering result, including a scene world coordinate buffer P, a world normal buffer N, a scene depth buffer D, a scene color buffer C, a scene diffuse reflection color buffer B, and a screen space coordinate Ps.

[0026] S200, calculating scene irradiance L using the scene diffuse reflection color buffer B and the scene color buffer C.

[0027] In this embodiment, the calculation method of the scene irradiance L is:

[0028] According to a rendering equation:

[0029]

[0030] F = c + (1.0 - c) * pow(1.0 - cosθ, 5.0)

[0031] C ≈ ∫L dp

[0032] In the above rendering equation, c is equivalent to the scene diffuse reflection color buffer B, therefore, based on the brightness formula Lf, the following is approximately extracted:

[0033] Lf(x) = x.r*0.3 + x.g*0.51 + x.b*0.11

[0034]

[0035] r, g, and b are red, green, and blue channels corresponding to the color, and correspond to the wavelength of the light in the irradiance calculation.

[0036] S300, using the world coordinate buffer P and the world normal buffer N, sampling different pen touch textures corresponding to the scene irradiance, and mixing different pen touch textures to obtain a sketch rendering base color C s.

[0037] The embodiment is based on scene irradiance L, and uses a curve to classify the scene irradiance L into three levels of bright part, gray part and dark part, and uses the world coordinate buffer P and the world normal buffer N to sample the stroke texture T1, T2 and T3 corresponding to the bright part, the gray part and the dark part. In the sketch, the stroke of the dark part is a plurality of lines, and presents a dense and black effect. The stroke of the bright part is a single line or a double cross line, and presents a sparse and light color effect. The stroke textures T1, T2 and T3 of the three parts are mixed to obtain the base color C of the sketch rendering. s .

[0038] In the embodiment, the world coordinate buffer P and the world normal buffer N are used to sample different stroke textures corresponding to different scene irradiance, and the base color C of the sketch rendering is obtained by mixing the different stroke textures. s The specific implementation manner is as follows:

[0039] S301, based on the scene irradiance L calculated in step S200, the scene irradiance L is divided into a bright part, a gray part and a dark part.

[0040] Based on the scene irradiance L calculated in step S200, the scene irradiance ranges from 0.0 to 1.0, 0 is the darkest area in the picture, and 1 is the brightest part in the picture. It is tested that the value range of the dark part is (0-0.05), the value range of the gray part is (0.03-0.15), and the value range of the bright part is (0.12-1.0). The overlapping part of the data is the transition area of the three parts. The value range of the bright-dark boundary line in the sketch is (0.2-0.3).

[0041] S302, a method for sampling stroke texture, using the world coordinate buffer P and the world normal buffer N to sample the stroke texture of the bright part, the gray part and the dark part.

[0042] P.xy, P.yz and P.xz are used to sample the stroke texture to obtain C xy , C yz and C xz , respectively, and then the x component and the z component of the normal vector are used to linearly difference the stroke textures C xy , C yz and C xz to obtain the stroke color C t .

[0043] S303, using the scene irradiance L to sample the curves of the dark part, the gray part and the bright part respectively to obtain the stroke weight W, and using the stroke weight W and the stroke textures of the bright part, the gray part and the dark part to calculate the stroke colors of the bright part, the gray part and the dark part.

[0044] The scene irradiance L (range 0-1) is sampled to obtain the stroke weight W (range 0-1) of the dark part, the gray part and the bright part; the stroke texture C xy , C yz , C xz of the bright part, the gray part and the dark part obtained in step S302 is linearly differentiated to obtain the stroke color of the bright part, the gray part and the dark part, respectively C td , C tg , C tl .

[0045] S304, the stroke color of the bright part, the gray part and the dark part is mixed by color multiplication to obtain the base color C s of sketch rendering.

[0046] In this embodiment, the stroke color obtained by sampling the dark part, the gray part and the bright part is C td , C tg , C tl , respectively, and the three groups of stroke colors C td , C tg , C tl are mixed by color multiplication to obtain the base color C s of sketch rendering, which shows that the intensity of the stroke decreases and the color gradually changes from black to the color of the paper from the dark part to the bright part.

[0047] This embodiment calculates the scene irradiance by using the ray tracing rendering result, classifies the scene irradiance by using the curve, and renders the base color of the three major faces in the sketch.

[0048] S400, the silhouette contour line S is extracted by using the world normal buffer N and the screen space coordinate Ps based on the edge detection convolution kernel E k , the contour line UV is reconstructed, the distortion map T s is sampled, and the contour line distortion mask M s is calculated.

[0049] In this embodiment, the specific implementation mode of step S400 is as follows:

[0050] S401, the silhouette contour line S is extracted based on the edge detection convolution kernel E k .

[0051] In this embodiment, the edge detection convolution kernel E k is:

[0052] E k = [-1 -3 -5 -7 0 7 5 3 1]

[0053] S402, the directionality convolution kernel S x , Sy The rotation radian r of the silhouette profile line in the screen space is calculated.

[0054] In this embodiment, the directional convolution kernel S x , S y is:

[0055]

[0056]

[0057] The x-axis variation gx and y-axis variation gy of the profile line are calculated by using the directional convolution kernel, and the slope is:

[0058] Therefore, the rotation radian r of the profile line in the screen space is:

[0059]

[0060] S403, the direction R of the profile line in the world coordinate system is calculated by using the rotation radian r and the world normal buffer N.

[0061] In this embodiment, the specific implementation of calculating the direction R of the profile line in the world coordinate system by using the rotation radian r and the world normal buffer N is as follows:

[0062] S403-1, a new left and right directional convolution kernel K L and K R is constructed by using the rotation radian r and the original convolution kernel.

[0063] The original left convolution kernel O KL is:

[0064]

[0065] The original left convolution kernel O KR is:

[0066]

[0067] A two-dimensional rotation matrix R is constructed by using the rotation radian r:

[0068]

[0069] The new directional convolution kernel K L and K R is obtained by transforming each element in the original convolution kernel by using the two-dimensional rotation matrix R, and the new directional convolution kernel K L and K R is:

[0070]

[0071]

[0072] S403-2, utilizing a new directional convolution kernel K L and K R Sample the normals of the three left and three right pixels from the world normal buffer N, and take the average value to obtain the left normal N. L and the right normal N R Then, the direction R of the contour line in the world coordinate system is obtained by using the cross product.

[0073] In this embodiment, the direction R of the contour line in the world coordinate system is:

[0074] R = cross(N) l N r )

[0075] S404 reconstructs the silhouette UV using the direction R of the silhouette outline in the world coordinate system and the world coordinate buffer P.

[0076] S405, using the reconstructed contour UV sampling distortion map T s The distorted outline is obtained, and multiplied by the silhouette outline S to obtain the outline distortion mask M. s .

[0077] S500, based on the silhouette outline S and geometric outline G, reconstructs the orientation E of the outline in world coordinates using the world normal buffer N and screen space coordinates Ps. d Using the direction E of the contour line in world coordinates d and world coordinate buffer P sampling frequency map T f Calculate the discontinuous masking Mp of the line segment.

[0078] In this embodiment, step S500 is implemented as follows:

[0079] S501 utilizes a world normal buffer N combined with a mean-fuzzy convolution kernel E. m The edge extraction algorithm extracts the geometric contour line G.

[0080] In this embodiment, the mean fuzzy convolution kernel E m for:

[0081]

[0082] S502 calculates the direction R of the contour line in world coordinates based on the world normal buffer N and the inverse convolution kernel. Using the direction R and the world coordinate buffer P, the world coordinate buffer P is projected onto the component sampling frequency map T of the direction R. f The frequency color C is obtained.f .

[0083] S503, utilizing frequency color C f The r-channel component (frequency map is a grayscale image, RGB values ​​are equal) is used as a discontinuous mask, and multiplied by the blending values ​​of the silhouette outline S and geometric outline G to obtain the line segment discontinuous mask M. p .

[0084] S600, Hybrid Contour Distortion Mask M s and line segment discontinuity mask M p Obtain the final mask M, and then use the final mask M and the base color C of the sketch rendering. s The final sketch rendering color C is obtained by using a linear interpolation formula.

[0085] In this embodiment, the final sketch rendering color C is calculated as follows:

[0086] C = C s *(M s *M p )

[0087] This embodiment extracts the outline of the sketch and draws the outline with a broken and twisted effect.

[0088] Compared to traditional solutions that use only the simplest Lambert lighting calculations to represent light and shadow, this embodiment, based on ray tracing rendering, can accurately represent the brightness and darkness of reflections and projections.

[0089] The aforementioned sketch-style scene rendering method uses the scene rendering results in screen space to calculate silhouette and geometric outlines, calculates the outline direction, reconstructs the outline UVs, and draws an effect resembling a hand-drawn line drawing, such as... Figure 2 As shown.

[0090] The sketch outlines obtained in this embodiment are all continuous in the temporal domain, and the rendering effect will not change with the movement of the viewpoint.

[0091] The aforementioned sketch-style scene rendering method, based on ray tracing rendering, can accurately represent the three planes and five tones. Utilizing the scene rendering results in screen space, it calculates and analyzes the sketch outline lines, calculates the direction of the lines in three-dimensional space, reconstructs the line UVs, and samples stylus maps to simulate the human effect of sketching.

[0092] For the method embodiments described above, see [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of a sketch-style scene rendering device provided in another embodiment of the present invention. The device 100 may include:

[0093] Memory 101 is used to store computer programs;

[0094] The processor 102, when executing the computer program stored in the memory 101, can implement the following steps:

[0095] obtaining a current scene rendering result; using the current scene rendering result, calculating scene irradiance, sampling different pen stroke textures corresponding to the scene irradiance, and mixing the different pen stroke textures to obtain a base color of sketch rendering; using the current scene rendering result, extracting a silhouette contour line based on an edge detection kernel, reconstructing the contour line, sampling a distortion map, and calculating a contour line distortion mask; using the current scene rendering result, reconstructing a direction of the contour line in a world coordinate based on the silhouette contour line and a geometric contour line, sampling a frequency map, and calculating a line segment intermittent mask; mixing the contour line distortion mask and the line segment intermittent mask to obtain a final mask, and using the final mask and the base color of sketch rendering to calculate a final sketch rendering color.

[0096] For the device provided by the application, refer to the above method embodiments, and the application will not be repeated here.

[0097] Corresponding to the above method embodiments, the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps can be implemented:

[0098] obtaining a current scene rendering result; using the current scene rendering result, calculating scene irradiance, sampling different pen stroke textures corresponding to the scene irradiance, and mixing the different pen stroke textures to obtain a base color of sketch rendering; using the current scene rendering result, extracting a silhouette contour line based on an edge detection kernel, reconstructing the contour line, sampling a distortion map, and calculating a contour line distortion mask; using the current scene rendering result, reconstructing a direction of the contour line in a world coordinate based on the silhouette contour line and a geometric contour line, sampling a frequency map, and calculating a line segment intermittent mask; mixing the contour line distortion mask and the line segment intermittent mask to obtain a final mask, and using the final mask and the base color of sketch rendering to calculate a final sketch rendering color.

[0099] The computer readable storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0100] For the computer readable storage medium provided by the application, refer to the above method embodiments, and the application will not be repeated here.

[0101] The various embodiments described in this specification are implemented in the context of one or more of the following embodiments. Each of the embodiments is described in terms of its differences from the other embodiments. Each of the embodiments is described in the context of the other embodiments by cross-referencing. The devices, apparatuses, and computer-readable storage media disclosed by the embodiments are described in less detail because they correspond to the methods disclosed by the embodiments. The devices, apparatuses, and computer-readable storage media are described by cross-referencing the method descriptions.

[0102] The specific embodiments described hereinabove are illustrative of various embodiments of the present application. Needless to say, modifications, combinations, sub-combinations, and alternatives can occur to one of ordinary skill in the art upon reading this disclosure, which are intended to be within the scope of the application. Accordingly, the present application is not defined by the description contained herein, but by the claims for protection, and its full scope is to be determined by the applicable law.

Claims

1. A method of sketch-style scene rendering, characterized by, The method comprises the following steps: obtaining a current scene rendering result; calculating scene irradiance using the current scene rendering result, sampling different stroke textures corresponding to the scene irradiance, mixing the different stroke textures to obtain a base color of sketch rendering; extracting a silhouette contour line based on an edge detection kernel using the current scene rendering result, reconstructing the contour line, sampling a distortion map, and calculating a contour line distortion mask; based on the silhouette contour line and a geometric contour line, reconstructing a direction of the contour line in a world coordinate system using the current scene rendering result, sampling a frequency map, and calculating a line segment discontinuous mask; mixing the contour line distortion mask and the line segment discontinuous mask to obtain a final mask, and calculating a final sketch rendering color using the final mask and the base color of sketch rendering; the current scene rendering result comprises a scene world coordinate buffer, a world normal buffer, a scene depth buffer, a scene color buffer, a scene diffuse color buffer, and a screen space coordinate; the scene irradiance is calculated using the scene diffuse color buffer and the scene color buffer; the step of sampling different stroke textures corresponding to the scene irradiance, mixing the different stroke textures to obtain a base color of sketch rendering comprises: dividing the scene irradiance into bright, gray, and dark parts; sampling stroke textures of the bright, gray, and dark parts using the world coordinate buffer and the world normal buffer; sampling curves of the dark, gray, and bright parts using the scene irradiance to obtain stroke weights; performing linear interpolation on the stroke textures of the bright, gray, and dark parts using the stroke weights to obtain stroke colors of the bright, gray, and dark parts; mixing the stroke colors of the bright, gray, and dark parts to obtain a base color of sketch rendering.

2. The sketch-style scene rendering method of claim 1, wherein, the step of extracting a silhouette contour line based on an edge detection kernel using the current scene rendering result, reconstructing the contour line, sampling a distortion map, and calculating a contour line distortion mask comprises: extracting a silhouette contour line based on an edge detection kernel; calculating a rotation radian of the contour line in a screen space using a directional kernel; calculating a direction of the contour line in a world coordinate system using the rotation radian and the world normal buffer; reconstructing the contour line using the direction of the contour line in the world coordinate system and the world coordinate buffer; sampling a distortion map using the reconstructed contour line to obtain a distorted contour line, and multiplying the distorted contour line by the silhouette contour line to obtain a contour line distortion mask.

3. The sketch-style scene rendering method of claim 2, wherein, the step of calculating a direction of the contour line in a world coordinate system using the rotation radian and the world normal buffer comprises: constructing a two-dimensional rotation matrix using the rotation radian; transforming each element in an original kernel to obtain a new directional kernel using the two-dimensional rotation matrix; sampling normals of left and right three pixels from the world normal buffer using the new directional kernel, taking the mean value to obtain left and right normals, and calculating the direction of the contour line in the world coordinate system using a cross product.

4. The sketch-style scene rendering method of claim 1, wherein, The step of reconstructing the direction of the contour line in the world coordinate and sampling the frequency map by using the current scene rendering result based on the silhouette contour line and the geometric contour line comprises: extracting the geometric contour line by using the world normal buffer combined with the mean blur kernel; calculating the direction of the contour line in the world coordinate based on the world normal buffer and the inverse kernel, and projecting the component of the world coordinate buffer to the direction of the contour line in the world coordinate to sample the frequency map to obtain the frequency color; multiplying the value of mixing the color channel component in the frequency color with the silhouette contour line and the geometric contour line with the line segment discontinuous mask.

5. The sketch-style scene rendering method of claim 1, wherein, The calculation method of the final sketch rendering color is: multiplying the contour line distortion mask and the line segment discontinuous mask to obtain the final mask; calculating the final sketch rendering color by using the final mask and the base color of the sketch rendering with the linear interpolation formula.

6. A sketch-style scene rendering device, characterized by, comprise: a memory for storing a computer program; a processor for executing the computer program to realize the steps of the scene rendering method in the sketch style according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the scene rendering method in the sketch style according to any one of claims 1 to 5.

Images