Sketch-style scene rendering method, device and storage medium

Abstract

The embodiment of the present application provides a sketch-style scene rendering method, device and storage medium. In the method, for a three-dimensional scene to be processed, the position of the stroke line in each scene element model is determined; according to the position of the stroke line, each scene element model is stroked and colored to obtain a three-dimensional scene containing the stroke line; according to the normal direction of each scene element model, the sketch stroke map is tiled to the three-dimensional scene containing the stroke line through the material editor; in the three-dimensional scene tiled with the sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map to obtain a sketch-stylized scene. The method can achieve sketch stylization of the three-dimensional scene by coloring the strokes of the scene element model and tiling the sketch stroke map. The three-dimensional scene to be processed can be converted into a sketch style without redesigning the modeling, which greatly improves the scene rendering efficiency and scene scalability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sketch-style scene rendering method, device and storage medium. Background Art

[0002] Sketching is a common way of painting. Generally speaking, in sketching works, lines and strokes are usually used to depict the relationship between objects and light sources, thereby reflecting the outline of the object and the detailed features of the object.

[0003] Taking game development as an example, there are usually multiple scenes in the game. In the related art, if you want to change the game scene from a realistic style to a sketch style, you need to redraw and model each scene in the game, which greatly reduces the efficiency of game development and leads to poor scalability of existing game scenes. Therefore, it is urgent to propose a new solution to overcome the current technical problems. Summary of the invention

[0004] Multiple aspects of the present application provide a sketch-style scene rendering method, device and storage medium to improve scene rendering efficiency and enhance scene scalability.

[0005] The present application also provides a sketch-style scene rendering method, the method comprising:

[0006] For a three-dimensional scene to be processed, determining the position of the stroke line in each scene element model;

[0007] According to the position of the stroke line, each scene element model is stroked and colored to obtain a three-dimensional scene containing the stroke line;

[0008] According to the normal direction of each scene element model, the sketch stroke map is tiled into the three-dimensional scene containing the stroke line through the material editor;

[0009] In a three-dimensional scene tiled with a sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map to obtain a sketch stylized scene.

[0010] Further optionally, the material editor includes a material ball acting in a post-processing stage.

[0011] Based on this, the method of tiling the sketch stroke map into the three-dimensional scene containing the stroke line through the material editor according to the normal direction of each scene element model includes:

[0012] Inputting the normal direction of each scene element model, the sketch stroke map, and the tiled position and area of ​​the sketch stroke map in the three-dimensional scene containing the stroke line into the world aligned texture node added in the material ball;

[0013] According to the tiling position of the sketch stroke map, the sketch stroke map is set to each scene element model of the three-dimensional scene containing the stroke line through the world aligned texture node; and

[0014] According to the normal direction of each scene element model and the initial normal direction of the sketch stroke map, the orientation of the sketch stroke map in each scene element model is adjusted through the world alignment texture node.

[0015] Further optionally, the orientation of the sketch stroke map in each scene element model includes:

[0016] The plane where the sketch stroke map is located is perpendicular to the normal direction of each scene element model.

[0017] Further optionally, the sketch stroke map is a map with a four-dimensionally continuous style and a sketch stroke effect.

[0018] Further optionally, the stroke line includes a contour line and / or a polyline, the contour line is a stroke line used to represent an external contour of the model, and the polyline is a stroke line used to represent an internal contour of the model;

[0019] Determining the position of the stroke line in each scene element model includes:

[0020] Obtain edge pixels on contour lines and / or polylines in each scene element model.

[0021] Further optionally, if the stroke line includes a contour line, then obtaining edge pixels on the contour line and / or the polyline in each scene element model includes:

[0022] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient;

[0023] Get the depth difference between the current pixel and the surrounding pixels;

[0024] If the depth difference is greater than a set depth difference threshold, the current pixel is used as an edge pixel on the contour line in the scene element model.

[0025] Further optionally, if the stroke line includes a polyline, the step of obtaining edge pixels on the contour line and / or the polyline in each scene element model includes:

[0026] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient;

[0027] Compare the normal values ​​of the current pixel and the surrounding pixels in the world coordinate system;

[0028] If the normal difference between the current pixel and the surrounding pixels is greater than the set normal difference threshold, the current pixel is used as the edge pixel on the polyline in the scene element model.

[0029] Further optionally, the step of performing stroke coloring on each scene element model according to the position of the stroke line to obtain a three-dimensional scene containing the stroke line includes:

[0030] Generate a first mask image including edge pixels in each scene element model according to edge pixels on the stroke line in each scene element model;

[0031] According to the scene color of each edge pixel, linear interpolation is performed between each edge pixel in the first mask image to obtain a three-dimensional scene including the stroke line.

[0032] Further optionally, the method further includes:

[0033] According to the texture coordinates of each edge pixel, the three-dimensional noise map is sampled to obtain the noise value corresponding to each edge pixel;

[0034] According to the noise value corresponding to each edge pixel, the scene color of each edge pixel is offset to obtain a three-dimensional scene containing discontinuous stroke lines.

[0035] Further optionally, the step of fusing the scene color with the stroke color of the sketch stroke map to obtain a sketch stylized scene includes:

[0036] The scene color of each pixel is superimposed with the stroke color of the corresponding pixel in the sketch stroke map to obtain the target scene color of each pixel;

[0037] Rendering is performed based on the target scene color of each pixel to obtain a sketch-styled scene.

[0038] Further optionally, if there are different types of sketch stroke maps for different scene elements, the method further includes:

[0039] Use custom depth stencil items to filter scene elements using different types of sketch stroke textures;

[0040] generating a second mask map matching different types of sketch stroke maps according to the filtering result;

[0041] Using different types of sketch stroke maps and a matching second mask map, a step of tiling the different types of sketch stroke maps into a three-dimensional scene containing the stroke line is performed.

[0042] Further optionally, the method further includes:

[0043] Use custom depth template items to filter target scene elements that do not require sketch stroke maps;

[0044] A third mask map for marking the target scene element is generated according to the filtering result, and the third mask map is used to execute the step of tiling different types of sketch stroke maps into the three-dimensional scene containing the stroke line, so as to retain the initial scene color of the target scene element in the sketch stylized scene.

[0045] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.

[0046] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed.

[0047] In the technical solution provided in the embodiment of the present application, for the three-dimensional scene to be processed, the position of the stroke line in each scene element model is determined; according to the position of the stroke line, each scene element model is stroked and colored to obtain a three-dimensional scene containing the stroke line, thereby depicting the outline of each scene element model. Then, according to the normal direction of each scene element model, the sketch stroke map is tiled to the three-dimensional scene containing the stroke line through the material editor; in the three-dimensional scene tiled with the sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map, thereby realizing the sketch strokes on the surface of each scene element model and obtaining a sketch stylized scene.

[0048] In the embodiment of the present application, the outline of each scene element model is depicted by performing edge detection and stroke coloring on each scene element model, and then the sketch stroke map is tiled in each scene element model through the material editor to fill the scene color with a sketch style, thereby realizing the sketch stylization of the three-dimensional scene. In the embodiment of the present application, the three-dimensional scene to be processed can be converted into a sketch style without redesigning the modeling, which greatly improves the scene rendering efficiency and enhances the scene scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0050] Figure 1 A schematic flow chart of a sketch-style scene rendering method provided by an exemplary embodiment of the present application;

[0051] Figure 2 A schematic diagram of the effect of a scene element provided by an exemplary embodiment of the present application;

[0052] Figure 3 A schematic diagram of a sketch stroke map provided for an exemplary embodiment of the present application;

[0053] Figure 4 A schematic diagram of another effect of a scene element provided by an exemplary embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0056] Sketching is a commonly used method of painting. At present, in scenes expressed by sketching, the relationship between objects and light sources is depicted through line strokes, thereby reflecting the outline of the object and the detailed features of the object.

[0057] Taking game development as an example, a game usually has multiple scenes. In the related art, if a game scene is to be changed from a realistic style to a sketch style, each scene in the game needs to be redrawn and modeled, which greatly reduces the efficiency of game development and leads to poor scalability of existing game scenes.

[0058] In view of the above technical problems, a solution is provided in some embodiments of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0059] The present application embodiment provides a sketch-style scene rendering method. Figure 1The following is a flow chart of a method for rendering a scene in a sketch style provided by an exemplary embodiment of the present application. Figure 1 As shown, the method includes:

[0060] 101. For the three-dimensional scene to be processed, determine the position of the stroke line in each scene element model;

[0061] 102. Stroke and color each scene element model according to the position of the stroke line to obtain a three-dimensional scene including the stroke line;

[0062] 103. According to the normal direction of each scene element model, the sketch stroke map is tiled into the three-dimensional scene containing the stroke line through the material editor;

[0063] 104. In a three-dimensional scene tiled with a sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map to obtain a sketch stylized scene.

[0064] Among them, the three-dimensional scene to be processed is the three-dimensional scene that needs to be rendered. For example, suppose the three-dimensional scene to be processed is a realistic game scene in a game. After post-processing, another game scene in the sketch style of the game is obtained. In the two game scenes before and after post-processing, the scene elements, scene element layout, and the attributes of each scene element are consistent, but the scene styles of the two game scenes are different. Specifically, the scene style can be reflected in the art style characteristics in the three-dimensional scene, such as color tone, lines, light and dark relationships, etc.

[0065] For a game scene, different scene styles can be understood as different art styles in the game scene. For example, in a realistic game scene, the scene color tone and the light and dark relationship of the scene elements are closer to the real world, and the scene elements are not outlined by obvious lines. In a sketch-style game scene, the outline and light and dark relationship of the scene elements are depicted by line strokes, thus reflecting the detailed characteristics of the objects.

[0066] In conjunction with specific embodiments, Figure 1 The various steps in the sketch-style scene rendering method are described below.

[0067] First, in step 101, for the three-dimensional scene to be processed, the position of the stroke line in each scene element model is determined.

[0068] Among them, the stroke line includes a contour line and / or a polyline. The contour line is a stroke line used to represent the outer contour of the model, and the polyline is a stroke line used to represent the inner contour of the model. Optionally, the edge of the scene element model is determined by edge detection to provide a basis for stroke coloring. In practical applications, starting from the currently detected pixel, the coordinates of the surrounding pixels are obtained to detect the pixels at the edge of the scene element model. The following describes how to obtain contour lines and polylines respectively.

[0069] Specifically, an optional embodiment of determining the position of the stroke line in each scene element model in 101 may be implemented as follows: obtaining edge pixels on the contour line and / or the polyline in each scene element model.

[0070] In an optional embodiment, if the stroke line includes a contour line, then obtaining edge pixels on the contour line and / or the polyline in each scene element model in the above step can be implemented as follows:

[0071] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; the depth difference between the current pixel and the surrounding pixels is obtained; if the depth difference is greater than the set depth difference threshold, the current pixel is used as the edge pixel on the contour line in the scene element model.

[0072] The offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient. Specifically, the normal direction of the current pixel is extracted from the scene texture as the offset direction; the current screen viewport size is extracted from the scene texture, and the product of the current screen viewport size and the offset coefficient is used as the offset distance.

[0073] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset based on the offset direction and offset distance described above to obtain the texture coordinates of the surrounding pixels. Specifically, the texture coordinates of the surrounding pixels are obtained through vector operations based on the coordinates of the current pixel in the screen space. Then, the corresponding depth difference is calculated based on the scene depth of the current pixel and the surrounding pixels. If the depth difference between the current pixel and any surrounding pixel is greater than the set depth difference threshold, the current pixel is used as the edge pixel on the contour line in the scene element model.

[0074] In another optional embodiment, if the stroke line includes a polyline, the above step of obtaining edge pixels on the contour line and / or polyline in each scene element model can be implemented as follows:

[0075] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; the normal values ​​of the current pixel and the surrounding pixels in the world coordinate system are compared; if the normal difference between the current pixel and the surrounding pixels is greater than the set normal difference threshold, the current pixel is considered to be in the contour of the scene element model. In this case, the current pixel is used as the edge pixel on the broken line in the scene element model.

[0076] The offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient. The specific method of obtaining the offset direction and offset distance of the current pixel is similar to the above, and will not be repeated here.

[0077] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset based on the offset direction and offset distance described above to obtain the texture coordinates of the surrounding pixels. According to the normal values ​​of the current pixel and the surrounding pixels in the world coordinate system, the corresponding normal difference is calculated. If the normal difference between the current pixel and any surrounding pixel is greater than the set normal difference threshold, the current pixel is considered to be in the inner contour of the scene element model. In this case, the current pixel is used as the edge pixel on the polyline in the scene element model.

[0078] In practical applications, the above thresholds can be set and optimized according to specific scenarios, and are not limited in the embodiments of the present application.

[0079] Then, after the position of the stroke line is detected, in step 102 , each scene element model is stroked and colored according to the position of the stroke line to obtain a three-dimensional scene including the stroke line.

[0080] Specifically, in an optional embodiment, in 102, each scene element model is stroked and colored according to the position of the stroke line to obtain a three-dimensional scene containing the stroke line, which can be implemented as follows:

[0081] According to the edge pixels on the stroke line in each scene element model, a first mask image including the edge pixels in each scene element model is generated; according to the scene color of each edge pixel, linear interpolation (Linear Lerp) is performed between each edge pixel in the first mask image to obtain a three-dimensional scene including the stroke line.

[0082] In the above steps, a first mask image is generated according to edge pixels on the stroke line in each scene element model, and the first mask image includes edge pixels in each scene element model. Then, for each edge pixel in the first mask image, a linear interpolation is performed on the scene color of each edge pixel and a preset stroke line color to obtain a three-dimensional scene including a stroke line.

[0083] In this way, through the above steps, the rendering of stroke lines can be realized in the three-dimensional scene, so that stroke lines with a sketch style are automatically generated, which helps to improve the rendering efficiency of the three-dimensional scene.

[0084] In fact, since sketch strokes in the real world are not all continuous line segments, continuous stroke lines will be obtained after stroking and coloring through the above steps, making the sketch effect unnatural. In order to solve this problem and make the sketch strokes have a more realistic effect, it is further optional to sample the three-dimensional noise map according to the texture coordinates of each edge pixel to obtain the noise value corresponding to each edge pixel; according to the noise value corresponding to each edge pixel, the scene color of each edge pixel is offset to obtain a three-dimensional scene containing non-continuous stroke lines.

[0085] For a three-dimensional scene containing stroked lines, in the above steps, the scene depth (SceneDepth) of the current pixel is first used for inverse operation to obtain the world coordinates of the current pixel in the world coordinate system. Then, the world coordinates of the current pixel are used for remapping to obtain the texture coordinates of the current pixel for the three-dimensional noise map (denoted as UVW). Furthermore, the texture coordinates of the current pixel are used to sample the three-dimensional Perlin Noise map to obtain continuous noise values. In this way, these continuous noise values ​​are used to offset the stroked lines to simulate the effect of short lines and obtain a three-dimensional scene containing discontinuous stroked lines. Among them, Perlin noise is a continuous noise that is often used to simulate clouds.

[0086] In simple terms, the 3D Perlin noise map is mapped with the texture coordinates of the current pixel to obtain a continuous noise value. In practical applications, the continuous noise value usually ranges from 0 to 1. Then, the continuous noise value is multiplied by the color value of the stroke line to obtain a stroke line containing an invalid value (for example, a continuous noise value of 0). Figure 2 Discontinuous dashes are shown in boxes.

[0087] Then, after the stroke coloring is completed, in 103, the sketch stroke map is tiled into the three-dimensional scene containing the stroke line through the material editor according to the normal direction of each scene element model.

[0088] Among them, the sketch stroke map is a map with a four-dimensional continuous style and a sketch stroke effect. Four-dimensional continuity means that the edges of the map texture are continuous, and the texture is continuous after multiple maps are connected. For example Figure 3 Sketch stroke map shown.

[0089] Optionally, different types of sketch stroke maps can be tiled into a three-dimensional scene containing stroke lines to adapt to different types of scene element models. In practical applications, the types of sketch stroke maps include, but are not limited to, at least one of map size, map texture, and line thickness.

[0090] In an optional embodiment, in 103, according to the normal direction of each scene element model, the sketch stroke map is tiled into the three-dimensional scene containing the stroke line through the material editor, which can be implemented as follows:

[0091] The normal direction of each scene element model, the sketch stroke map, and the tiling position and area of ​​the sketch stroke map in the three-dimensional scene containing the stroke line are input into the World AlignedTexture node added in the material ball.

[0092] Among them, the material editor includes a material ball that acts on the post-processing stage. Various material nodes can be called in the material ball to implement corresponding functions. The material node is a visual script node that converts the blueprint state machine into a material expression. In an optional embodiment, a world-aligned texture node is added to the material ball. Specifically, in the world-aligned texture node, projection textures at different positions and under different axes can be specified in the world coordinate system, and the texture size is kept from scaling with the scaling of the scene element model.

[0093] Then, according to the tiling position of the sketch brush map, the sketch brush map is set to each scene element model of the three-dimensional scene containing the stroke line through the world alignment texture node. And, according to the normal direction of each scene element model and the initial normal direction of the sketch brush map, the orientation of the sketch brush map in each scene element model is adjusted through the world alignment texture node. Among them, the orientation of the sketch brush map in each scene element model at least includes: the plane where the sketch brush map is located is perpendicular to the normal direction of each scene element model.

[0094] For example, suppose the scene element model is Figure 4 In the box shown, the six surfaces of the box have different orientations. Therefore, it is necessary to adjust the tiling direction of the sketch stroke map according to the orientations of the six surfaces to obtain a sketch texture effect that adapts to each surface of the box.

[0095] Simply put, the sketch stroke map (Texture Object), texture size (Texture Size), the position coordinates of each pixel in screen space (World Position) and the normal information (World Space Normal) are input into the World Aligned Texture Node, so that the orientation of the sketch stroke map superimposed on these pixels can be adjusted through the World Aligned Texture Node.

[0096] Finally, in 104, in the three-dimensional scene tiled with the sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map to obtain a sketch stylized scene.

[0097] Specifically, in an optional embodiment, the scene color of each pixel can be superimposed with the stroke color of the corresponding pixel in the sketch stroke map to obtain the target scene color of each pixel; rendering is performed based on the target scene color of each pixel to obtain a sketch stylized scene.

[0098] In this embodiment, by performing edge detection and stroke coloring on each scene element model, the outline of each scene element model is depicted, and then the sketch stroke map is tiled in each scene element model through the material editor to fill the scene color with a sketch style, so as to achieve the sketch stylization of the three-dimensional scene. In this embodiment of the application, the three-dimensional scene to be processed can be converted into a sketch style without redesigning the modeling, which greatly improves the scene rendering efficiency and enhances the scene scalability.

[0099] In the above or following embodiments, in actual sketch works, the brush strokes used on the surfaces of different objects are also different. For example, the brush strokes on the surface of a large object may be more spaced apart, while the brush strokes on the surface of a small object may be more delicate and spaced apart. Therefore, different types of sketch brush stroke maps need to be used for rendering for different types of objects, wherein the type of sketch brush stroke map includes at least one of the following: map size, map texture, and line thickness. In an optional embodiment, the following steps are performed:

[0100] If there are different types of sketch stroke maps for different scene elements, a custom depth stencil item (Custom Stencil) is used to filter the scene elements using different types of sketch stroke maps; a second mask map matching the different types of sketch stroke maps is generated based on the filtering results; and the different types of sketch stroke maps and the matching second mask map are used to execute the step of tiling the different types of sketch stroke maps into the three-dimensional scene containing stroke lines.

[0101] It is worth noting that the steps of tiling different types of sketch stroke maps into a three-dimensional scene containing stroke lines are similar to the above step 103 and will not be repeated here.

[0102] Therefore, through the above steps, different types of brush strokes can be achieved in different types of scene elements, so that the sketch brush stroke effects of each scene element are more consistent with the brush stroke effects in actual sketch works, improving the brush stroke effects and improving the scene rendering efficiency of the sketch stylized scene.

[0103] In the above or following embodiments, since some scene elements in the three-dimensional scene do not need to be covered with the sketch texture, these scene elements need to be filtered, and a third mask image is used to prevent these scene elements from being covered with the sketch-style texture, so that these scene elements retain the original scene color.

[0104] Specifically, a custom depth template item can be used to filter target scene elements that do not require a sketch stroke map; a third mask map is generated based on the filtering results to mark the target scene elements, and the third mask map is used to perform the step of tiling different types of sketch stroke maps into a three-dimensional scene containing stroke lines, so as to retain the initial scene color of the target scene elements in the sketch-stylized scene.

[0105] The aforementioned partial scene elements include, for example, various scene icons, virtual characters, and other scene elements that do not need to be covered with sketch textures.

[0106] For example, according to the marks corresponding to various scene icons, the corresponding marks to be filtered are set in the custom depth template item to filter out various scene icons, and generate the corresponding third mask map to flexibly retain the initial scene color of various scene icons and improve rendering efficiency.

[0107] For example, according to the scene depth of the virtual character in the three-dimensional scene, the corresponding scene depth is set in the custom depth template item to filter out the virtual character, and generate a corresponding third mask map to retain the appearance image of the virtual character in the three-dimensional scene.

[0108] In this way, through the above steps, in the sketch-stylized scene, the initial scene colors of some scene elements are retained, thereby improving the scene rendering efficiency of the sketch-stylized scene.

[0109] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0110] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0111] Figure 5 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, such as Figure 5As shown, the electronic device includes: a memory 501 , a processor 502 , a communication component 503 and a display component 504 .

[0112] The memory 501 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0113] Among them, the memory 501 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0114] The processor 502 is coupled to the memory 501 and is used to execute the computer program in the memory 501, so as to: determine the position of the stroke line in each scene element model to be processed; perform stroke coloring on each scene element model according to the position of the stroke line to obtain a three-dimensional scene containing the stroke line; tile the sketch stroke map into the three-dimensional scene containing the stroke line through the material editor according to the normal direction of each scene element model; and in the three-dimensional scene tiled with the sketch stroke map, merge the scene color with the stroke color of the sketch stroke map to obtain a sketch stylized scene.

[0115] Further optionally, the material editor includes a material ball acting in a post-processing stage.

[0116] Based on this, when the processor 502 tiles the sketch stroke map into the three-dimensional scene containing the stroke line through the material editor according to the normal direction of each scene element model, it is specifically used to:

[0117] The normal direction of each scene element model, the sketch stroke map, and the tiling position and area of ​​the sketch stroke map in the three-dimensional scene containing the stroke line are input into the world-aligned texture node added in the material ball; according to the tiling position of the sketch stroke map, the sketch stroke map is set to each scene element model of the three-dimensional scene containing the stroke line through the world-aligned texture node; and according to the normal direction of each scene element model and the initial normal direction of the sketch stroke map, the orientation of the sketch stroke map in each scene element model is adjusted through the world-aligned texture node.

[0118] Among them, further optionally, when the processor 502 determines the orientation of the sketch stroke map in each scene element model, it is specifically used to: the plane where the sketch stroke map is located is perpendicular to the normal direction of each scene element model.

[0119] Among them, further optionally, the sketch stroke map is a map with a four-way continuous style and a sketch stroke effect.

[0120] Further optionally, the stroke line includes a contour line and / or a polyline, the contour line is a stroke line used to represent an external contour of the model, and the polyline is a stroke line used to represent an internal contour of the model.

[0121] When the processor 502 determines the position of the stroke line in each scene element model, it is specifically used to: obtain edge pixels on the contour line and / or the polyline in each scene element model.

[0122] Wherein, further optionally, if the stroke line includes a contour line, when the processor 502 obtains edge pixels on the contour line and / or the polyline in each scene element model, it is specifically used to:

[0123] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient; the depth difference between the current pixel and the surrounding pixels is obtained; if the depth difference is greater than the set depth difference threshold, the current pixel is used as the edge pixel on the contour line in the scene element model.

[0124] Wherein, further optionally, if the stroke line includes a polyline, when the processor 502 obtains edge pixels on the contour line and / or polyline in each scene element model, it is specifically used to:

[0125] For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient; the normal values ​​of the current pixel and the surrounding pixels in the world coordinate system are compared; if the normal difference between the current pixel and the surrounding pixels is greater than the set normal difference threshold, the current pixel is used as the edge pixel on the broken line in the scene element model.

[0126] Further optionally, the processor 502 performs stroke coloring on each scene element model according to the position of the stroke line to obtain a three-dimensional scene including the stroke line, specifically for:

[0127] According to the edge pixels on the stroke line in each scene element model, a first mask image including the edge pixels in each scene element model is generated; according to the scene color of each edge pixel, linear interpolation is performed between the edge pixels in the first mask image to obtain a three-dimensional scene including the stroke line.

[0128] Further optionally, the processor 502 is further configured to:

[0129] According to the texture coordinates of each edge pixel, the three-dimensional noise map is sampled to obtain the noise value corresponding to each edge pixel; according to the noise value corresponding to each edge pixel, the scene color of each edge pixel is offset to obtain a three-dimensional scene containing discontinuous stroke lines.

[0130] Among them, further optionally, when the processor 502 merges the scene color with the stroke color of the sketch stroke map to obtain a sketch stylized scene, it is specifically used to:

[0131] The scene color of each pixel is superimposed with the stroke color of the corresponding pixel in the sketch stroke map to obtain the target scene color of each pixel; rendering is performed based on the target scene color of each pixel to obtain a sketch stylized scene.

[0132] Wherein, further optionally, if there are different types of sketch stroke maps for different scene elements, the processor 502 is further configured to:

[0133] A custom depth template item is used to filter scene elements using different types of sketch stroke maps; a second mask map matching the different types of sketch stroke maps is generated based on the filtering result; and different types of sketch stroke maps and the matching second mask map are used to execute the step of tiling the different types of sketch stroke maps into the three-dimensional scene containing the stroke lines.

[0134] Further optionally, the processor 502 is further configured to:

[0135] A custom depth template item is used to filter target scene elements that do not require a sketch stroke map; a third mask map is generated based on the filtering result to mark the target scene elements, and the third mask map is used to execute the step of tiling different types of sketch stroke maps into the three-dimensional scene containing the stroke line, so as to retain the initial scene color of the target scene elements in the sketch stylized scene.

[0136] Further, if Figure 5 As shown, the electronic device also includes: a power supply component 505, an audio component 506 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0137] Among them, the communication component 503 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0138] Among them, the display component 504 can be implemented as a display, which includes a screen, and the screen can include a liquid crystal display (LCD) and a touch panel (TP). The screen includes a touch panel, and the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0139] The power supply component 505 provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0140] In this embodiment, by performing edge detection and stroke coloring on each scene element model, the outline of each scene element model is depicted, and then the sketch stroke map is tiled in each scene element model through the material editor to fill the scene color with a sketch style, thereby realizing the sketch stylization of the three-dimensional scene. In the embodiment of the present application, the three-dimensional scene to be processed can be converted into a sketch style without redesigning the modeling, which greatly improves the scene rendering efficiency and enhances the scene scalability.

[0141] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0148] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carriers.

[0149] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0150] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A sketch-style scene rendering method, characterized in that: The method comprises: For a three-dimensional scene to be processed, determining the position of the stroke line in each scene element model; According to the position of the stroke line, each scene element model is stroked and colored to obtain a three-dimensional scene containing the stroke line; Inputting the normal direction of each scene element model, the sketch stroke map, and the tiling position and area of ​​the sketch stroke map in the three-dimensional scene containing the stroke line into the world aligned texture node added in the material ball; According to the tiling position of the sketch stroke map, the sketch stroke map is set to each scene element model of the three-dimensional scene containing the stroke line through the world aligned texture node; and According to the normal direction of each scene element model and the initial normal direction of the sketch stroke map, the orientation of the sketch stroke map in each scene element model is adjusted through the world alignment texture node; In a three-dimensional scene tiled with a sketch stroke map, the scene color is merged with the stroke color of the sketch stroke map to obtain a sketch stylized scene.

2. The method according to claim 1, characterized in that The orientation of the sketch stroke map in each scene element model includes: The plane where the sketch stroke map is located is perpendicular to the normal direction of each scene element model.

3. The method according to any one of claims 1 to 2, characterized in that: Sketch stroke textures are textures with a four-dimensional continuous style and sketch stroke effects.

4. The method according to claim 1, characterized in that: The stroke lines include contour lines and / or polylines, the contour lines are stroke lines used to represent the external contour of the model, and the polylines are stroke lines used to represent the internal contour of the model; Determining the position of the stroke line in each scene element model includes: Obtain edge pixels on contour lines and / or polylines in each scene element model.

5. The method according to claim 4, characterized in that If the stroke line includes a contour line, the step of obtaining edge pixels on the contour line and / or the polyline in each scene element model includes: For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient; Get the depth difference between the current pixel and the surrounding pixels; If the depth difference is greater than a set depth difference threshold, the current pixel is used as an edge pixel on the contour line in the scene element model.

6. The method according to claim 4, characterized in that If the stroke line includes a polyline, the step of obtaining edge pixels on the contour line and / or polyline in each scene element model includes: For each pixel to be detected in the scene element model, the texture coordinates of the current pixel are offset to obtain the texture coordinates of the surrounding pixels; wherein the offset direction is the normal direction of the current pixel, and the offset distance is the product of the current screen viewport size and the offset coefficient; Compare the normal values ​​of the current pixel and the surrounding pixels in the world coordinate system; If the normal difference between the current pixel and the surrounding pixels is greater than the set normal difference threshold, the current pixel is used as the edge pixel on the polyline in the scene element model.

7. The method according to claim 1, characterized in that The step of performing stroke coloring on each scene element model according to the position of the stroke line to obtain a three-dimensional scene containing the stroke line includes: Generate a first mask image including edge pixels in each scene element model according to edge pixels on the stroke line in each scene element model; According to the scene color of each edge pixel, linear interpolation is performed between each edge pixel in the first mask image to obtain a three-dimensional scene including the stroke line.

8. The method according to claim 7, characterized in that Also includes: According to the texture coordinates of each edge pixel, the three-dimensional noise map is sampled to obtain the noise value corresponding to each edge pixel; According to the noise value corresponding to each edge pixel, the scene color of each edge pixel is offset to obtain a three-dimensional scene containing discontinuous stroke lines.

9. The method according to claim 1, characterized in that: The step of fusing the scene color with the stroke color of the sketch stroke map to obtain a sketch stylized scene includes: The scene color of each pixel is superimposed with the stroke color of the corresponding pixel in the sketch stroke map to obtain the target scene color of each pixel; Rendering is performed based on the target scene color of each pixel to obtain a sketch-styled scene.

10. The method according to claim 1, characterized in that If there are different types of sketch stroke maps for different scene elements, the method further includes: Use custom depth stencil items to filter scene elements using different types of sketch stroke textures; generating a second mask map matching different types of sketch stroke maps according to the filtering result; Using different types of sketch stroke maps and a matching second mask map, a step of tiling the different types of sketch stroke maps into a three-dimensional scene containing the stroke line is performed.

11. The method according to claim 1, characterized in that: Also includes: Use custom depth template items to filter target scene elements that do not require sketch stroke maps; A third mask map for marking the target scene element is generated according to the filtering result, and the third mask map is used to execute the step of tiling different types of sketch stroke maps into the three-dimensional scene containing the stroke line, so as to retain the initial scene color of the target scene element in the sketch stylized scene.

12. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program; When the computer program is executed, it is used to implement the steps in the method according to any one of claims 1 to 11.

Images