Method and device for rendering image in ink-wash style

Abstract

The invention provides a method and a device for rendering an image in an ink-wash style, and relates to the technical field of computer vision. According to the specific implementation scheme, the method comprises the steps of performing edge detection on a target object in a to-be-rendered image to obtain contour line features of the target object; according to the contour line features and / or a preset noise texture map, performing hook-line shading on the target object to obtain an initial rendering image; based on the scene space position, mapping the chapping texture map to a screen space where the initial rendering image is located to obtain an intermediate rendering image; and performing shading effect control based on an illumination calculation result on the intermediate rendering image to obtain a target rendering image in an ink-wash style.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an ink-and-wash style image rendering method and device. Background Art

[0002] With the in-depth development of computer vision technology, image style transfer has received widespread attention, and the demand for ink-style image rendering has also been growing.

[0003] However, in some ink-and-wash style image rendering practices, there are problems such as limited detail control and insufficient support for dynamic effects. Summary of the invention

[0004] The present disclosure provides an ink-and-wash style image rendering method, device, electronic device, and medium.

[0005] According to one aspect of the present disclosure, a method for rendering an image in ink painting style is provided, the method comprising: performing edge detection on a target object in an image to be rendered to obtain contour features of the target object; outlining and shading the target object according to the contour features and / or a preset noise texture map to obtain an initial rendered image; based on a scene spatial position, mapping the texture map to a screen space where the initial rendered image is located to obtain an intermediate rendered image; and controlling a shading effect on the intermediate rendered image based on a lighting calculation result to obtain a target rendered image in ink painting style.

[0006] By adopting technologies such as image space edge detection, screen mapping and light shading, the three core painting links of traditional ink painting, namely "outlining, incising and dyeing", are digitally simulated, which is beneficial to improving the detail and diversity of ink style image rendering, and can effectively improve the rendering quality of ink style image rendering, which is beneficial to providing an immersive virtual pen and ink painting experience and can meet the application needs of ink style image rendering in virtual scenes.

[0007] In some embodiments, the performing edge detection on the target object in the image to be rendered to obtain the contour line features of the target object includes: performing edge detection on the target object based on scene depth information to obtain the outer contour features of the target object; and performing edge detection on the target object based on the surface normal of the object to obtain the inner contour features of the target object, the outer contour features and the inner contour features constitute the contour line features of the target object.

[0008] In some embodiments, the contour line features include contour line pixel coordinates and contour line pixel size, and the target object is outlined and smudged according to the contour line features and / or a preset noise texture map to obtain an initial rendered image, including: outlining and smudged the target object in at least one of the following ways to obtain the initial rendered image: adjusting the contour line pixel coordinates and / or the contour line pixel size of the target object; mapping the noise texture map to the screen space where the image to be rendered is located, and adjusting the pixel coordinates corresponding to the noise texture map to displace, rotate and scale the noise texture map; and performing a product operation on the contour line features and the noise texture map to generate simulated brush strokes with a flying white and smudge effect.

[0009] In some embodiments, mapping the texture map to the screen space where the initial rendered image is located based on the scene space position to obtain an intermediate rendered image includes: mapping the texture map to the screen space where the initial rendered image is located to obtain the intermediate rendered image in at least one of the following ways: mapping the texture map to the inner contour position of the target object in the initial rendered image; adjusting the pixel coordinates corresponding to the texture map to displace, rotate and scale the texture map; and using lighting information as a mask to map the texture map to the shadow area in the initial rendered image.

[0010] In some embodiments, the method further includes: scanning the drawn initial texture to obtain an intermediate texture; and adjusting at least one of the following parameters of the intermediate texture to obtain the texture map: texture, size, resolution and color depth.

[0011] In some embodiments, controlling the blending effect of the intermediate rendering image based on the illumination calculation result to obtain the target rendering image in ink style includes: obtaining the intrinsic color information of the target object from the diffuse reflection color buffer; obtaining the screen color information of the intermediate rendering image based on the screen space from the post-processing input buffer; calculating the illumination buffer result according to the screen color information and the intrinsic color information; and controlling the blending effect of the intermediate rendering image according to the illumination buffer result to obtain the target rendering image in ink style.

[0012] In some embodiments, calculating the illumination buffer result based on the picture color information and the intrinsic color information includes: dividing the picture color information by the intrinsic color information to obtain illumination intensity information in the intermediate rendered image as the illumination buffer result.

[0013] In some embodiments, the step of controlling the blending effect on the intermediate rendering image according to the illumination buffer result to obtain the target rendering image in ink painting style comprises: performing noise texture map mask control and blending effect intensity control on the intermediate rendering image according to the illumination buffer result to obtain the target rendering image in ink painting style, wherein the blending effect intensity control comprises mask intensity control and shadow intensity control.

[0014] In some embodiments, the method further includes: in response to received image rendering instructions, providing a global control interface and a local control interface for image rendering control, wherein the global control interface is used to edit the scene appearance parameters of the image rendering, and the local control interface is used to edit the line drawing and shading parameters, brushstroke texture parameters and color adjustment parameters of the image rendering.

[0015] In some embodiments, the method further includes: in response to a received parameter editing instruction, providing a material parameter setting node for image rendering control, wherein the material parameter setting node includes at least one of a scalar parameter setting node, a vector parameter setting node, a texture parameter setting node, and a Boolean value setting node.

[0016] In some embodiments, the method further includes: creating a dynamic material instance in response to a received image rendering instruction, wherein the dynamic material instance includes material parameters involved in the image rendering process, and the dynamic material instance supports setting the material parameters in a graphical interface.

[0017] According to another aspect of the present disclosure, an ink-and-wash style image rendering device is provided, comprising: a first processing module, for performing edge detection on a target object in an image to be rendered to obtain contour features of the target object; a second processing module, for outlining and blending the target object according to the contour features and / or a preset noise texture map, to obtain an initial rendered image; a third processing module, for mapping the brushstroke texture map to the screen space where the initial rendered image is located based on the scene space position, to obtain an intermediate rendered image; and a fourth processing module, for controlling the blending effect of the intermediate rendered image based on a lighting calculation result, to obtain an ink-and-wash style target rendered image.

[0018] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned image rendering method.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned image rendering method.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned image rendering method when executed by a processor.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. Figure 1 The system architecture of the ink-and-wash style image rendering method and device according to an embodiment of the present disclosure is schematically shown; Figure 2 A flowchart of an ink-and-wash style image rendering method according to an embodiment of the present disclosure is schematically shown; Figure 3 A schematic diagram of a Perlin noise texture map according to an embodiment of the present disclosure is schematically shown; Figure 4 A schematic diagram of a texture map according to an embodiment of the present disclosure is schematically shown; Figure 5 The schematic diagram of the architecture of the image rendering system according to an embodiment of the present disclosure is shown schematically; Figure 6 A block diagram of an image rendering device according to an embodiment of the present disclosure is schematically shown; Figure 7 A block diagram of an electronic device for executing an ink-and-wash style image rendering method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0025] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0027] With the in-depth development of computer vision technology, image style transfer has received widespread attention, and the demand for ink-style image rendering is also increasing. However, in some ink-style image rendering practices, there are problems such as limited detail control and insufficient support for dynamic effects.

[0028] The disclosed embodiments provide an ink-and-wash style image rendering method, device, and computer-readable storage medium. The image rendering device can be integrated into an electronic device, and the electronic device can be a terminal device or a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal device can be a mobile phone, a computer, a tablet computer, a smart screen, a smart home appliance, a car terminal, etc.

[0029] According to an ink-and-wash style image rendering method of an embodiment of the present disclosure, edge detection is performed on a target object in an image to be rendered to obtain contour features of the target object. According to the contour features and / or a preset noise texture map, the target object is outlined and shaded to obtain an initial rendered image. Based on the scene space position, the texture map is mapped to the screen space where the initial rendered image is located to obtain an intermediate rendered image. And, the intermediate rendered image is controlled for a shade effect based on a lighting calculation result to obtain an ink-and-wash style target rendered image.

[0030] Figure 1 The system architecture of the ink-and-wash style image rendering method and device according to an embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] The system architecture 100 according to this embodiment may include a client 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the client 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links or optical fiber cables, etc. The server 103 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.

[0032] The client 101 interacts with the server 103 via the network 102 to receive or send data, etc. The client 101 is used, for example, to receive an image rendering instruction from a user, and send the image to be rendered indicated by the image rendering instruction to the server 103 .

[0033] The server 103 may be a server that provides various services, for example, it may be a processing server that performs ink-style rendering on an image to be rendered (only an example).

[0034] For example, the server 103 is used to perform edge detection on the target object in the image to be rendered to obtain the contour line features of the target object. The server 103 is also used to outline and smudge the target object according to the contour line features and / or the preset noise texture map to obtain an initial rendered image. And, based on the scene space position, the texture map is mapped to the screen space where the initial rendered image is located to obtain an intermediate rendered image. Then, the intermediate rendered image is controlled to have a smudge effect based on the illumination calculation result to obtain an ink-and-wash style target rendered image.

[0035] It should be noted that the ink-and-wash style image rendering method provided in the embodiment of the present disclosure can be executed by the server 103. Accordingly, the image rendering device provided in the embodiment of the present disclosure can be set in the server 103. The ink-and-wash style image rendering method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 103 and can communicate with the client 101 and / or the server 103. Accordingly, the image rendering device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 103 and can communicate with the client 101 and / or the server 103.

[0036] It should be understood that Figure 1 The number of clients, networks and servers in the embodiment is only for illustration. Any number of clients, networks and servers may be provided as required.

[0037] The present disclosure provides an ink-and-wash style image rendering method. Figure 1 The system architecture of Figure 2~Figure 7 The image rendering method of the ink painting style according to the exemplary embodiment of the present disclosure is described as follows. The image rendering method of the embodiment of the present disclosure can be, for example, Figure 1 The server 103 shown is used for execution.

[0038] Figure 2 The flowchart of the ink-and-wash style image rendering method according to an embodiment of the present disclosure is schematically shown.

[0039] like Figure 2 As shown, the ink-and-wash style image rendering method 200 of the embodiment of the present disclosure may include, for example, operations S210 to S240.

[0040] In operation S210 , edge detection is performed on a target object in the image to be rendered to obtain contour features of the target object.

[0041] Operation S220 : outlining and shading the target object according to the contour line features and / or the preset noise texture map to obtain an initial rendered image.

[0042] Operation S230 : Mapping the texture map to the screen space where the initial rendering image is located based on the scene space position to obtain an intermediate rendering image.

[0043] In operation S240 , a blurring effect is controlled on the intermediate rendering image based on the illumination calculation result to obtain a target rendering image in ink painting style.

[0044] The following is an example flow of each operation of the ink-and-wash image rendering method of this embodiment.

[0045] In operation S210 , edge detection is performed on a target object in the image to be rendered to obtain contour features of the target object.

[0046] Exemplarily, edge detection based on scene depth information can be performed on the target object to obtain the outer contour feature of the target object. Edge detection based on the surface normal of the target object can be performed on the target object to obtain the inner contour feature of the target object. The outer contour feature and the inner contour feature constitute, for example, the contour line feature of the target object.

[0047] For example, the depth edge detection technology can be used to perform edge detection on the target object based on the scene depth information to obtain the outer contour features of the target object. Specifically, the depth map of the image to be rendered can be obtained, and the depth map can be convolved with a Gaussian kernel to obtain a smoothed depth map. The gradient of the smoothed depth map is calculated to obtain the grayscale gradient value at each pixel point in the image to be rendered. The grayscale gradient value indicates the grayscale change rate at the corresponding pixel point. When the grayscale gradient value is greater than the preset gradient value threshold, it is determined that the corresponding pixel point is located on the outer contour line of the target object.

[0048] The surface normal of an object can be used to indicate the orientation of the surface of the object, and the directional difference between the surface normals of an object can be used to identify the contour of the object. For example, when there is a significant difference in the direction of the surface normals of adjacent pixels, the corresponding pixels are determined to be located at the contour of the object. Normal edge detection can be used to detect the external contour of an object, as well as the internal contour of an object. It can also be used to assist in internal texture tracing and restore the effect of brushstrokes.

[0049] For example, edge detection based on the surface normal of the target object is performed on the target object to obtain the inner contour feature of the target object. Assume that the surface normal of the object at the target pixel in the image to be rendered is p. Based on the target pixel, offset the preset coordinate units in the up, down, left, and right directions to obtain the neighboring pixel points. Assume that the surface normals of the object at the neighboring pixel points are p1, p2, p3, and p4 respectively, and calculate the vector dot product result. .

[0050] According to the vector dot product result, it can be determined whether the corresponding pixel point is located on a given plane. When the vector dot product result is greater than zero, it is determined that the corresponding pixel point is not located on the given plane. When the vector dot product result is greater than a preset threshold, it is determined that the corresponding pixel point is located on the contour line of the target object. Among them, the threshold selection has an important influence on the sensitivity and noise resistance of the contour detection result.

[0051] By performing edge detection on the target object in the rendered image, the pixel coordinates and pixel size of the target object's contour are obtained as contour features. The pixel coordinates of the contour indicate the position of the target object's contour, and the pixel size of the contour indicates the size value of a single pixel in the target object's contour.

[0052] Operation S220 : outlining and shading the target object according to the contour line features and / or the preset noise texture map to obtain an initial rendered image.

[0053] In the example, at least one of the following methods is used to outline and smudge the target object to obtain an initial rendered image: adjusting the contour pixel coordinates and / or contour pixel size of the target object; mapping the noise texture map to the screen space where the image to be rendered is located, and adjusting the pixel coordinates corresponding to the noise texture map to displace, rotate and scale the noise texture map; and performing a product operation on the contour features and the noise texture map to generate simulated brushstrokes with a flying white smudge effect.

[0054] For example, according to the pixel coordinates of the target object's contour line, a two-dimensional vector offset operation in the up, down, left, right, etc. directions can be performed on the current pixel position of the contour line to change the position of the target object's contour line. In addition, the pixel size of the target object's contour line can be multiplied by the number of pixels n to adjust the width of the target object's contour line. By adjusting the position and width of the target object's contour line, a dissolving or smudge visual effect can be produced, which is conducive to increasing the flexibility and diversity of rendering effects.

[0055] As an optional manner, a preset noise texture map may be mapped to the screen space where the image to be rendered is located. The noise texture map may be, for example, a Perlin noise texture map. Figure 3 A schematic diagram of a Perlin noise texture map according to an embodiment of the present disclosure is schematically shown. The Perlin noise texture is a gradient noise generation algorithm widely used in computer graphics. As a procedural texture, the Perlin noise texture can be used to generate smooth and coherent noise patterns. The Perlin noise texture map is visually closer to the texture and form in nature.

[0056] In the process of mapping the noise texture map to the screen space, the pixel coordinates corresponding to the noise texture map can be adjusted to displace, rotate and scale the noise texture map to achieve more precise control over the visual appearance of the image or shape.

[0057] Optionally, a product operation can be performed on the contour line features and the noise texture map to generate a simulated brushstroke of a Feibai smudge effect. For example, the outer contour pixel coordinates and the inner contour pixel coordinates of the target object can be alpha blended to obtain the contour pixel coordinates of the target object. The contour pixel coordinates and the noise texture map are producted to generate a visual representation of a simulated Feibai brushstroke effect.

[0058] For example, for any nearest contour pixel pair of the target object, assuming that the coordinates of the outer contour pixel point in the contour pixel pair are , the coordinates of the inner contour pixel are , perform Alpha blending on the outer contour pixel coordinates and the inner contour pixel coordinates to obtain the contour pixel coordinates of the target object , α is the preset Alpha blending coefficient.

[0059] In addition, the outer contour pixel size and the inner contour pixel size of the target object can be alpha blended to obtain the contour pixel size of the target object. The contour pixel size is multiplied with the noise texture map to independently control the intensity and range of the shading effect in the X and Y directions.

[0060] For example, for any nearest contour pixel pair of the target object, assuming that the size of the outer contour pixel in the contour pixel pair is , the size of the inner contour pixel is , perform Alpha blending on the outer contour pixel size and the inner contour pixel size to obtain the contour pixel size C=α of the target object + (1-α) , α is the preset Alpha blending coefficient.

[0061] Based on the contour features of the target object and / or the preset noise texture map, the target object is outlined and shaded to obtain an initial rendered image, which is conducive to showing excellent realism and artistry in simulating the contour of ink painting, and can effectively enhance the visual diversity of ink style image rendering.

[0062] Operation S230 : Mapping the texture map to the screen space where the initial rendering image is located based on the scene space position to obtain an intermediate rendering image.

[0063] In the practice of ink-and-wash style image rendering, in order to accurately reproduce the effect of brushstrokes, it is necessary not only to consider the fineness of the lines in the picture, but also the ink concentration and paper texture, so as to achieve an effect as close to traditional ink painting as possible.

[0064] Exemplarily, at least one of the following methods can be used to achieve mapping of the texture map to the screen space where the initial rendered image is located to obtain an intermediate rendered image: mapping the texture map to the inner contour position of the target object in the initial rendered image; adjusting the pixel coordinates corresponding to the texture map to displace, rotate and scale the texture map; and using lighting information as a mask to map the texture map to the shadow area in the initial rendered image.

[0065] Figure 4 A schematic diagram of a texture map according to an embodiment of the present disclosure is schematically shown, which can map the texture map to the inner contour position of the target object in the initial rendered image, and achieve simulated ink painting effects of various kinds by adjusting the texture intensity, direction and diffusion.

[0066] The pixel coordinates corresponding to the brushstroke texture map can be adjusted to displace, rotate and scale the brushstroke texture map, which is conducive to further achieving flexible adjustment and diverse changes in texture expression.

[0067] Since the texture is usually used to simulate the texture features of the dark part of an object, the texture effect in the bright area is often not significant. In the embodiment disclosed herein, the illumination calculation technology can be introduced, and the illumination information can be used as a mask to map the texture texture map to the shadow area in the initial rendered image. This design can not only make the application of the texture effect more targeted, but also be closer to the traditional expression techniques of light and shadow contrast in ink painting.

[0068] As an optional method, use the lighting information as a mask to map the texture map to the shadow area in the initial rendered image. During the image rendering process, you can use the lighting information as a mask to control the occlusion or display of specific areas in the image. Using the lighting information as a mask can ensure that the texture map is mainly effective in the shadow area in the initial rendered image.

[0069] Optionally, the user can manually draw the texture by using a brush and ink to draw the initial texture on rice paper. The drawn initial texture can be scanned to obtain an intermediate texture. And, at least one of the following parameters of the intermediate texture is adjusted to obtain a texture texture map: texture, size, resolution, and color depth. This design can effectively improve the diversity and layering of image rendering effects, and is conducive to ensuring the excellent performance of ink style image rendering.

[0070] In operation S240 , a blurring effect is controlled on the intermediate rendering image based on the illumination calculation result to obtain a target rendering image in ink painting style.

[0071] Exemplarily, the intrinsic color information of the target object can be obtained from the diffuse color buffer, and the screen color information of the intermediate rendering image based on the screen space can be obtained from the post-processing input buffer. The illumination buffer result is calculated based on the screen color information and the intrinsic color information. And, based on the illumination buffer result, the intermediate rendering image is controlled to perform a smudge effect to obtain a target rendering image in an ink style.

[0072] The diffuse color buffer stores the basic color information of the object, which can be the color information of the object before any lighting or post-processing effects. The basic color information can reflect the diffuse properties of each object in the screen space, that is, the inherent color information of the object in the absence of lighting. The diffuse color buffer can be the basic data source for obtaining the color information of each object in the scene.

[0073] When calculating the illumination buffer result according to the picture color information and the intrinsic color information, the picture color information may be divided by the intrinsic color information to obtain the illumination intensity information in the intermediate rendering image as the illumination buffer result.

[0074] For example, the relationship between the image color information and the inherent color information is expressed using equation (1): Diffuse Color×Light=FinalInput Formula (1) Among them, Diffuse Color represents the inherent color information of the target object, FinalInput represents the screen color information of the target object, and Light represents the light intensity information in the intermediate rendering image. The light intensity information, for example, constitutes the light buffer result.

[0075] Optionally, noise texture map mask control and smudge effect intensity control may be performed on the intermediate rendering image according to the illumination buffer result to obtain a target rendering image in ink style, wherein the smudge effect intensity control includes mask intensity control and shadow intensity control.

[0076] The noise texture map is masked and controlled on the intermediate rendering image. For example, the pixel coordinates corresponding to the noise texture map can be adjusted to displace, rotate and scale the noise texture map. In addition, the dissolving intensity of the noise texture map can be controlled.

[0077] By adopting technologies such as image space edge detection, screen mapping and light shading, the three core painting links of traditional ink painting, namely "outlining, incising and dyeing", are digitally simulated, which is beneficial to improving the detail and diversity of ink style image rendering, and can effectively improve the rendering quality of ink style image rendering, which is beneficial to providing an immersive virtual pen and ink painting experience and can meet the application needs of ink style image rendering in virtual scenes.

[0078] As an optional manner, the image rendering method of the embodiment of the present disclosure may further include: in response to the received image rendering instruction, providing a global control interface and a local control interface for image rendering control. The local control interface includes, for example, a hook rendering control interface, a texture rendering control interface, and a dye rendering control interface.

[0079] Exemplarily, the global control interface can be used to edit scene appearance parameters of image rendering, and the local control interface can be used to edit line drawing and shading parameters, brushstroke texture parameters, and color adjustment parameters of image rendering.

[0080] Figure 5 The schematic diagram of the architecture of the image rendering system according to an embodiment of the present disclosure is shown schematically. Figure 5 As shown, the image rendering system may provide, for example, a global control interface, a local control interface, and a post-processing component.

[0081] The global control interface includes, for example, a scene appearance control interface, which can be used to edit the scene appearance parameters of image rendering. The scene appearance control interface can be used to uniformly adjust the appearance of materials in the scene, such as adjusting parameters such as image brightness, image contrast, rice paper texture, background material, and ink color.

[0082] The hook rendering control interface is used to edit the hook and shading parameters of image rendering. The hook rendering control interface can provide functions such as hook rendering global control, shading control and flying white control. This design is conducive to achieving a more refined hook and shading effect, which can effectively enhance the naturalness and prominence of the edge of the object.

[0083] The texture rendering control interface is used to edit the texture parameters of image rendering. The texture rendering control interface can provide functions such as global texture rendering control, smudge control, and mask control. This design is conducive to better reproducing the texture details and surface texture of the object surface, and can effectively enhance the authenticity of ink style image rendering.

[0084] The dye rendering control interface is used to edit the color adjustment parameters of image rendering, and the dye rendering control interface can provide functions such as global control of dye rendering and dissolution effect control. This design is conducive to the detailed regulation of the color changes on the surface of the object, thereby achieving rich color levels and visual effects.

[0085] As an optional manner, in response to the received parameter editing instruction, a material parameter setting node for image rendering control is provided. The material parameter setting node includes at least one of a scalar parameter setting node, a vector parameter setting node, a texture parameter setting node, and a Boolean value setting node.

[0086] Exemplarily, the disclosed embodiments focus on implementing four subdivisions of material control: global control, outlining (outlining), texture refinement (texture) and color adjustment (dyeing) parameter management. Each subdivision involves fine management of specific material properties, which requires systematic configuration based on the adjustable parameters and their identifiers in the material instance. The adjustable parameters in the material instance can cover four types of parameters: scalar, vector, texture and Boolean.

[0087] The disclosed embodiment provides a material parameter setting node for image rendering control, and the material parameter setting node may include a scalar parameter setting node, a vector parameter setting node, a texture parameter setting node, and a Boolean value setting node. This design is conducive to enhancing the flexibility and application scope of material instances, and is conducive to building a highly flexible and customizable dynamic material system, which can provide full support for precise control and personalized adjustment of post-processing effects.

[0088] like Figure 5 As shown, the image rendering system may also provide a post-processing component, which may provide functions such as switch selection, rendering range control, and dynamic instance update. As an optional method, a dynamic material instance is created in response to a received image rendering instruction. The dynamic material instance includes material parameters involved in the image rendering process, and the dynamic material instance supports setting material parameters in a graphical interface.

[0089] Figure 6 A block diagram of an image rendering device according to an embodiment of the present disclosure is schematically shown.

[0090] like Figure 6 As shown, the ink-and-wash style image rendering device 600 of the embodiment of the present disclosure includes, for example, a first processing module 610 , a second processing module 620 , a third processing module 630 and a fourth processing module 640 .

[0091] The first processing module 610 performs edge detection on the target object to obtain the contour features of the target object. The second processing module 620 is used to outline and smudge the target object according to the contour features and / or the preset noise texture map to obtain an initial rendered image. The third processing module 630 is used to map the texture map to the screen space where the initial rendered image is located based on the scene space position to obtain an intermediate rendered image. And the fourth processing module 640 is used to control the smudge effect of the intermediate rendered image based on the illumination calculation result to obtain an ink-and-wash style target rendered image.

[0092] By adopting technologies such as image space edge detection, screen mapping and light shading, the three core painting links of traditional ink painting, namely "outlining, incising and dyeing", are digitally simulated, which is beneficial to improving the detail and diversity of ink style image rendering, and can effectively improve the rendering quality of ink style image rendering, which is beneficial to providing an immersive virtual pen and ink painting experience and can meet the application needs of ink style image rendering in virtual scenes.

[0093] In some embodiments, the first processing module includes: a first processing sub-module, used to perform edge detection on the target object based on scene depth information to obtain the outer contour features of the target object; and a second processing sub-module, used to perform edge detection on the target object based on the surface normal of the object to obtain the inner contour features of the target object, and the outer contour features and the inner contour features constitute the contour line features of the target object.

[0094] In some embodiments, the contour line features include contour line pixel coordinates and contour line pixel size, and the second processing module includes: a third processing sub-module, used to adjust the contour line pixel coordinates and / or contour line pixel size of the target object; a fourth processing sub-module, used to map the noise texture map to the screen space where the image to be rendered is located, and adjust the pixel coordinates corresponding to the noise texture map to displace, rotate and scale the noise texture map; and a fifth processing sub-module, used to perform a product operation on the contour line features and the noise texture map to generate simulated brush strokes with a flying white smudge effect.

[0095] In some embodiments, the third processing module includes: a sixth processing sub-module, used to map the texture map to the inner contour position of the target object in the initial rendered image; a seventh processing sub-module, used to adjust the pixel coordinates corresponding to the texture map to displace, rotate and scale the texture map; and an eighth processing sub-module, used to use lighting information as a mask to map the texture map to the shadow area in the initial rendered image.

[0096] In some embodiments, the device also includes a fifth processing module for scanning the drawn initial brushstroke texture to obtain an intermediate brushstroke texture; and for adjusting at least one of the following parameters of the intermediate brushstroke texture to obtain a brushstroke texture map: texture, size, resolution and color depth.

[0097] In some embodiments, the fourth processing module includes: a ninth processing sub-module, which is used to obtain the intrinsic color information of the target object from the diffuse reflection color buffer; a tenth processing sub-module, which is used to obtain the screen color information of the intermediate rendering image based on the screen space from the post-processing input buffer; an eleventh processing sub-module, which is used to calculate the lighting buffer result according to the screen color information and the intrinsic color information; and a twelfth processing sub-module, which is used to control the blending effect of the intermediate rendering image according to the lighting buffer result to obtain a target rendering image in ink style.

[0098] In some embodiments, the eleventh processing submodule includes: a first processing unit, configured to perform a division operation on the picture color information and the inherent color information to obtain the illumination intensity information in the intermediate rendering image as the illumination buffer result.

[0099] In some embodiments, the twelfth processing sub-module includes: a second processing unit, which is used to perform noise texture map mask control and blending effect intensity control on the intermediate rendering image according to the lighting buffer result, to obtain a target rendering image in ink style, and the blending effect intensity control includes mask intensity control and shadow intensity control.

[0100] In some embodiments, the device also includes a sixth processing module, which is used to provide a global control interface and a local control interface for image rendering control in response to a received image rendering instruction, the global control interface is used to edit the scene appearance parameters of the image rendering, and the local control interface is used to edit the line drawing and shading parameters, brushstroke texture parameters and color adjustment parameters of the image rendering.

[0101] In some embodiments, the sixth processing module includes a thirteenth processing sub-module, which is used to provide a material parameter setting node for image rendering control in response to a received parameter editing instruction, wherein the material parameter setting node includes at least one of a scalar parameter setting node, a vector parameter setting node, a texture parameter setting node, and a Boolean value setting node.

[0102] In some embodiments, the device also includes a seventh processing module for creating a dynamic material instance in response to a received image rendering instruction, the dynamic material instance including material parameters involved in the image rendering process, and the dynamic material instance supports setting the material parameters in a graphical interface.

[0103] It should be noted that the information collection, storage, use, processing, transmission, provision and disclosure involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0104] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0105] Figure 7 A block diagram of an electronic device for executing an ink-and-wash style image rendering method according to an embodiment of the present disclosure is schematically shown.

[0106] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 700 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0107] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0108] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0109] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the ink-style image rendering method. For example, in some embodiments, the ink-style image rendering method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the ink-style image rendering method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the ink-style image rendering method in any other appropriate manner (e.g., by means of firmware).

[0110] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable ink-style image rendering device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with an object, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the object; and a keyboard and pointing device (e.g., a mouse or trackball) through which the object can provide input to the computer. Other types of devices can also be used to provide interaction with an object; for example, the feedback provided to the object can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the object can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., an object computer having a graphical object interface or a web browser through which the object can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0115] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0116] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0117] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for rendering an ink-and-wash style image, characterized in that: The method comprises: Performing edge detection on a target object in the image to be rendered to obtain contour features of the target object; Outlining and shading the target object according to the contour line features and / or the preset noise texture map to obtain an initial rendered image; Based on the scene space position, mapping the texture map to the screen space where the initial rendered image is located to obtain an intermediate rendered image; and The intermediate rendering image is subjected to a blurring effect control based on the illumination calculation result to obtain a target rendering image in ink painting style.

2. The method according to claim 1, characterized in that The performing edge detection on the target object in the image to be rendered to obtain the contour line features of the target object comprises: Performing edge detection on the target object based on scene depth information to obtain an outer contour feature of the target object; and Performing edge detection on the target object based on the surface normal of the object to obtain the inner contour feature of the target object, The outer contour feature and the inner contour feature constitute the contour line feature of the target object.

3. The method according to claim 1, characterized in that The contour line features include contour line pixel coordinates and contour line pixel sizes. The target object is outlined and shaded according to the contour line features and / or a preset noise texture map to obtain an initial rendered image, including: At least one of the following methods is used to outline and shade the target object to obtain the initial rendered image: Adjusting the contour line pixel coordinates and / or the contour line pixel size of the target object; Mapping the noise texture map to the screen space where the image to be rendered is located, and adjusting the pixel coordinates corresponding to the noise texture map to displace, rotate and scale the noise texture map; and A product operation is performed on the contour line feature and the noise texture map to generate a simulated brushstroke with a flying white smudge effect.

4. The method according to claim 1, characterized in that: The step of mapping the texture map to the screen space where the initial rendered image is located based on the scene space position to obtain an intermediate rendered image comprises: At least one of the following methods is used to map the texture map to the screen space where the initial rendered image is located to obtain the intermediate rendered image: Mapping the texture map to the inner contour position of the target object in the initial rendered image; Adjusting pixel coordinates corresponding to the texture map to displace, rotate and scale the texture map; and The texture map is mapped to the shadow area in the initial rendered image using the lighting information as a mask.

5. The method according to claim 4, characterized in that The method further comprises: Scanning the drawn initial texture to obtain an intermediate texture; and At least one of the following parameters of the intermediate texture is adjusted to obtain the texture map: texture, size, resolution and color depth.

6. The method according to claim 1, characterized in that The step of controlling the blurring effect on the intermediate rendered image based on the illumination calculation result to obtain a target rendered image in ink painting style includes: Acquire the intrinsic color information of the target object from the diffuse reflection color buffer; Acquire picture color information of the intermediate rendering image based on the screen space from a post-processing input buffer; Calculating a lighting buffer result according to the picture color information and the inherent color information; According to the illumination buffer result, the intermediate rendering image is controlled for a blurring effect to obtain the target rendering image in ink painting style.

7. The method according to claim 6, characterized in that The calculating the illumination buffer result according to the picture color information and the inherent color information includes: The picture color information is divided by the inherent color information to obtain the light intensity information in the intermediate rendering image as the light buffer result.

8. The method according to claim 6, characterized in that The step of controlling the blurring effect on the intermediate rendering image according to the illumination buffer result to obtain the target rendering image in ink painting style includes: According to the illumination buffer result, the noise texture map mask control and the smudge effect intensity control are performed on the intermediate rendering image to obtain the target rendering image in ink painting style. The blending effect intensity control includes mask intensity control and shadow intensity control.

9. The method according to claim 1, characterized in that: The method further comprises: In response to the received image rendering instruction, a global control interface and a local control interface for image rendering control are provided, The global control interface is used to edit the scene appearance parameters of the image rendering, and the local control interface is used to edit the line drawing and shading parameters, brushstroke texture parameters and color adjustment parameters of the image rendering.

10. The method according to claim 9, characterized in that The method further comprises: In response to the received parameter editing instruction, a material parameter setting node for image rendering control is provided, wherein the material parameter setting node includes at least one of a scalar parameter setting node, a vector parameter setting node, a texture parameter setting node, and a Boolean value setting node.

11. The method according to claim 10, characterized in that The method further comprises: In response to the received image rendering instruction, creating a dynamic material instance, The dynamic material instance includes material parameters involved in the image rendering process, and the dynamic material instance supports setting the material parameters in a graphical interface.

12. An ink-and-wash style image rendering device, characterized in that: The device comprises: A first processing module is used to perform edge detection on a target object in the image to be rendered to obtain contour features of the target object; A second processing module is used to outline and smudge the target object according to the contour line features and / or the preset noise texture map to obtain an initial rendered image; A third processing module is used to map the texture map to the screen space where the initial rendering image is located based on the scene space position to obtain an intermediate rendering image; and The fourth processing module is used to control the blurring effect of the intermediate rendering image based on the illumination calculation result to obtain a target rendering image in ink painting style.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image rendering method described in any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the image rendering method according to any one of claims 1 to 11.

Images