Image rendering method, system, device and medium

Abstract

The invention discloses an image rendering method, system and device and a medium. The method comprises the steps of obtaining an original oil painting image of an oil painting; determining a normal map corresponding to the original oil painting image; and performing real-time illumination rendering based on the original oil painting image and the normal map to obtain a target oil painting image. The problem that in the prior art, color and texture between the shot and displayed oil painting image and the original painting are different, so that the appreciation experience of a user is poor is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image rendering method, system, device and medium. Background Art

[0002] As living standards improve, people are increasingly focused on their artistic cultivation, and the demand for oil painting appreciation is also increasing. Oil paintings are unique, and in order to appreciate them, they are often photographed to obtain images, which are then exhibited in different locations to meet the appreciation needs of different cities.

[0003] However, due to hardware defects in the shooting equipment, there are differences in color and texture between the photographed and exhibited oil painting images and the original paintings, resulting in a poor user appreciation experience. Summary of the Invention

[0004] In order to overcome the problem in the prior art that there are differences in color and texture between the photographed and exhibited oil painting images and the original paintings, resulting in a poor user appreciation experience, the present application provides an image rendering method, system, device and medium.

[0005] In a first aspect, in order to solve the above technical problems, the present application provides an image rendering method, comprising:

[0006] Obtaining the original oil painting image of the oil painting;

[0007] Determine the normal map corresponding to the original oil painting image;

[0008] Real-time lighting rendering is performed based on the original oil painting image and the normal map to obtain the target oil painting image.

[0009] Furthermore, the normal map corresponding to the original oil painting image is determined, including:

[0010] Convert the original oil painting image into a grayscale image;

[0011] Extract the brightness gradient of the oil painting pixels in the grayscale image to obtain the normal vector of the oil painting pixels;

[0012] A normal map corresponding to the original oil painting image is formed based on multiple normal vectors.

[0013] Furthermore, the brightness gradient of the oil painting pixels in the grayscale image is extracted to obtain the normal vector of the oil painting pixels, including:

[0014] The preset edge detection algorithm is used to extract the horizontal brightness change rate of the oil painting pixels in the grayscale image and the vertical brightness change rate of the oil painting pixels in the vertical direction;

[0015] Determining the maximum horizontal brightness change rate among the multiple horizontal brightness change rates as the horizontal brightness gradient of the oil painting pixel, and determining the maximum vertical brightness change rate among the multiple vertical brightness change rates as the vertical brightness gradient of the oil painting pixel;

[0016] The horizontal brightness gradient is used as the x-axis component and the vertical brightness gradient is used as the y-axis component to form the normal vector of the oil painting pixel.

[0017] Furthermore, real-time lighting rendering is performed based on the original oil painting image and the normal map to obtain the target oil painting image, including:

[0018] Performing illumination processing on the original oil painting image based on preset light source parameters and normal map to obtain diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image;

[0019] Real-time acquisition of ambient light parameters of the environment where the original oil painting image is located;

[0020] Based on the ambient light parameters, diffuse reflection parameters and specular reflection parameters, the target rendering color parameters of the oil painting pixels are calculated;

[0021] The oil painting pixels are rendered based on the target rendering color parameters to obtain target rendering pixels, and a target oil painting image is formed based on the plurality of target rendering pixels.

[0022] Furthermore, the light source parameters include the light source color and the relative position parameters between the light source and the oil painting, and the map pixels in the normal map include the normal vector;

[0023] The original oil painting image is illuminated based on the preset light source parameters and normal map to obtain the diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image, including:

[0024] A spatial coordinate system is established based on the position of the oil painting to obtain the pixel coordinates of the oil painting pixels in the original oil painting image, where the positions of the original oil painting image and the normal map coincide;

[0025] Based on the pixel coordinates and the normal vector of the corresponding texture pixel, the normal parameters of the oil painting pixel are calculated;

[0026] Determine the light direction of the painting pixels based on relative positions and pixel coordinates;

[0027] Based on the light direction, normal parameters and light source color, the diffuse reflection parameters and specular reflection parameters of the oil painting pixels are calculated.

[0028] Furthermore, based on the relative position and pixel coordinates, the light direction of the oil painting pixel is determined, including:

[0029] Based on the relative position, the light source position coordinates of the light source in the spatial coordinate system are determined;

[0030] Based on the light source position coordinates and pixel coordinates, the light direction of the oil painting pixel is calculated.

[0031] Furthermore, the target oil painting image is displayed on a display screen, and the method further includes:

[0032] Calculate the average pixel brightness of the target oil painting image;

[0033] Mapping the average pixel brightness to PWM duty cycle;

[0034] The backlight brightness of the display screen is adjusted based on the PWM duty cycle so that the brightness of the target oil painting image matches the brightness of the environment in which the target oil painting image is located.

[0035] In a second aspect, the present application further provides an image rendering system, comprising:

[0036] An acquisition module, used for acquiring the original oil painting image of the oil painting;

[0037] A determination module, used to determine the normal map corresponding to the original oil painting image;

[0038] The lighting rendering module is used to perform real-time lighting rendering based on the original oil painting image and the normal map to obtain the target oil painting image.

[0039] In a third aspect, the present application also provides a computing device, comprising a memory, a processor, and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of an image rendering method as described above are implemented.

[0040] In a fourth aspect, the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the steps of an image rendering method.

[0041] The beneficial effects of the present application are as follows: first, by determining the normal map corresponding to the original oil painting image of the oil painting, the lighting stereo effect of the oil painting surface can be digitally displayed on the normal map. Since the lighting stereo effect can not only show high color contrast, but also contain rich micro-concave and convex detail features, real-time lighting rendering based on the original oil painting image and the normal map can render and fuse the lighting stereo effect of the oil painting with the original oil painting image, so that the high color contrast and rich micro-concave and convex detail features of the oil painting surface can be integrated into the rendered target oil painting image, which can not only improve the color contrast of the target oil painting image, but also enhance the micro-concave and convex detail features of the target oil painting image, thereby reducing the color and texture differences between the target oil painting image and the oil painting, improving the authenticity of the target oil painting image, and further improving the user's appreciation experience of the target oil painting. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of an image rendering method according to an exemplary embodiment of the present application;

[0043] Figure 2 The figure is a schematic structural diagram of an image rendering system according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0044] The following examples are provided to further explain and supplement the present application and do not constitute any limitation to the present application.

[0045] Currently, the technologies available on the market for oil painting effect display mainly include:

[0046] Hardware optical components: by adding diffuser film and filter plate to achieve the desired effect, the cost is high and the effect is fixed;

[0047] Based on 3D measurement model: Requires multi-angle surface measurement of the physical object, complex equipment, high cost, and not suitable for flat images;

[0048] Simple software effects: Post-processing such as filters is used, but the effect lacks lighting interaction and the texture is unrealistic.

[0049] However, the above technologies have the following problems: they cannot render realistic oil painting textures on ordinary displays at low cost and in real time; existing technologies are resource-intensive and unsuitable for implementation on embedded platforms.

[0050] In order to solve the above problems, embodiments of the present application provide an image rendering method, system, device and medium, which will be described in detail below.

[0051] An image rendering method provided in an embodiment of the present application can be specifically executed by a server. It should be noted that the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and is not limited here.

[0052] See also Figure 1 , Figure 1 An image rendering method is shown as an exemplary embodiment of the present application. Figure 1 As shown, the present application provides an image rendering method, comprising:

[0053] S11, obtaining an original oil painting image;

[0054] S12, determining a normal map corresponding to the original oil painting image;

[0055] S13, performing real-time lighting rendering based on the original oil painting image and the normal map to obtain a target oil painting image.

[0056] The image rendering method of this embodiment provided by the present application first determines the normal map corresponding to the original oil painting image of the oil painting, and can digitally display the lighting stereo effect of the oil painting surface on the normal map. Since the lighting stereo effect can not only show high color contrast but also contain rich micro-concave and convex detail features, real-time lighting rendering based on the original oil painting image and the normal map can render and fuse the lighting stereo effect of the oil painting with the original oil painting image, so that the high color contrast and rich micro-concave and convex detail features of the oil painting surface can be integrated into the rendered target oil painting image, which not only improves the color contrast of the target oil painting image, but also enhances the micro-concave and convex detail features of the target oil painting image, thereby reducing the color and texture differences between the target oil painting image and the oil painting, improving the realism of the target oil painting image, and further enhancing the user's appreciation experience of the target oil painting.

[0057] Optionally, determining a normal map corresponding to the original oil painting image includes:

[0058] Convert the original oil painting image into a grayscale image;

[0059] Extract the brightness gradient of the oil painting pixels in the grayscale image to obtain the normal vector of the oil painting pixels;

[0060] A normal map corresponding to the original oil painting image is formed based on multiple normal vectors.

[0061] In the embodiment provided by the present application, brightness gradient extraction is performed on the oil painting pixels in the grayscale image converted from the original oil painting image, so that the color contrast and micro-concave and convex detail features in the oil painting pixels can be extracted, and the normal vector of the oil painting pixels can be obtained, so that the normal map formed by multiple normal vectors can not only show the high color contrast of the oil painting surface, but also contain rich micro-concave and convex detail features, which is convenient for improving the color contrast of the target oil painting image obtained by subsequent normal map rendering and fusion, and improving its micro-concave and convex detail features, thereby improving the user's appreciation experience of the target oil painting.

[0062] Optionally, brightness gradient extraction is performed on the oil painting pixels in the grayscale image to obtain the normal vectors of the oil painting pixels, including:

[0063] The preset edge detection algorithm is used to extract the horizontal brightness change rate of the oil painting pixels in the grayscale image and the vertical brightness change rate of the oil painting pixels in the vertical direction;

[0064] Determining the maximum horizontal brightness change rate among the multiple horizontal brightness change rates as the horizontal brightness gradient of the oil painting pixel, and determining the maximum vertical brightness change rate among the multiple vertical brightness change rates as the vertical brightness gradient of the oil painting pixel;

[0065] The horizontal brightness gradient is used as the x-axis component and the vertical brightness gradient is used as the y-axis component to form the normal vector of the oil painting pixel.

[0066] In the embodiment provided herein, a preset edge detection algorithm is first used to extract the horizontal brightness change rate of the painting pixels in the grayscale image, as well as the vertical brightness change rate of the painting pixels in the vertical direction, to digitally quantify the color contrast and rich microscopic concave-convex detail features in the painting. Then, because the horizontal gradient direction is the same as the direction of the maximum horizontal brightness change rate, and the vertical gradient direction is the same as the direction of the maximum vertical brightness change rate, the maximum horizontal brightness change rate is determined as the horizontal brightness gradient of the painting pixel, which is used as the x-axis component, and the maximum vertical brightness change rate is determined as the vertical brightness gradient of the painting pixel, which is used as the y-axis component to form the normal vector of the painting pixel. In this way, the most representative color contrast and microscopic concave-convex detail features can be digitally represented in the form of a normal vector, facilitating the subsequent digital lighting rendering of the normal map formed based on multiple normal vectors, reducing the rendering difficulty and thereby improving the rendering efficiency of the target painting image. The edge detection algorithm can be a Sobel operator. The z-axis component of the normal vector is a set constant.

[0067] In an exemplary embodiment provided herein, the process of obtaining a normal map can be considered as a preprocessing process for the original oil painting image. The steps of this preprocessing process can be implemented by code development using the OpenCV (Open Source Computer Vision Library) development tool. The specific steps of the preprocessing process can be as follows:

[0068] 1. Original Image Acquisition: The user captures an original oil painting image (RGB image) using a camera or uploads it from the gallery. The original oil painting image is loaded into memory as a three-channel matrix for storage and subsequent use. For example, the original oil painting image might be "Sunflowers," with an image size of 1920×1080 and a JPG or PNG format.

[0069] 2. Grayscale Conversion: Use the cv2.cvtColor function in OpenCV to convert the original painting image to grayscale. The corresponding code is: gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY). This function extracts the brightness information from the original painting image and preserves the light and dark variations. This is a prerequisite for texture extraction and facilitates subsequent edge detection and structural feature analysis.

[0070] 3. Sobel Derivation: The Sobel operator calculates the horizontal (X) and vertical (Y) brightness gradients of the painting pixels in the grayscale image. The gradients are then intensity-corrected. The maximum horizontal brightness gradient is determined as the horizontal brightness gradient of the painting pixel, and the maximum vertical brightness gradient is determined as the vertical brightness gradient of the painting pixel. This function digitizes the surface "concavity" of the painting pixels into horizontal and vertical brightness gradients, facilitating subsequent conversion into normal vectors.

[0071] 4. Generate a normal map: Based on the horizontal brightness gradient in the x-direction, the vertical brightness gradient in the y-direction, and a set constant in the z-direction, a normal vector is generated for each painting pixel, and the normal map is output. For example, a normal vector can be expressed as N(x,y,z), where x = -Gx, y = -Gy, and z = 1. Each normal vector is normalized to generate a standard normal map with a unified format. Its purpose is to simulate the undulations of the painting surface through the normal vectors, making the lighting produce realistic changes.

[0072] 5. Add blur: Use the cv2.GaussianBlur function in OpenCV to blur the normal map, which can simulate the transition of brush strokes, make the transition between adjacent oil painting pixels smooth, eliminate the sharp noise in the normal map, and enhance the natural feeling of subsequent rendering.

[0073] 6. Image output: The original oil painting image and normal map are saved as OpenGL (Open Graphics Library) texture files respectively, which are then used for real-time lighting rendering based on the original oil painting image and normal map in the GPU to obtain the target oil painting image.

[0074] Optionally, performing real-time lighting rendering based on the original oil painting image and the normal map to obtain a target oil painting image includes:

[0075] Performing illumination processing on the original oil painting image based on preset light source parameters and normal map to obtain diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image;

[0076] Real-time acquisition of ambient light parameters of the environment where the original oil painting image is located;

[0077] Based on the ambient light parameters, diffuse reflection parameters and specular reflection parameters, the target rendering color parameters of the oil painting pixels are calculated;

[0078] The target rendering color parameters are calculated as follows:

[0079] vec3 ambient=ambientStrength*lightColor*albedo;

[0080] vec3 color=ambient+diffuse+specular;

[0081] Where ambient represents the ambient light parameter, ambientStrength represents the ambient light coefficient (float) with a value range of (0.05-0.2), lightColor represents the light source color or light source intensity (vec3), albedo represents the surface base color of the original oil painting image (vec3), color represents the target rendering color parameter, diffuse represents the diffuse reflection parameter, and specular represents the specular reflection parameter;

[0082] The oil painting pixels are rendered based on the target rendering color parameters to obtain target rendering pixels, and a target oil painting image is formed based on the plurality of target rendering pixels.

[0083] In the embodiment provided herein, an original oil painting image is first illuminated based on preset light source parameters and a normal map to obtain diffuse and specular reflection parameters for the oil painting pixels in the original oil painting image. Target rendering color parameters for the oil painting pixels are then calculated based on the ambient light parameters, diffuse and specular reflection parameters of the environment in which the original oil painting image resides. The target rendering color parameters for the oil painting pixels are then rendered to simulate the color and texture exhibited by the oil painting under illumination corresponding to the light source parameters, thereby obtaining target rendered pixels. A target oil painting image is then formed based on the multiple target rendered pixels containing color and texture. This target oil painting image not only exhibits high color contrast but also contains rich microscopic concave and convex detail features, thereby reducing the color and texture differences between the target oil painting image and the painting itself, improving the realism of the target oil painting image, and thereby enhancing the user's appreciation experience of the target oil painting. The ambient light parameters are acquired in real time every 16 milliseconds, meaning the target oil painting image is refreshed every 16 milliseconds to achieve a dynamic effect of lighting and viewing angle linkage.

[0084] In an exemplary embodiment provided by the present application, light source parameters are set, and the original oil painting image is illuminated based on the preset light source parameters and normal map to obtain the diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image. The ambient light parameters, diffuse reflection parameters, and specular reflection parameters of the environment in which the original oil painting image is located are integrated into the target rendering color parameters of the oil painting pixels, and the target rendering color parameters are rendered to the corresponding oil painting pixels of the original oil painting image using the GLSL (fragment shader) Shader (shader program), realizing dynamic lighting interaction of the picture, simulating the light exposure of the rough surface of the canvas and the reflection of the oil paint on the oil painting, presenting a realistic thick paint texture, obtaining the target oil painting image, and realizing real-time lighting rendering and synthesis of the original oil painting image and the normal map.

[0085] Optionally, the light source parameters include light source color, relative position parameters between the light source illuminating the oil painting and the oil painting, and the map pixels in the normal map include normal vectors;

[0086] The original oil painting image is illuminated based on the preset light source parameters and normal map to obtain the diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image, including:

[0087] A spatial coordinate system is established based on the position of the oil painting to obtain the pixel coordinates of the oil painting pixels in the original oil painting image, where the positions of the original oil painting image and the normal map coincide;

[0088] Based on the pixel coordinates and the normal vector of the corresponding texture pixel, the normal parameters of the oil painting pixel are calculated;

[0089] The calculation formula of normal parameters is as follows:

[0090] vec3 normal=normalize(texture(normalMap,texCoord).rgb*2.0-1.0);

[0091] Among them, vec3 represents the three-dimensional vector format, vec3 normal represents the normal parameter in the form of a three-dimensional vector, normalize() represents unitization, which is used to obtain a normal vector with a length of 1, texture represents texture, normalMap represents the tangent space XYZ normal vector collected by the sampler (sampler2D) of the normal map, encoded as [0,1], texCoord represents the 2D texture pixel coordinates (vec2) of the current fragment, which is interpolated from the vertex shader, and *2.0-1.0 represents converting the encoding [0,1] of the normal vector to [-1,1] to ensure that the true vector component is obtained; in addition, if the normal map is in tangent space, the TBN matrix needs to be used to transform the normal parameters to world / view space;

[0092] Determine the light direction of the painting pixels based on relative positions and pixel coordinates;

[0093] Based on the light direction, normal parameters and light source color, the diffuse reflection parameters and specular reflection parameters of the oil painting pixels are calculated;

[0094] The calculation formula of diffuse reflection parameters is as follows:

[0095] float diff=max(dot(normal,lightDir),0.0);

[0096] vec3 diffuse=diff *lightColor*albedo;

[0097] Where float diff represents the diffuse reflection coefficient in real form (float), diff>=0, dot(normal,lightDir) represents the cosine of the angle between the normal parameter and the light direction, vec3 diffuse represents the diffuse reflection parameter in three-dimensional vector form, lightColor represents the light source color or light source intensity (vec3), and albedo represents the surface base color of the original oil painting image (vec3).

[0098] The calculation formula for specular reflection parameters is as follows:

[0099] vec3 viewDir = normalize(viewPos - fragPos); / / view direction

[0100] vec3 reflectDir = reflect(-lightDir,normal); / / Reflection direction

[0101] float specAngle=max(dot(viewDir,reflectDir),0.0);

[0102] float spec=pow(specAngle,shininess);

[0103] vec3 specular=specularStrength*spec*lightColor;

[0104] Among them, viewDir represents the observer's line of sight, viewPos represents the camera / observer position (vec3), which is the world space coordinate, fragPos represents the position of the fragment shader, reflectDir represents the reflection direction vector, reflect() represents the GLSL / HLSL built-in function, specAngle represents the observer direction, shininess represents the highlight exponent (float), the value range is (8-256), the larger the value, the sharper, spec represents the camera's specular reflection vector, pow represents the power operation, specularStrength represents the highlight intensity coefficient (float), the value range is (0-1), and specular represents the highlight reflection parameter.

[0105] In the embodiment provided by this application, a spatial coordinate system is established based on the position of the oil painting, which can map the position of the original oil painting image in the spatial coordinate system, thereby directly reading the pixel coordinates of the oil painting pixels in the original oil painting image. The position of the original oil painting image and the normal map overlap, indicating that the pixel coordinates of the map pixel in the normal map are the same as the corresponding oil painting pixel in the original oil painting image. Based on the pixel coordinates and the normal vector of the corresponding map pixel, the normal parameters of the oil painting pixel are calculated, and the light direction of the oil painting pixel is determined based on the relative position and pixel coordinates. Then, based on the light direction, normal parameters, and light source color, the diffuse reflection parameters and specular reflection parameters of the oil painting pixel are calculated, facilitating the subsequent determination of the target rendering color parameters of the oil painting pixel based on the diffuse reflection parameters and specular reflection parameters and rendering. The color and texture exhibited by the oil painting under the light source corresponding to the light source parameters are rendered into the oil painting pixels, resulting in a target oil painting image with color contrast and microscopic concave and convex detail features, which can improve the realism of the target oil painting image and thus enhance the user's appreciation experience of the target oil painting.

[0106] In an exemplary embodiment provided in the present application, first, as an alternative solution, the light source intensity can be used instead of the light source color to participate in the calculation of the diffuse reflection parameters and the specular reflection parameters of the oil painting pixels.

[0107] Secondly, in the calculation formula of the specular reflection parameter, the calculation formula of the camera's mirror reflection vector spec can also be a Blinn-Phong variant, and its calculation formula is as follows:

[0108] vec3 halfDir=normalize(lightDir+viewDir);

[0109] float spec=pow(max(dot(normal,halfDir),0.0),shininess);

[0110] Among them, halfDir represents the half-angle vector.

[0111] Optionally, determining the light direction of the oil painting pixel based on the relative position and the pixel coordinates includes:

[0112] Based on the relative position, the light source position coordinates of the light source in the spatial coordinate system are determined;

[0113] Based on the light source position coordinates and pixel coordinates, the light direction of the oil painting pixel is calculated;

[0114] The formula for calculating the direction of light is as follows:

[0115] vec3 lightDir=normalize(lightPos-fragPos);

[0116] Among them, vec3 represents the three-dimensional vector format, vec3 lightDir represents the light direction of the three-dimensional vector, which is a unit vector pointing to the light source and is used for point light / spot light. If the light source is a parallel light, a fixed direction is used. normalize() represents normalization and is used to obtain a normal vector with a length of 1. lightPos represents the light source position coordinates (vec3) in world space coordinates. fragPos represents the current fragment position (vec3) in world space coordinates, which is interpolated by the vertex shader.

[0117] In the embodiment provided by the present application, the light direction of the oil painting pixel is calculated based on the light source position coordinates and pixel coordinates of the light source determined by relative position in the spatial coordinate system, which facilitates the subsequent determination of the target rendering color parameters of the oil painting pixel based on the light direction and rendering, so as to improve the color contrast and texture contrast of the rendered oil painting pixels, thereby further improving the realism of the target oil painting image, and further improving the user's appreciation experience of the target oil painting.

[0118] In an exemplary embodiment provided by the present application, the process steps of real-time lighting rendering can be implemented through code development using OpenGLES 3.0, which is deployed on the RK3288 Android (Rockchip's high-performance processor, used for development and application of the Android platform) platform. OpenGL ES 3.0 has a graphics pipeline with programmable shading capabilities. The graphics pipeline includes a fragment shader (FS, also known as GLSL) and a vertex shader (VS). The vertex shader is used to process the data of each vertex, while the fragment shader processes the data of each pixel, thereby achieving complex graphics effects. OpenGL ES 3.0 is a 3D graphics API designed for embedded systems, especially handheld and mobile devices such as smartphones and tablets.

[0119] The specific steps of the real-time lighting rendering process are as follows:

[0120] 1. Texture input:

[0121] The original oil painting image is loaded into the GPU shader as a diffuse map (a diffuse map used to define the basic color and diffuse reflection characteristics of the object surface. The diffuse reflection characteristics refer to the color of the material under uniform lighting), and the processed normal map is loaded into the GPU shader as a normal map (a normal map used to simulate microscopic concave and convex details such as scratches and brick joints by changing the normal direction of the image surface to enhance the three-dimensional effect of lighting).

[0122] 2. Lighting calculation: The shader includes a vertex shader and a fragment shader. First, in the vertex shader, the vertex coordinates, UV texture coordinates, light source position, viewing direction, etc. of the original oil painting image are passed in; in the GLSL fragment shader, the original oil painting image is illuminated based on the preset light source parameters and normal map, and the diffuse reflection parameters and specular reflection parameters are calculated. Among them, the diffuse reflection parameter (diff) is expressed as: dot(normal, lightDir), which is used to reflect the degree of light received by the surface; the specular reflection parameter (spec) is used to reflect the specular light reflected by the raised part. Then, based on the ambient light parameters, diffuse reflection parameters, and specular reflection parameters of the environment in which the original oil painting image is located, the target rendering color parameters of the oil painting pixels are calculated to achieve lighting superposition. The final color of the oil painting pixel is the product of the original image color and the light intensity.

[0123] The code snippets corresponding to the steps performed in the shader are as follows:

[0124] vec3 normal = normalize(texture(normalMap, texCoord).rgb * 2.0 - 1.0);

[0125] vec3 lightPos = vec3(0.5, 0.5, 1.0);

[0126] vec3 lightColor = vec3(1.0, 1.0, 1.0);

[0127] vec3 viewPos = vec3(0.5, 0.5, 2.0);

[0128] vec3 fragPos = vec3(texCoord, 0.0);

[0129] vec3 lightDir = normalize(lightPos - fragPos);

[0130] vec3 viewDir = normalize(viewPos - fragPos);

[0131] / / Diffuse

[0132] float diff = max(dot(normal, lightDir), 0.0);

[0133] vec3 diffuse = diff * lightColor;

[0134] / / Specular

[0135] vec3 reflectDir = reflect(-lightDir, normal);

[0136] float spec = pow(max(dot(viewDir, reflectDir), 0.0), 32.0);

[0137] vec3 specular = spec * vec3(1.0);

[0138] / / Combine final color

[0139] vec3 color = (diffuse + specular) * texture(diffuseMap, texCoord).rgb; fragColor = vec4(color, 1.0);

[0140] The description of the parameters and variables in the above code is shown in Table 1:

[0141] Table 1

[0142]

[0143]

[0144] 3. Rendering results: Render the oil painting pixels based on the target rendering color parameters to simulate the three-dimensional lighting effect of the oil painting, obtain the target rendering pixels, and form the target oil painting image based on multiple target rendering pixels. This makes the real-time target oil painting image have a three-dimensional sense and realism, enhancing the immersive experience when appreciating the oil painting.

[0145] The following points should be noted when rendering real-time lighting:

[0146] ①Coordinate space consistency: Make sure normal, lightDir, and viewDir are all in world space or observation space.

[0147] ② If the normal map is in tangent space, it needs to be converted to the corresponding space using the TBN matrix.

[0148] ③Shininess (the value is 32.0 in this implementation) determines the sharpness of highlights and can be adjusted according to the material characteristics.

[0149] ④ If you need to add ambient light, you can additionally calculate ambient = ambientStrength * lightColor * texture(diffuseMap,texCoord).rgb and add it to the synthesis.

[0150] In another exemplary embodiment provided herein, a lighting model performs real-time lighting rendering based on an original oil painting image and a normal map to produce a target oil painting image. The three elements of the lighting model include a single-point light source module, a diffuse reflection module, and a specular reflection module. The single-point light source module is formed by setting light source parameters. Table 2 shows the applicable scenarios for the three elements of the lighting model.

[0151] Table 2

[0152]

[0153]

[0154] Without a single point light source, the image lacks a sense of direction; without diffuse reflection, the image becomes flat; and without highlights, the three-dimensional and realistic oil paint reflection texture is lost. Therefore, when using a lighting model for rendering, the lighting model includes a single point light source module: similar to a spotlight, it brightens one side of the image and darkens the other, enhancing the sense of direction and realism; a diffuse reflection module: it displays the roughness of the canvas after being illuminated, emphasizing the bumps and depressions of the brushstrokes; and a highlight reflection module: it displays areas of bright paint accumulation (such as white highlights and reflective pigments), with the highlights changing with viewing angle. Therefore, all three modules are indispensable and together form the core of dynamic lighting simulation for oil paintings.

[0155] Optionally, the target oil painting image is displayed on a display screen, and the method further includes:

[0156] Calculate the average pixel brightness of the target oil painting image;

[0157] Mapping the average pixel brightness to PWM duty cycle;

[0158] The backlight brightness of the display screen is adjusted based on the PWM duty cycle to match the brightness of the target oil painting image with the brightness of the environment in which the target oil painting image is located.

[0159] In the embodiment provided by the present application, the calculated average pixel brightness of the target oil painting image is mapped to a PWM duty cycle, and the backlight brightness of the display screen is adjusted based on the PWM duty cycle to match the brightness of the target oil painting image with the brightness of the environment in which the target oil painting image is located. This can improve the visual comfort of the user when viewing the target oil painting image, thereby further improving the user's appreciation experience of the target oil painting.

[0160] In this application, when calculating the average pixel brightness of the target painting image, the pixel colors of the target painting image are weighted and converted to grayscale values ​​using fragColor (the output variable of the fragment shader). All grayscale values ​​are averaged to obtain the average pixel brightness of the target painting image. The PWM duty cycle value range is (0-255).

[0161] In one exemplary embodiment provided herein, the backlight adjustment operating environment utilizes an RK3288 I2C controller and a TILP8556 (LED backlight driver chip) backlight driver IC. The RK3288 is a high-performance processor from Rockchip, widely used in various smart devices and development boards. I2C (Inter-Integrated Circuit) is a two-wire serial bus used to connect microcontrollers and their peripherals.

[0162] The target painting image is displayed on a display, which can be an LCD (liquid crystal display). By analyzing the average brightness of the target painting image in the current frame, the PWM duty cycle of the LCD backlight brightness is dynamically adjusted. This is achieved by automatically brightening or dimming the display's backlight brightness based on the PWM duty cycle mapped to the average pixel brightness of the target painting image, enhancing immersion and saving energy to match the user's visual experience and improve their appreciation of the target painting. The PWM duty cycle is written to the PWM control register of the TI LP8556 via I2C. I2C communication manages the high and low levels of the backlight PWM, enabling real-time synchronous adjustment.

[0163] Backlight brightness significantly impacts the display image, including brightness, color rendering, visual comfort, power consumption, and visual quality. By sending the PWM duty cycle in the PWM control register to the backlight driver IC via I2C, the display backlight brightness is adjusted based on the PWM duty cycle. This synchronizes the screen image and backlight brightness, enhancing visual expression and energy savings. This improves image quality and reduces eye fatigue, ensuring that the target oil painting image matches the brightness of its surroundings.

[0164] In an exemplary embodiment provided herein, a visual adjustment interface is provided on the display screen, allowing users to customize light source parameters and backlight response. The visual adjustment interface includes a light source angle adjustment control, a light intensity adjustment control, and a backlight response adjustment control. The light source angle adjustment control is used to modify the light source position vector via a slider control; the light intensity adjustment control is used to control a variable to change the light intensity; and the backlight response adjustment control is used to enable or disable PWM synchronization or adjust the sensitivity coefficient.

[0165] In an exemplary embodiment provided in this application, the main important parameters used include the Sobel operator, illumination model, backlight PWM and rendering resolution, and the parameter definitions are shown in Table 3.

[0166] Table 3

[0167]

[0168] The image rendering method of this application can achieve the following effects: ① Authentic effects, reproducing the heavy, perforated effect of oil paintings: By dynamically overlaying normal maps with the GLSL lighting model (single-point light source + diffuse reflection + highlights), the thick paint, highlights, and brushstrokes realistically change under different lighting angles, simulating the volume and texture of oil paintings. ② Low power consumption, compatible with embedded platforms: Image preprocessing is performed only during initialization; OpenGL ES 3.0 can run efficiently on low-power chips such as the RK3288; backlight PWM control avoids full-brightness driving, achieving energy-saving display. ③ Maintaining original image color without destroying details: Rendering uses a "non-destructive" overlay method that multiplies the light with the original image color, preserving the original RGB image color. All lighting effects are achieved through layer-by-layer overlay, without changing the pixel data itself, preserving details. ④ No physical filter materials are required, reducing costs: The overall solution simulates thick paint effects and spatial reflections through software, eliminating the need for traditional hardware such as diffuser film, AG film, or textured glass. This significantly reduces display material and assembly costs and is compatible with existing LCD modules of OEM manufacturers. ⑤ Adaptable to various screens and lighting environments: The virtual light source position and intensity can be dynamically adjusted according to the screen type and display scene; the backlight brightness is automatically adjusted according to real-time feedback from the picture, adapting to various environments such as indoor / outdoor / strong light / low light; providing good cross-device adaptability (10.15-inch to 110-inch LCD screen)

[0169] See also Figure 2 , Figure 2 An image rendering system is shown as an exemplary embodiment of the present application. Figure 2 As shown, the present application provides an image rendering system 200, comprising:

[0170] An acquisition module 201 is used to acquire an original oil painting image;

[0171] Determination module 202, for determining a normal map corresponding to the original oil painting image;

[0172] The lighting rendering module 203 is used to perform real-time lighting rendering based on the original oil painting image and the normal map to obtain a target oil painting image.

[0173] The image rendering system of this embodiment provided by the present application first determines, through the determination module 202, the normal map corresponding to the original oil painting image obtained by the acquisition module 201, and can digitally display the lighting stereo effect of the oil painting surface on the normal map. Because the lighting stereo effect can not only display high color contrast but also contain rich micro-concave and convex details, the lighting rendering module 203 performs real-time lighting rendering based on the original oil painting image and the normal map, and can render and fuse the lighting stereo effect of the oil painting with the original oil painting image. The high color contrast and rich micro-concave and convex details of the oil painting surface can be incorporated into the rendered target oil painting image, thereby improving the color contrast of the target oil painting image and enhancing the micro-concave and convex details of the target oil painting image. This can reduce the color and texture differences between the target oil painting image and the oil painting, improve the realism of the target oil painting image, and thus enhance the user's appreciation experience of the target oil painting.

[0174] Optionally, the determination module 202 is specifically configured to:

[0175] Convert the original oil painting image into a grayscale image;

[0176] Extract the brightness gradient of the oil painting pixels in the grayscale image to obtain the normal vector of the oil painting pixels;

[0177] A normal map corresponding to the original oil painting image is formed based on multiple normal vectors.

[0178] Optionally, the determination module 202 is specifically configured to:

[0179] The preset edge detection algorithm is used to extract the horizontal brightness change rate of the oil painting pixels in the grayscale image and the vertical brightness change rate of the oil painting pixels in the vertical direction;

[0180] Determining the maximum horizontal brightness change rate among the multiple horizontal brightness change rates as the horizontal brightness gradient of the oil painting pixel, and determining the maximum vertical brightness change rate among the multiple vertical brightness change rates as the vertical brightness gradient of the oil painting pixel;

[0181] The horizontal brightness gradient is used as the x-axis component and the vertical brightness gradient is used as the y-axis component to form the normal vector of the oil painting pixel.

[0182] Optionally, the lighting rendering module 203 is specifically configured to:

[0183] Performing illumination processing on the original oil painting image based on preset light source parameters and normal map to obtain diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image;

[0184] Real-time acquisition of ambient light parameters of the environment where the original oil painting image is located;

[0185] Based on the ambient light parameters, diffuse reflection parameters and specular reflection parameters, the target rendering color parameters of the oil painting pixels are calculated;

[0186] The oil painting pixels are rendered based on the target rendering color parameters to obtain target rendering pixels, and a target oil painting image is formed based on the plurality of target rendering pixels.

[0187] Optionally, the light source parameters include light source color, relative position parameters between the light source illuminating the oil painting and the oil painting, and the map pixels in the normal map include normal vectors;

[0188] The lighting rendering module 203 is specifically used to:

[0189] A spatial coordinate system is established based on the position of the oil painting to obtain the pixel coordinates of the oil painting pixels in the original oil painting image, where the positions of the original oil painting image and the normal map coincide;

[0190] Based on the pixel coordinates and the normal vector of the corresponding texture pixel, the normal parameters of the oil painting pixel are calculated;

[0191] Determine the light direction of the painting pixels based on relative positions and pixel coordinates;

[0192] Based on the light direction, normal parameters and light source color, the diffuse reflection parameters and specular reflection parameters of the oil painting pixels are calculated.

[0193] Optionally, the lighting rendering module 203 is specifically configured to:

[0194] Based on the relative position, the light source position coordinates in the spatial coordinate system are determined;

[0195] Based on the light source position coordinates and pixel coordinates, the light direction of the oil painting pixel is calculated.

[0196] Optionally, the target oil painting image is displayed on a display screen, and the system further includes a backlight brightness adjustment module; the backlight brightness adjustment module is specifically configured to:

[0197] Calculate the average pixel brightness of the target oil painting image;

[0198] Mapping the average pixel brightness to PWM duty cycle;

[0199] The backlight brightness of the display screen is adjusted based on the PWM duty cycle to match the brightness of the target oil painting image with the brightness of the environment in which the target oil painting image is located.

[0200] It should be noted that the image rendering system provided in the above embodiment and the image rendering method provided in the above embodiment are based on the same concept. The specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the image rendering system provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0201] A computing device according to an embodiment of the present application includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, some or all steps of the above-mentioned image rendering method are implemented.

[0202] Among them, the computing device can be a computer, and correspondingly, its program is computer software. The above-mentioned parameters and steps in a computing device of the present application can refer to the parameters and steps in an embodiment of an image rendering method above, and will not be repeated here.

[0203] In an embodiment of the present application, a computer-readable storage medium is provided, in which instructions are stored. When the instructions are executed, the steps of the above-mentioned image rendering method are executed.

[0204] The computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0205] The technical solution of the embodiments of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiments of the present disclosure. The aforementioned computer-readable storage medium can be a non-transitory computer-readable storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient computer-readable storage medium.

[0206] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] Those skilled in the art will appreciate that the present application may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "module" or "system." Furthermore, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, the computer-readable medium containing a computer-readable program code. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof.

[0208] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0209] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image rendering method, characterized in that: include: Obtaining the original oil painting image of the oil painting; Determining a normal map corresponding to the original oil painting image; Real-time lighting rendering is performed based on the original oil painting image and the normal map to obtain a target oil painting image.

2. The method according to claim 1, characterized in that Determining the normal map corresponding to the original oil painting image includes: Converting the original oil painting image into a grayscale image; performing brightness gradient extraction on the oil painting pixels in the grayscale image to obtain normal vectors of the oil painting pixels; A normal map corresponding to the original oil painting image is formed based on the multiple normal vectors.

3. The method according to claim 2, characterized in that The step of extracting brightness gradients of the oil painting pixels in the grayscale image to obtain normal vectors of the oil painting pixels includes: Extracting the horizontal brightness change rate of the oil painting pixels in the grayscale image and the vertical brightness change rate of the oil painting pixels in the vertical direction using a preset edge detection algorithm; determining the maximum horizontal brightness change rate among the plurality of horizontal brightness change rates as the horizontal brightness gradient of the oil painting pixel, and determining the maximum vertical brightness change rate among the plurality of vertical brightness change rates as the vertical brightness gradient of the oil painting pixel; The horizontal brightness gradient is used as the x-axis component, and the vertical brightness gradient is used as the y-axis component to form a normal vector of the oil painting pixel.

4. The method according to claim 1, wherein The performing real-time lighting rendering based on the original oil painting image and the normal map to obtain a target oil painting image includes: Performing illumination processing on the original oil painting image based on preset light source parameters and the normal map to obtain diffuse reflection parameters and specular reflection parameters of oil painting pixels in the original oil painting image; Acquiring in real time the ambient light parameters of the environment in which the original oil painting image is located; Calculating a target rendering color parameter of the oil painting pixel based on the ambient light parameter, the diffuse reflection parameter, and the specular reflection parameter; The oil painting pixels are rendered based on the target rendering color parameters to obtain target rendering pixels, and a target oil painting image is formed based on a plurality of the target rendering pixels.

5. The method according to claim 4, characterized in that The light source parameters include light source color and relative position parameters between the light source illuminating the oil painting and the oil painting, and the map pixels in the normal map include normal vectors; The performing illumination processing on the original oil painting image based on the preset light source parameters and the normal map to obtain diffuse reflection parameters and specular reflection parameters of the oil painting pixels in the original oil painting image includes: Establishing a spatial coordinate system based on the position of the oil painting to obtain pixel coordinates of the oil painting pixels in the original oil painting image, wherein the positions of the original oil painting image and the normal map coincide with each other; Calculating the normal parameters of the oil painting pixel based on the pixel coordinates and the normal vector of the corresponding map pixel; determining a light direction of the oil painting pixel based on the relative position and the pixel coordinates; Based on the light direction, the normal parameter and the light source color, diffuse reflection parameters and specular reflection parameters of the oil painting pixel are calculated.

6. The method according to claim 5, characterized in that The determining the light direction of the oil painting pixel based on the relative position and the pixel coordinates includes: Based on the relative position, determining the light source position coordinates of the light source in the spatial coordinate system; The light direction of the oil painting pixel is calculated based on the light source position coordinates and the pixel coordinates.

7. The method according to any one of claims 1 to 6, characterized in that The target oil painting image is displayed on a display screen, and the method further includes: Calculating the average pixel brightness of the target oil painting image; Mapping the pixel brightness average value to a PWM duty cycle; The backlight brightness of the display screen is adjusted based on the PWM duty cycle, so that the brightness of the target oil painting image matches the brightness of the environment in which the target oil painting image is located.

8. An image rendering system, characterized in that: include: An acquisition module, used for acquiring the original oil painting image of the oil painting; a determination module, configured to determine a normal map corresponding to the original oil painting image; The lighting rendering module is used to perform real-time lighting rendering based on the original oil painting image and the normal map to obtain a target oil painting image.

9. A computing device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the image rendering method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the steps of an image rendering method according to any one of claims 1 to 7.

Images