Image rendering method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image rendering method and device, electronic equipment and storage medium, which are applied to the field of image processing. The method comprises the following steps: determining a target position in an original image where a special effect needs to be added and a size parameter of a special effect region; the special effect region is a region occupied by the special effect to be added; determining texture coordinates of each pixel point in a surrounding region of the target position according to distance information of the each pixel point relative to the target position and the size parameter of the special effect region; the surrounding region is equal in size to the special effect region; sampling a preset texture map according to the texture coordinates of the each pixel point to obtain a special effect map; and fusing the special effect map with the original image at the target position to obtain a rendered image. The method can realize personalized special effect rendering.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to an image rendering method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] In a video special effect service, there is a common special effect called "star flash". Star flash is an optical phenomenon. When observing a bright part in a scene, a layer of glow is often found around the bright part. Some of the glow is directional, thus the effect of star flash appears.

[0003] A common algorithm for generating a special effect such as star flash is to use a fixed convolution kernel with the same shape as the special effect to perform convolution processing on an original image to achieve the special effect. However, the special effect shape achieved in this method is fixed, and it is difficult to achieve personalized special effect. SUMMARY

[0004] The present disclosure provides an image rendering method and device, electronic equipment, storage medium and program product to at least solve the problem that it is difficult to achieve personalized special effect in related technologies. The technical solutions of the present disclosure are as follows.

[0005] According to a first aspect of an embodiment of the present disclosure, an image rendering method is provided, comprising:

[0006] determining a target position in an original image where a special effect needs to be added and a size parameter of a special effect region; the special effect region is a region occupied by the special effect to be added;

[0007] determining a texture coordinate of each pixel point in a surrounding region of the target position according to distance information of the each pixel point relative to the target position and the size parameter of the special effect region; the surrounding region is equal in size to the special effect region;

[0008] sampling a preset texture map according to the texture coordinate of the each pixel point to obtain a special effect map;

[0009] fusing the special effect map with the original image at the target position to obtain a rendered image.

[0010] In an exemplary embodiment, the determination of the target position in the original image where the special effect needs to be added comprises:

[0011] generating a planar space grid of the original image, and generating a sampling point in each grid cell of the planar space grid;

[0012] selecting a target sampling point from the sampling points, and taking the position of the target sampling point as the target position where the special effect needs to be added.

[0013] In an example embodiment, the filtering of the target sampling point from the sampling points comprises:

[0014] The luminance value of each sampling point is obtained respectively;

[0015] The sampling point with a luminance value greater than a threshold value is filtered from the sampling points, and the sampling point with the luminance value greater than the threshold value is taken as the target sampling point.

[0016] In an example embodiment, the distance information comprises a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image;

[0017] The texture coordinates of each pixel point in the surrounding area of the target position are determined according to the distance information of the pixel point relative to the target position and the size parameter of the special effect region, comprising:

[0018] For each pixel point, the value of the red channel of the pixel point is determined based on a ratio of the horizontal distance of the pixel point relative to the target position to the size parameter of the special effect region;

[0019] The value of the green channel of the pixel point is determined based on a ratio of the vertical distance of the pixel point relative to the target position to the size parameter of the special effect region;

[0020] The value of the red channel and the value of the green channel constitute the texture coordinates of the pixel point.

[0021] In an example embodiment, the size parameter of the special effect region is determined, comprising:

[0022] An initial size parameter preset for the special effect region and an expansion size parameter allowing the target position to expand around are obtained; the expansion size parameter represents a range size of the expansion around;

[0023] The minimum value of the initial size parameter and the expansion size parameter is determined as the size parameter of the special effect region.

[0024] In an example embodiment, the method further comprises:

[0025] The luminance value of the target position is obtained;

[0026] The size parameter of the special effect region is adjusted according to the luminance value under the condition that the luminance value is positively correlated with the size parameter of the special effect region, to obtain a first size parameter;

[0027] The texture coordinates of each pixel point in the surrounding area of the target position are determined according to the distance information of the pixel point relative to the target position and the first size parameter.

[0028] In one exemplary embodiment, the method further includes:

[0029] Using time and the target location where special effects need to be added as variables, an adjustment coefficient is generated by adding noise.

[0030] The size parameters of the special effects area are adjusted according to the adjustment coefficient to obtain a second size parameter that changes with time and target position;

[0031] The texture coordinates of each pixel are determined based on the distance information of each pixel in the surrounding area of ​​the target location relative to the target location and the second size parameter.

[0032] In an exemplary embodiment, the texture map includes multiple image frames; the step of sampling the preset texture map according to the texture coordinates of each pixel to obtain an effects texture map includes:

[0033] By adjusting the texture coordinates of each pixel, multiple special effects textures are obtained by sampling each image frame separately.

[0034] The method further includes:

[0035] At the target location, each special effects texture is merged with the original image to obtain multiple rendered images;

[0036] The rendered images are displayed in sequence.

[0037] According to a second aspect of the present disclosure, an image rendering apparatus is provided, comprising:

[0038] The first determining unit is configured to determine the target location and size parameters of the effect region in the original image to which the effect needs to be added; the effect region is the area occupied by the effect to be added.

[0039] The second determining unit is configured to determine the texture coordinates of each pixel based on the distance information of each pixel in the surrounding area of ​​the target location relative to the target location and the size parameters of the effect area; the surrounding area is the same size as the effect area;

[0040] The texture sampling unit is configured to sample a preset texture map based on the texture coordinates of each pixel to obtain an effect texture map;

[0041] An image rendering unit is configured to perform an operation at the target location to fuse the effect texture with the original image to obtain a rendered image.

[0042] In an example embodiment, the first determining unit is further configured to perform generating a planar space grid of the original image, generating a sampling point in each grid cell of the planar space grid; and selecting a target sampling point from the sampling points, and taking a position of the target sampling point as a target position to which the special effect is to be added.

[0043] In an example embodiment, the first determining unit is further configured to perform respectively acquiring luminance values of the sampling points; and selecting a sampling point with a luminance value greater than a threshold value from the sampling points as the target sampling point.

[0044] In an example embodiment, the distance information comprises a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image; and the second determining unit is further configured to perform, for each pixel point, determining a value of a red channel of the pixel point based on a ratio of a horizontal distance of the pixel point relative to the target position to a size parameter of the special effect region; determining a value of a green channel of the pixel point based on a ratio of a vertical distance of the pixel point relative to the target position to the size parameter of the special effect region; and constructing the value of the red channel and the value of the green channel as a texture coordinate of the pixel point.

[0045] In an example embodiment, the first determining unit is further configured to perform acquiring a preset initial size parameter of the special effect region and an expansion size parameter allowing the target position to expand to the surroundings; the expansion size parameter represents a range size of expansion to the surroundings; and determining a minimum value of the initial size parameter and the expansion size parameter as the size parameter of the special effect region.

[0046] In an example embodiment, the apparatus further comprises a parameter adjusting unit configured to perform acquiring a luminance value of the target position; and adjusting the size parameter of the special effect region according to the luminance value under a condition that the luminance value is positively correlated with the size parameter of the special effect region, to obtain a first size parameter.

[0047] The second determining unit is further configured to perform determining texture coordinates of each pixel point in a surrounding area of the target position according to distance information of the pixel point relative to the target position and the first size parameter.

[0048] In an example embodiment, the parameter adjusting unit is further configured to perform generating an adjustment coefficient by adding noise with time and the target position to which the special effect is to be added as variables; and adjusting the size parameter of the special effect region according to the adjustment coefficient to obtain a second size parameter varying with time and the target position.

[0049] The second determining unit is further configured to determine a texture coordinate of each pixel point in the surrounding area of the target position according to distance information of the each pixel point relative to the target position and the second size parameter.

[0050] In an example embodiment, the texture map includes a plurality of image frames; and the texture sampling unit is further configured to sample each image frame by adjusting the texture coordinate of each pixel point to obtain a plurality of special effect maps.

[0051] The image rendering unit is further configured to fuse each special effect map with the original image at the target position to obtain a plurality of rendering images, and sequentially display the rendering images.

[0052] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising:

[0053] a processor;

[0054] a memory for storing instructions executable by the processor;

[0055] The processor is configured to execute the instructions to implement the method according to any one of the preceding aspects.

[0056] According to a fourth aspect of embodiments of the present disclosure, a computer-readable storage medium is provided, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of the preceding aspects.

[0057] According to a fifth aspect of embodiments of the present disclosure, a computer program product is provided, the computer program product includes instructions, when the instructions are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of the preceding aspects.

[0058] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects: after determining a target position in an original image where a special effect needs to be added, a texture coordinate of each pixel point is determined according to a size parameter of a special effect region and distance information of the each pixel point relative to the target position, a texture map is sampled according to the texture coordinate of each pixel point to obtain a special effect map, and a rendering image is obtained by fusing the special effect map with the original image. The method constructs a mapping relationship from the distance, the size parameter of the special effect region to the texture coordinate, which facilitates adjusting the texture coordinate of each pixel point by adjusting the size parameter of the special effect region, and further facilitates adjusting the special effect map, achieving the purpose of generating personalized special effect according to requirements, and the method is simple to operate, does not need to modify a convolution kernel, and has high rendering efficiency.

[0059] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and serve to explain the principles of the disclosure, and are not intended to represent the only embodiments consistent with the disclosure.

[0061] Figure 1 is a flowchart of an image rendering method according to an exemplary embodiment.

[0062] Figure 2 is a schematic diagram of a rendered image after adding a starburst effect according to an exemplary embodiment.

[0063] Figure 3 is a schematic diagram of a planar space grid according to an exemplary embodiment.

[0064] Figure 4 is a schematic diagram of a relationship between a planar space grid density and a number of effects according to an exemplary embodiment.

[0065] Figure 5 is a schematic diagram of mapping a distance of a pixel point relative to a target position as a texture coordinate according to an exemplary embodiment.

[0066] Figure 6 is a schematic diagram of a starburst effect varying with brightness according to an exemplary embodiment.

[0067] Figure 7 is a flowchart of an image rendering method according to another exemplary embodiment.

[0068] Figure 8 is a structural block diagram of an image rendering apparatus according to an exemplary embodiment.

[0069] Figure 9 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0070] In order to make the ordinary person in the art better understand the technical solutions of the disclosure, the technical solutions in the embodiments of the disclosure will be described clearly and completely below in conjunction with the drawings.

[0071] It should be noted that the implementations described in the following exemplary embodiments do not represent all implementations consistent with the disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the disclosure as detailed in the appended claims.

[0072] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0073] In an exemplary embodiment, as shown in Figure 1 An image rendering method is provided, and the embodiment is exemplified by applying the method to a terminal. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be realized through the interaction of the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server can be realized by an independent server or a server cluster composed of multiple servers. In the embodiment, the method includes the following steps:

[0074] In step S110, the target position in the original image where the special effect needs to be added and the size parameter of the special effect region are determined.

[0075] The original image can be an RGB color image.

[0076] The special effect to be added can be a star flashing special effect, the special effect region represents the region occupied by the special effect to be added, and the size parameter is a parameter representing the size of the special effect region, for example, the size parameter can be a radius.

[0077] In a specific implementation, the color features of each pixel point in the original image can be determined one by one, and the target position where the special effect needs to be added is determined according to the color features of each pixel point. Alternatively, the original image can be divided into a plurality of grid units, and whether a grid unit is a target position where the special effect needs to be added can be determined by the color feature of a pixel point in the grid unit. The size parameter of the special effect region can be determined according to actual needs, and can also be adjusted after being determined.

[0078] In step S120, the texture coordinates of each pixel point are determined according to the distance information of each pixel point in the surrounding region of the target position relative to the target position and the size parameter of the special effect region.

[0079] The surrounding area is equal in size to the special effect area.

[0080] The distance information includes a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image.

[0081] The texture coordinates (u, v) can be understood as an index used by a texture drawing primitive (such as a map), the texture coordinates are on the x-axis and the y-axis, and the range is [0, 1]. Using the texture coordinates to obtain the texture color is called sampling. u and v are respectively used to represent the numerical values of the red and green channels.

[0082] In a specific implementation, considering that the texture information of each pixel point in the special effect area is not completely the same when adding a special effect, that is, the texture colors of each pixel point are different, and the factors causing the difference are the size of the special effect area and the distance of each pixel point from the center of the special effect, therefore, the texture coordinates of each pixel point can be determined according to the two influencing factors, and the texture color is obtained by sampling according to the texture coordinates, so as to generate a special effect map.

[0083] Specifically, after determining the size parameter of the special effect area to which the special effect needs to be added, the area determined based on the size parameter of the special effect area is taken as the surrounding area of the target position, and the distance information of each pixel point in the surrounding area relative to the target position is calculated. For each pixel point, the texture coordinates of the pixel point are calculated based on the distance information of the pixel point relative to the target position and the size parameter of the special effect area, and thus the texture coordinates of each pixel point are obtained.

[0084] In step S130, the texture map is obtained by sampling the preset texture map according to the texture coordinates of each pixel point.

[0085] In a specific implementation, when the texture map is sampled according to the texture coordinates of each pixel point, the texture coordinates are used as indexes to find the corresponding colors in the texture map, and the special effect map is generated. Specifically, for any pixel point, the texture coordinates of the pixel point are mapped in the texture map, and the color of the corresponding mapping position in the texture map is taken as the texture color of the pixel point. Each pixel point performs this operation, so that the texture color of each pixel point can be obtained. According to the texture coordinates and the texture color of each pixel point, the corresponding map can be generated as the special effect map.

[0086] In step S140, the special effect map is fused with the original image at the target position to obtain a rendered image.

[0087] In a specific implementation, after obtaining the special effect map of the target position, the special effect map and the original image can be fused at the target position, and the fused image is the rendering image after adding the special effect corresponding to the original image.

[0088] For example, taking the star flash special effect as an example, referring to Figure 2 FIG. 2 is a schematic diagram of a rendering image after adding a star flash special effect according to an embodiment, wherein the left image shows an effect diagram of a star flash special effect added when the target position determined from the original image is one, and the right image shows an effect diagram of a star flash special effect added when the target position determined from the original image is five.

[0089] In the image rendering method, after determining the target position in the original image to which the special effect is added, the texture coordinates of each pixel point are determined according to the size parameter of the special effect region and the distance information of each pixel point in the surrounding region of the target position relative to the target position, the preset texture map is sampled according to the texture coordinates of each pixel point to obtain the special effect map, and the rendering image is obtained by fusing the special effect map and the original image. The method constructs a mapping relationship from the distance, the size parameter of the special effect region to the texture coordinates, which facilitates adjusting the texture coordinates of each pixel point by adjusting the size parameter of the special effect region, and further adjusting the special effect map, so as to achieve the purpose of generating personalized special effect according to requirements, and the method is simple to operate, does not need to modify the convolution kernel, and has high rendering efficiency.

[0090] In an example embodiment, in step S110, the target position in the original image to which the special effect is added is determined, including:

[0091] In step S111, a planar space grid of the original image is generated, and a sampling point is generated in each grid cell of the planar space grid.

[0092] In step S112, the target sampling point is selected from each sampling point, and the position of the target sampling point is taken as the target position to which the special effect is added.

[0093] In a specific implementation, before the planar space grid of the original image is generated, the size of the original image can be determined first, a planar space with the same size as the original image is generated, and then the planar space is divided into regular grid cells to obtain the planar space grid of the original image. When dividing the grid cells, the division parameters representing the density of the grid cells, such as the number of grid cells and the size of the grid cells, can be determined first, and the planar space is divided according to the division parameters.

[0094] A pixel point can be randomly generated in each grid cell, so that each grid cell of the planar space grid has a sampling point. Then, whether star flash will occur in the corresponding grid cell is determined according to the color characteristics of the sampling point, and target sampling points are selected from the sampling points according to the determination result. Specifically, sampling points in which star flash effects will occur are selected as target sampling points, and the positions of the target sampling points are taken as target positions to which effects need to be added.

[0095] For example, as shown in FIG. 1, which is a schematic diagram of a planar space grid generated for a 16:9 frame, the white points in the figure are generated sampling points. As shown in FIG. 2, the density of the planar space grid can be adjusted. The density can be determined according to the number and density of effects to be added. For example, if the number of effects to be added is small and the density is small, the grid cells can be larger and sparser; otherwise, if the number of effects to be added is large and the density is large, the grid cells can be smaller and denser. Figure 3 Figure 3 For example, as shown in FIG. 3, the planar space grid on the left side of the figure needs to add one effect, and the planar space grid on the right side of the figure needs to add five effects. Therefore, the density of the grid cells on the left side is smaller than that of the grid cells on the right side. Figure 4

[0096] In this embodiment, by generating a planar space grid and generating a sampling point in each grid cell, the positions of effects can be uniformly distributed in the entire image, avoiding the situation that effects are concentrated or densely distributed in some areas. After the target sampling points are selected, the positions to which effects need to be added can be accurately determined, ensuring effective display and presentation of the effects. In addition, the traditional method of determining positions prone to star flash through corner point detection needs to rely on access to third-party libraries, thereby increasing the computational overhead. Compared with the corner point detection method, the method of determining target positions provided in this embodiment can reduce the computational cost and save computing resources.

[0097] Further, in an example embodiment, in step S112, the target sampling points are selected from the sampling points, including: obtaining the brightness values of the sampling points respectively; selecting sampling points with brightness values greater than a threshold value from the sampling points, and taking the sampling points with brightness values greater than the threshold value as the target sampling points.

[0098] The brightness value of the sampling point is used to represent the brightness of the sampling point. For example, the gray value of the sampling point is between [0, 255], the closer the gray value is to 0, the lower the brightness, and the closer the gray value is to 255, the higher the brightness.

[0099] ​​Specifically, the luminance value of each sampling point can be obtained by the color feature of the sampling point, specifically, for each sampling point, the color feature of the sampling point, such as the RGB value, is obtained, and then the color feature of the sampling point is converted into the luminance value by the gray scale conversion formula. Each sampling point performs the operation, and thus the luminance value of each sampling point is obtained.

[0100] After obtaining the luminance value of each sampling point, the luminance value of each sampling point can be compared with a preset threshold value respectively, and a target sampling point is selected from each sampling point according to the comparison result, and the selection principle can be represented by the following formula (1):

[0101] (1)

[0102] Wherein, intensity represents the output value of the Step function, threshold represents the threshold value, lumiance represents the luminance value, and Step is a function with discrete output, the output value of which is one of 0 and 1, if the luminance value is greater than the threshold value, the output value is 1, otherwise, the output value is 0. When the output value is 1, it is determined that star flash will occur, and when the output value is 0, it is determined that star flash will not occur. Therefore, the target sampling point selected from each sampling point is the sampling point corresponding to the luminance value greater than the threshold value.

[0103] In the embodiment, by obtaining the luminance value of each sampling point and selecting the sampling point with the luminance value greater than the threshold value, the position of the star flash can be effectively determined, so that the star flash special effect is added at the target sampling point.

[0104] In an exemplary embodiment, the distance information includes a horizontal distance along the width direction of the original image and a vertical distance along the height direction of the original image; correspondingly, in step S120, the texture coordinates of each pixel point in the surrounding area of the target position are determined according to the distance information of each pixel point relative to the target position and the size parameter of the special effect area, which can be specifically implemented by the following steps:

[0105] In step S121, for each pixel point, the value of the red channel of the pixel point is determined based on the ratio of the horizontal distance of the pixel point relative to the target position to the size parameter of the special effect area;

[0106] In step S122, the value of the green channel of the pixel point is determined based on the ratio of the vertical distance of the pixel point relative to the target position to the size parameter of the special effect area;

[0107] In step S123, the value of the red channel and the value of the green channel are used to constitute the texture coordinates of the pixel point.

[0108] In a specific implementation, the two coordinate values u and v of the texture coordinates are positively correlated with the horizontal distance and the vertical distance of the pixel point relative to the target position, and are negatively correlated with the size parameter of the special effect region. The texture coordinates of a pixel point are determined according to the distance information of the pixel point relative to the target position and the size parameter of the special effect region, including determining the two coordinate values u and v of the texture coordinates. The coordinate u is determined by the horizontal distance of the corresponding pixel point relative to the target position, and the coordinate v is determined by the vertical distance of the corresponding pixel point relative to the target position, and u and v represent the values of the red and green channels respectively.

[0109] More specifically, since the coordinate value range of the texture coordinates is [0, 1], after calculating the ratio of the horizontal distance of the pixel point relative to the target position to the size parameter of the special effect region and the ratio of the vertical distance of the pixel point relative to the target position to the size parameter of the special effect region, the two ratios also need to be adjusted to the interval [0, 1], and the specific adjustment method is as shown in formula (2):

[0110] (2)

[0111] wherein, represents the distance information of the pixel point relative to the target position, including the horizontal distance and the vertical distance, represents the size parameter of the special effect region, representing the special effect size, which can be adjusted to the interval [0, 1] by multiplying the ratio by 0.5 and then adding 0.5.

[0112] Reference Figure 5 is a schematic diagram for mapping the distance of the pixel point relative to the target position into the texture coordinates, Figure 5 Each block 50 in corresponds to a pixel point of the special effect region, representing the u v information of each pixel point, represented by red and green, as shown in Figure 5 In the distance field, the closer to the distance sampling point, the darker the color of the pixel point, and after conversion to u v information, the closer to the distance sampling point, the greater the red channel value and the green channel value of the pixel point.

[0113] In this embodiment, the horizontal distance of the pixel point relative to the target position is mapped to the red channel value u, and the vertical distance of the pixel point relative to the target position is mapped to the green channel value v, according to the size parameter of the special effect region, and (u, v) is taken as the texture coordinates of the pixel point, realizing the mapping from the distance field to the texture coordinates, so that subsequent texture sampling can be performed according to the texture coordinates, and special effect rendering is realized.

[0114] In an example embodiment, determining the size parameter of the special effect region comprises: obtaining an initial size parameter preset for the special effect region, and an expansion size parameter allowing the target position to expand to the surroundings; the expansion size parameter represents the size of the expansion to the surroundings; and determining the minimum of the initial size parameter and the expansion size parameter as the size parameter of the special effect region.

[0115] Specifically, the size parameter of the special effect region represents the size of the special effect, and adjusting the size of the size parameter of the special effect region will affect the size of the final displayed special effect. If the size parameter of the special effect region is too large, it will cause the phenomenon of truncation of the special effect outside the iteration region (expansion region). Therefore, to avoid this phenomenon, the size of the actual special effect should meet the requirement of formula (3) as follows:

[0116] (3)

[0117] wherein, represents the determined size parameter of the special effect region, represents the initial size parameter of the special effect region, represents the expansion size parameter, and the min function is a function for finding the minimum value of a set of numbers.

[0118] That is, the size of the actual special effect should not exceed the range of its iteration region (i.e. the range of expansion to the surroundings). If the expansion size parameter corresponds to the size of one grid unit, the size of the actual special effect should be within the range of expansion of the target position to the outside by one grid unit. For example, as shown in the image representing the texture coordinates (UV information) in Figure 5 each box 50 in the figure can be regarded as the special effect region of one sampling point, and the area occupied by each box is within the range formed by expansion to the surroundings by one grid unit.

[0119] In this embodiment, the size parameter of the special effect region is set to the minimum of the initial size parameter of the special effect region and the expansion size parameter allowing the target position to expand to the surroundings, so that the size of the finally obtained actual special effect is within the allowed expansion range, avoiding the phenomenon of truncation outside the iteration region.

[0120] Considering that the brightness is different at different positions, the size of the corresponding displayed special effect will also be different. The present disclosure further proposes a method for adjusting the size parameter of the special effect region according to the brightness of the target position. Specifically, in an example embodiment, the method further comprises: obtaining the brightness value of the target position; adjusting the size parameter of the special effect region according to the brightness value under the condition that the brightness value is positively correlated with the size parameter of the special effect region, to obtain a first size parameter; and determining the texture coordinates of each pixel point in the surrounding region of the target position according to the distance information of each pixel point relative to the target sampling point and the first size parameter.

[0121] In the implementation, the size parameter of the special effect region is adjusted according to the luminance value, and the luminance value can be obtained by first obtaining the color feature of the target position and then converting the color feature by a gray scale conversion formula. Then, the size parameter of the special effect region is adjusted according to the condition that the luminance value is positively correlated with the size parameter of the special effect region, that is, the greater the luminance value, the greater the size parameter of the special effect region, so that the final special effect is larger; otherwise, the smaller the luminance value, the smaller the size parameter of the special effect region, so that the final special effect is smaller. The texture coordinates of each pixel point in the surrounding region of the target position are determined according to the first size parameter obtained after the adjustment and the distance information of each pixel point relative to the target sampling point, so as to realize the adjustment of the texture coordinates and the adjustment of the special effect map.

[0122] More specifically, the adjustment of the size parameter of the special effect region can be realized by multiplying the preset multiple of the luminance value and the size parameter of the special effect region, and the adjustment process can be represented by the following formula (4):

[0123] (4)

[0124] wherein, represents the first size parameter after the adjustment, represents the size parameter of the original special effect region, is the luminance value, m is the preset multiple, and pow(x, y) is used to calculate the yth power of x. Correspondingly, is used to calculate the mth power of .

[0125] Referring to Figure 6 , which is a schematic diagram of a star flash effect varying with luminance, five star flash effects are shown in the diagram. As can be seen from the diagram, the luminance of the position where the middle region 60 is located is higher than that of the region 61, and therefore, the star flash of the region 60 is larger than that of the region 61.

[0126] In the embodiment, the size parameter of the special effect region is adjusted according to the luminance value, so that the special effect effect shown by the sampling map determined based on the first size parameter after the adjustment is more realistic, and the special effect effect varying with the luminance is realized.

[0127] In view of the fact that the size of the special effect is not fixed, the embodiment further proposes a method for adjusting the size of the special effect by adding noise to adjust the size parameter of the special effect region. Specifically, in an exemplary embodiment, the method further comprises: generating an adjustment coefficient by adding noise with time and a target position to which the special effect is added as variables; adjusting the size parameter of the special effect region according to the adjustment coefficient to obtain a second size parameter that varies with time and the target position; and determining the texture coordinates of each pixel point according to distance information of each pixel point in a surrounding region of the target position relative to the target position and the second size parameter.

[0128] The noise can be Berlin noise.

[0129] Specifically, the size of the special effect can be adjusted from two dimensions of time and position, so that different sizes of the special effect are obtained for different positions and different sizes of the special effect are obtained for the same position at different time points, that is, the size of the special effect can be simultaneously affected by both position and time.

[0130] Specifically, the adjustment coefficient generated by adding noise with time and a target position to which the special effect is added as variables can be implemented by a trigonometric function, such as a sine function. Specifically, time and Berlin noise can be taken as input values of the sine function to obtain an adjustment coefficient that oscillates randomly, so that the second size parameter obtained according to the adjustment coefficient dynamically changes with time and position. The process of determining the second size parameter can be represented by the following formula (5):

[0131] (5)

[0132] wherein, represents the adjusted second size parameter, represents the size parameter of the original special effect region, represents Berlin noise, represents time, represents a sine function, is a function for calculating the absolute value of data, represents an adjustment coefficient.

[0133] In actual application, for different target positions at the same time point, different Berlin noises can be added to determine the adjustment coefficient with the target position as a variable. For the same target position, different Berlin noises can be added at different time points to determine the adjustment coefficient with time as a variable. The adjustment coefficient thus determined can change with time and the target position, thereby achieving the purpose of displaying different sizes of the special effect at different target positions and different time points.

[0134] In this embodiment, the size parameters of the special effect area are adjusted in combination with the trigonometric function and the Berlin noise, with time and the target position to which the special effect needs to be added as variables, so that the adjustment coefficient can change with time and the target position, thereby realizing the effect of displaying different special effects at different target positions and different time points.

[0135] In addition to realizing the dynamically changing special effect by adjusting the size parameters of the special effect area, the special effect effect can also be realized by sampling the sequence frame map. Specifically, in an example embodiment, the texture map includes a plurality of image frames; in step S130, the preset texture map is sampled according to the texture coordinates of each pixel point to obtain a special effect map, including: adjusting the texture coordinates of each pixel point to sample each image frame to obtain a plurality of special effect maps; at the target position, each special effect map is fused with the original image to obtain a plurality of rendering images; and each rendering image is sequentially displayed.

[0136] In a specific implementation, the preset texture map can be a sequence of image frames, that is, it includes a plurality of different image frames, and each image frame can be regarded as a video frame of a dynamic video, that is, the contents of each image frame are associated, for example, the sequence of image frames can be a series of images corresponding to the same object. When sampling the texture map according to the texture coordinates of each pixel point, the texture coordinates of each pixel point can be adjusted so that any image frame can be sampled.

[0137] More specifically, the adjustment parameters of the texture coordinates of each pixel point can be determined according to the position of the image frame to be sampled in the texture map, and the texture coordinates of each pixel point are adjusted according to the adjustment parameters, so that when the texture map is sampled with the adjusted texture coordinates, the sampling position is exactly at the position of the image frame to be sampled. If multiple image frames need to be sampled, the texture coordinates of each pixel point are adjusted multiple times according to this method, and one image frame is sampled each time, thereby obtaining a plurality of special effect maps. Each special effect map is fused with the original image to obtain a plurality of rendering images, and each rendering image is sequentially displayed when the special effect is displayed, thereby realizing the dynamically changing special effect effect, and the patterns of each special effect effect are different.

[0138] Further, which time point to display which rendering image can be determined by the following formula (6), and the Berlin noise can also be used in this method to help increase randomness:

[0139] (6)

[0140] wherein, represents the currently displayed rendering image, is a modulo function, represents the current time, represents a frame rate at which each rendered image is displayed, represents a total number of rendered images.

[0141] In this embodiment, by adjusting the texture coordinates of each pixel point, different image frames in the special effect map can be sampled at different pixel points in the texture map, so that a plurality of special effect maps with different patterns can be obtained, dynamic special effect effect can be realized, and the special effect can present a plurality of different patterns or animations at the target position, thereby improving the visual effect and expressiveness.

[0142] In another example embodiment, as shown in FIG. 8, which is a flowchart of another image rendering method according to an example embodiment, the method includes the following steps: Figure 7

[0143] Step S710, generating a planar space grid of the original image, and generating a sampling point in each grid cell of the planar space grid;

[0144] Step S720, obtaining the brightness value of each sampling point respectively; and selecting a sampling point with a brightness value greater than a threshold value from each sampling point as a target sampling point;

[0145] Step S730, obtaining an initial size parameter preset for the special effect region, and an expansion size parameter allowing the target sampling point to expand around;

[0146] Step S740, determining the minimum value of the initial size parameter and the expansion size parameter as the size parameter of the special effect region;

[0147] Step S750, for each pixel point in the surrounding area of the target sampling point, determining the value of the red channel of the pixel point based on the ratio of the horizontal distance of the pixel point relative to the target sampling point to the size parameter of the special effect region;

[0148] Step S760, determining the value of the green channel of the pixel point based on the ratio of the vertical distance of the pixel point relative to the target sampling point to the size parameter of the special effect region;

[0149] Step S770, constructing the texture coordinates of the pixel point by the value of the red channel and the value of the green channel;

[0150] Step S780, sampling the preset texture map according to the texture coordinates of each pixel point to obtain a special effect map;

[0151] Step S790, fusing the special effect map with the original image at the target position to obtain a rendered image.

[0152] ​The image rendering method provided in the application utilizes the characteristics of the grid noise suitable for some specific pattern generation requirements, supports sampling of a map, thereby avoiding the method of modifying a convolution kernel in the traditional method, and facilitates the production of various styles of star flash effects. Meanwhile, the generation of the star flash is based on the efficient implementation of the modification of the grid noise, and the overhead is not much different from the traditional filtering method, so that the method can be used on a mobile terminal, and the drawing efficiency and drawing effect can be considered.

[0153] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0154] It can be understood that the same / similar parts between each embodiment of the above method in the specification can be mutually referred to, and each embodiment focuses on the differences from other embodiments, and the related parts can be referred to the description of other method embodiments.

[0155] Based on the same inventive concept, the disclosure also provides an image rendering device for implementing the above-mentioned image rendering method.

[0156] Figure 8 is a structural block diagram of an image rendering device according to an exemplary embodiment. Referring to Figure 8 , the device comprises:

[0157] The first determination unit 810 is configured to determine the target position in the original image where the special effect needs to be added and the size parameter of the special effect area; the special effect area is the area occupied by the special effect to be added;

[0158] The second determination unit 820 is configured to determine the texture coordinates of each pixel point according to the distance information of each pixel point in the surrounding area of the target position relative to the target position and the size parameter of the special effect area; the surrounding area is equal in size to the special effect area;

[0159] The map sampling unit 830 is configured to sample the preset texture map according to the texture coordinates of each pixel point to obtain the special effect map;

[0160] The image rendering unit 840 is configured to perform fusing the special effect map with the original image at the target position to obtain a rendered image.

[0161] In an example embodiment, the first determining unit 810 is further configured to perform generating a planar space grid of the original image, generating a sampling point in each grid cell of the planar space grid, and selecting a target sampling point from the sampling points, wherein a position of the target sampling point is the target position to which the special effect is to be added.

[0162] In an example embodiment, the first determining unit 810 is further configured to perform respectively acquiring a brightness value of each sampling point, and selecting a sampling point with a brightness value greater than a threshold value from the sampling points as the target sampling point.

[0163] In an example embodiment, the distance information includes a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image, and the second determining unit 820 is further configured to perform, for each pixel point, determining a value of a red channel of the pixel point based on a ratio of a horizontal distance of the pixel point relative to the target position to the size parameter of the special effect region, determining a value of a green channel of the pixel point based on a ratio of a vertical distance of the pixel point relative to the target position to the size parameter of the special effect region, and constructing the texture coordinate of the pixel point by using the value of the red channel and the value of the green channel.

[0164] In an example embodiment, the first determining unit 810 is further configured to perform acquiring a preset initial size parameter of the special effect region and an expansion size parameter allowing the target position to expand to the surrounding, the expansion size parameter representing a range size of the expansion to the surrounding, and determining a minimum value between the initial size parameter and the expansion size parameter as the size parameter of the special effect region.

[0165] In an example embodiment, the apparatus further includes a parameter adjusting unit configured to perform acquiring a brightness value of the target position, and adjusting the size parameter of the special effect region according to the brightness value under a condition that the brightness value is positively correlated with the size parameter of the special effect region to obtain a first size parameter.

[0166] The second determining unit 820 is further configured to perform determining a texture coordinate of each pixel point in the surrounding region of the target position according to distance information of the pixel point relative to the target position and the first size parameter.

[0167] In an example embodiment, the parameter adjusting unit is further configured to perform generating an adjustment coefficient by adding noise with time and the target position to which the special effect is to be added as variables, and adjusting the size parameter of the special effect region according to the adjustment coefficient to obtain a second size parameter varying with time and the target position.

[0168] The second determining unit 820 is further configured to determine the texture coordinate of each pixel according to the distance information of each pixel in the surrounding area of the target position relative to the target position and the second size parameter.

[0169] In an example embodiment, the texture map includes a plurality of image frames; the texture sampling unit 830 is further configured to sample each image frame to obtain a plurality of special effect maps by adjusting the texture coordinate of each pixel, respectively.

[0170] The image rendering unit 840 is further configured to fuse each special effect map with the original image at the target position to obtain a plurality of rendering images, and sequentially display each rendering image.

[0171] As to the apparatus in the above-mentioned embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0172] Figure 9 FIG. 9 is a block diagram of an electronic device 900 for implementing the image rendering method according to an example embodiment. The electronic device 900 can be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0173] Referring to Figure 9 The electronic device 900 can include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0174] The processing component 902 generally controls the overall operation of the electronic device 900 such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 902 can include one or more processors 920 to execute instructions and manipulate data to perform all or a subset of the steps described in the above methods. In some example embodiments, the processing component 902 can further include one or more modules to facilitate interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0175] The memory 904 is configured to store various types of data to support the operation of the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disc, optical disc, or graphene memory.

[0176] The power supply component 906 supplies power for various components of the electronic device 900. The power supply component 906 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 900.

[0177] The multimedia component 908 includes a screen providing an output interface between the electronic device 900 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 908 includes a front camera and / or a back camera. The front camera and / or the back camera can receive external multimedia data when the electronic device 900 is in an operation mode, such as a photographing mode or a video mode. Each of the front camera and the back camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0178] The audio component 910 is configured to output and / or input an audio signal. For example, the audio component 910 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 900 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting an audio signal.

[0179] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which can be a keypad, a click wheel, buttons, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0180] The sensor component 914 includes one or more sensors for providing status assessments for various aspects of the electronic device 900. For example, the sensor component 914 can detect an open / closed position of the electronic device 900, relative positioning of components, such as a display and a keypad of the electronic device 900, a change in position of the electronic device 900 or a component of the electronic device 900, presence or absence of user contact with the electronic device 900, orientation or acceleration / deceleration / g-force and temperature of the electronic device 900. The sensor component 914 can include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0181] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G or 5G), or a combination thereof. In an example embodiment, the communication component 916 receives broadcast signals or broadcast-related information from external broadcast management systems via a broadcast channel. In an example embodiment, the communication component 916 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques and other techniques.

[0182] In an example embodiment, the electronic device 900 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic elements to perform the above-described methods.

[0183] In an example embodiment, a computer-readable storage medium, such as the memory 904 including instructions stored thereon is also provided, which can be executed by the processor 920 of the electronic device 900 to perform the above-described methods. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0184] In an exemplary embodiment, a computer program product is also provided, which comprises instructions executable by the processor 920 of the electronic device 900 to perform the above method.

[0185] It should be noted that the above apparatus, electronic device, computer readable storage medium, computer program product and the like according to the description of the method embodiments can also include other implementation manners, and the specific implementation manners can be referred to the description of the related method embodiments, which will not be repeated here.

[0186] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the concepts disclosed herein. The disclosure is intended to cover any variations, uses or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the appended claims.

[0187] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. An image rendering method, characterized by, The method comprises: determining a target position and a size parameter of a special effect region in an original image, wherein the special effect to be added is a star flash effect, the special effect region is a region occupied by the special effect to be added, and the target position is obtained according to brightness characteristics of each pixel point in the original image; determining distance information of each pixel point in a surrounding region of the target position relative to the target position, wherein the surrounding region is equal in size to the special effect region, and the distance information comprises a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image; for each pixel point, determining a value of a red channel of the pixel point based on a ratio of the horizontal distance of the pixel point relative to the target position to the size parameter of the special effect region, and determining a value of a green channel of the pixel point based on a ratio of the vertical distance of the pixel point relative to the target position to the size parameter of the special effect region, and constructing the value of the red channel and the value of the green channel into a texture coordinate of the pixel point; sampling a preset texture map according to the texture coordinates of the pixel points to obtain a special effect map; fusing the special effect map and the original image at the target position to obtain a rendered image.

2. The method of claim 1, wherein, The method for determining the target position in the original image comprises: generating a planar space grid of the original image, and generating a sampling point in each grid cell of the planar space grid; selecting a target sampling point from the sampling points, and taking a position of the target sampling point as the target position of the special effect to be added.

3. The method of claim 2, wherein, The method for selecting the target sampling point from the sampling points comprises: obtaining brightness values of the sampling points respectively; selecting a sampling point with a brightness value greater than a threshold value from the sampling points as the target sampling point.

4. The method of claim 1, wherein, The method for determining the size parameter of the special effect region comprises: obtaining a preset initial size parameter for the special effect region and an expansion size parameter allowing the target position to expand to the surrounding, wherein the expansion size parameter represents a range size of the expansion to the surrounding; and determining a minimum value of the initial size parameter and the expansion size parameter as the size parameter of the special effect region.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: obtaining a brightness value of the target position; adjusting the size parameter of the special effect region according to the brightness value to obtain a first size parameter, under a condition that the brightness value is positively correlated with the size parameter of the special effect region; and determining texture coordinates of each pixel point in the surrounding region of the target position according to the distance information of the pixel point relative to the target position and the first size parameter.

6. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: generating an adjustment coefficient by adding noise with time and the target position of the special effect to be added as variables; adjusting the size parameter of the special effect region according to the adjustment coefficient to obtain a second size parameter varying with time and the target position; and determining texture coordinates of each pixel point in the surrounding region of the target position according to the distance information of the pixel point relative to the target position and the second size parameter.

7. The method according to any one of claims 1 to 4, characterized in that, The texture map includes a plurality of image frames; and the texture map is sampled according to the texture coordinates of the respective pixel points to obtain a special effect map, including: The texture coordinates of the respective pixel points are adjusted to sample each image frame to obtain a plurality of special effect maps; The method further includes: Each special effect map is fused with the original image at the target position to obtain a plurality of rendered images; The rendered images are sequentially displayed.

8. An image rendering apparatus, characterized by comprising: It includes: A first determination unit configured to determine a target position in an original image where a special effect needs to be added and a size parameter of a special effect region; wherein the special effect to be added is a star flash effect, and the special effect region is a region occupied by the special effect to be added; the target position is obtained by filtering according to brightness characteristics of respective pixel points in the original image; A second determination unit configured to determine distance information of respective pixel points in a surrounding region of the target position relative to the target position; the surrounding region is equal in size to the special effect region, and the distance information includes a horizontal distance along a width direction of the original image and a vertical distance along a height direction of the original image; for each of the pixel points, a value of a red channel of the pixel point is determined based on a ratio of the horizontal distance of the pixel point relative to the target position to the size parameter of the special effect region, and a value of a green channel of the pixel point is determined based on a ratio of the vertical distance of the pixel point relative to the target position to the size parameter of the special effect region; and the value of the red channel and the value of the green channel constitute a texture coordinate of the pixel point; A map sampling unit configured to sample a preset texture map according to the texture coordinates of the respective pixel points to obtain a special effect map; An image rendering unit configured to fuse the special effect map with the original image at the target position to obtain a rendered image.

9. An electronic device, comprising: It includes: A processor; A memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image rendering method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the image rendering method of any one of claims 1 to 7.

Images