A stylization method, device, and storage medium

Abstract

The application discloses a stylization method, device and storage medium, the method comprising: obtaining original image data; detecting the contour of each element in the original image data; detecting the contour point representing color change in the contour; constructing a plurality of color blocks in the original image data according to the contour point; filling the color blocks with a uniform color according to the color of each element in the original image data, and obtaining target image data. The color blocks maintain the approximate contour of each element in the original image data, and also maintain the approximate color of each element in the original image data at the position of the color blocks, so that the content of the target image data maintains the approximate texture of the original image data, the contour and color change have strong analyticality, the contour and color change have strong analyticality, the change of the structure and color of the picture during stylization is predictable, and it is convenient for the user to find a suitable stylization effect.

Description

Technical Field

[0001] This invention relates to the field of multimedia technology, and more particularly to a stylization method, apparatus, and storage medium. Background Technology

[0002] In scenarios such as social media, short videos, and advertising, users create a large number of multimedia files in the form of image or video data. After generating the original multimedia files, they are usually post-processed to personalize them.

[0003] One common post-processing technique used by users is low-polygon. Currently, if a user is a professional artist, they can use professional modeling tools to create low-poly models when making multimedia files. However, this method has a high technical threshold and is considered pre-production, resulting in high production costs.

[0004] Currently, in order to reduce the technical threshold and cost of achieving low-poly models, most methods involve collecting image data with similar styles as samples to train neural networks, and then using the neural networks to perform style transfer on multimedia files, thereby transforming the style of the multimedia file's visuals.

[0005] However, neural networks have a large structure, consume a lot of resources, are slow in stylization, and have poor resolution. They are unpredictable when performing style transfer on the structure and color of an image, and may require repeated operations to find a suitable transfer effect. Summary of the Invention

[0006] This invention provides a stylization method, apparatus, and storage medium to address how to improve the speed and predictability of implementing impressionistic blurring of multimedia files.

[0007] According to one aspect of the present invention, a stylization method is provided, comprising:

[0008] Obtain the raw image data;

[0009] The contours of each element are detected in the original image data;

[0010] Detect contour points representing color changes within the contour;

[0011] Multiple color blocks are constructed in the original image data based on the contour points;

[0012] Using the colors of each element in the original image data as a reference, the color blocks are filled with a uniform color to obtain the target image data.

[0013] According to another aspect of the present invention, a stylization method is provided, comprising:

[0014] The content to be acquired is the original video data introducing the game, which contains multiple frames of original image data;

[0015] The contours of each element are detected in the original image data;

[0016] Detect contour points representing color changes within the contour;

[0017] Multiple adjacent color blocks are constructed in the original image data based on the contour points;

[0018] Using the colors of each element in the original image data as a reference, the color blocks are filled with a uniform color to obtain the target image data;

[0019] The target image data is replaced in the original video data to obtain the target video data.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] At least one processor; and

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the stylization method described in any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program configured to cause a processor to execute and implement the stylization method described in any embodiment of the present invention.

[0025] In this embodiment, the original image data is acquired; the contours of each element in the original image data are detected; contour points representing color changes are detected in the contours; multiple color blocks are constructed in the original image data based on the contour points; and the color blocks are filled with a uniform color using the colors of each element in the original image data as a reference, thus obtaining the target image data. This embodiment uses contours and color changes to construct color blocks and fills the color blocks with color with reference to the colors of the original image data to achieve a low-polygon style. This process does not rely on professional editing tools, has a low technical threshold, and does not rely on neural networks. The entire process is simple to operate and has low computational load, which can greatly reduce resource consumption and improve the speed and efficiency of stylization. Furthermore, while maintaining the approximate contours of each element in the original image data, the color blocks also maintain the approximate colors of each element in the original image data relative to the position of the color block. This ensures that the content of the target image data maintains the approximate texture of the original image data, and the contours, color changes, and color blocks have strong resolution. The changes in the structure and color of the image during stylization are predictable, making it easier for users to find suitable stylization effects.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a stylization method provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is an example diagram of a low-polygon style according to Embodiment 1 of the present invention;

[0030] Figure 3 This is a flowchart of a stylization method provided according to Embodiment 2 of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of a stylization device according to Embodiment 3 of the present invention;

[0032] Figure 5 This is a schematic diagram of the structure of a stylization device according to Embodiment 4 of the present invention;

[0033] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 This is a flowchart of a stylization method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where color blocks are constructed based on contours and color changes to achieve a low-polygon style for image data. This method can be executed by a stylization device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0038] Step 101: Obtain the raw image data.

[0039] Low-polygon, also known as low-poly model, is the opposite of high-poly model. High-poly model is characterized by a high number of faces and many details, while low-poly model has fewer faces and fewer details.

[0040] In the era of low computing power, in order to ensure smooth visuals, especially in game development and real-time 3D rendering, scene complexity was often reduced, polygon count was cut, and textures were used to compensate for lost details, gradually forming the low-polygon style.

[0041] With the development of computer technology and the power of game engines, it is possible to create models with rich details, more realistic visual styles, and greatly improve the sense of immersion, engagement, and user experience.

[0042] However, with the popularity of realistic style, aesthetic fatigue and the pursuit of individuality are easy to occur. In order to enrich the monotonous visual style, styles such as retro pixel and low-polygon have gradually emerged.

[0043] Furthermore, the Low Polygon style involves consciously cutting and stacking a certain number of simple geometric shapes according to the rules of object shaping. It emphasizes the overall effect of the image and the high degree of generalization of objects, color matching and light and shadow relationships, and has an artistic style of abstract expression and minimalism.

[0044] In this embodiment, the computer program for stylizing low-polygons is relatively simple. Therefore, the computer program for stylizing low-polygons can be deployed on the server side to provide low-polygon stylization services to users on the local area network and / or the public network via API (Application Programming Interface), or it can be deployed on the client side as a plugin, hard-coded, etc. to provide low-polygon stylization services to client users. This embodiment does not limit this.

[0045] If the computer program that implements the stylization of low-polygons is deployed on the server, then the user can call the API provided by the server in the client to upload the image data to be stylized to the server, or provide the network address of the image data to be stylized to the server, and the server will download the image data to be stylized based on the network address.

[0046] If the computer program that implements the stylization of low-polygons is deployed on the server, then users can select image data located in the local directory on the client and wait for stylization, or call the camera to capture a frame of image data and wait for stylization.

[0047] For ease of distinction, the image data to be stylized is referred to as the original image data, which takes different forms in different scenarios, such as photos, posters, portraits, etc.

[0048] Step 102: Detect the contours of each element in the original image data.

[0049] Generally, the content of raw image data contains a variety of elements. In scenarios such as game production and animation production, these elements may be virtual elements, such as virtual portraits, virtual grass, virtual trees, and virtual buildings. In scenarios such as user selfies and short video recordings, these elements may be real elements, such as user avatars and buildings.

[0050] In this embodiment, the contours (also known as edges) of each element can be detected in the original image data, thereby achieving a low-polygon style for the original image data based on the contours of each element, while preserving the basic state of each element.

[0051] Furthermore, edge detection operators can be used to detect the contours of each element in the original image data, such as the Sobel operator, Prewitt operator, and Canny operator. Alternatively, usable mathematical models can be extracted from the vision system to detect the contours of each element in the original image data, such as the Gabor filter and the CORE model.

[0052] Different detection contour methods have different focuses. In this embodiment, the detection contour method can be set according to business needs. The contours of the elements detected in different detection contour methods are also different. This embodiment does not impose any restrictions on this.

[0053] In one embodiment of the present invention, step 102 may include the following steps:

[0054] Step 1021: Calculate the grayscale value of the original image data to obtain grayscale image data.

[0055] In general, the original image data is color image data. In order to speed up the processing, the gray value of each pixel in the original image data can be calculated and cached. The gray value of each pixel represents a frame of image data, which is called grayscale image data.

[0056] For example, the color value of each pixel in the original image data can be substituted into any of the following formulas according to business requirements to output its grayscale value:

[0057] (1), Gray=0.3*R+0.59*G+0.11*B

[0058] Where Gray is the grayscale value, R is the red component of the color value, G is the green component of the color value, and B is the blue component of the color value.

[0059] This formula has a wide range of applications and can meet general business needs. However, this formula uses floating-point calculations, which involve a large amount of computation.

[0060] (2), Gray=77*R+150*G+29*B+128

[0061] Where Gray is the grayscale value, R is the red component of the color value, G is the green component of the color value, and B is the blue component of the color value.

[0062] This formula uses integer and bitwise operations, which can greatly reduce the amount of calculation and improve speed.

[0063] Step 1022: Calculate the gradient value of the grayscale image data to obtain the gradient image data.

[0064] In this embodiment, gradient values ​​can be calculated for each pixel in the grayscale image data. Each frame of image data is represented by its gradient value, denoted as gradient image data. The gradient value characterizes the rate of change in the original image data and is considered edge information of the original image data. For the edge portions of the original image data, the grayscale value changes significantly, resulting in larger gradient values; conversely, for smoother portions of the original image data, the grayscale value changes less significantly, resulting in smaller gradient values.

[0065] In one approach, a first convolutional kernel and a second convolutional kernel can be loaded separately, and the weights in the first and second convolutional kernels can be amplified to increase the differences between pixels.

[0066] For example, the first convolution kernel G x Second convolution kernel G y They are respectively:

[0067]

[0068] The first convolution kernel is used to perform a two-dimensional first convolution operation on the grayscale image data in the horizontal direction (x direction) to obtain the gradient value in the horizontal direction.

[0069] A second convolution kernel is used to perform a two-dimensional second convolution operation on the grayscale image data in the vertical direction (y direction) to obtain the gradient value in the vertical direction.

[0070] The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are merged into the gradient values ​​of the grayscale image data to obtain the gradient image data.

[0071] For example, the absolute values ​​of the gradients in the horizontal direction and the vertical direction are summed to obtain the gradient value of the grayscale image data, thus yielding the gradient image data.

[0072] For example, the square root of the sum of the squares of the gradient values ​​in the horizontal direction and the squares of the gradient values ​​in the vertical direction can be used as the gradient value of the grayscale image data to obtain the gradient image data.

[0073] Step 1023: Perform normalization processing on the gradient values ​​in the gradient image data.

[0074] In this embodiment, the gradient values ​​of each pixel in the gradient image data can be normalized using methods such as min-max (minimum value - maximum value) to adjust the scale of the gradient values ​​of each pixel to a similar range, which facilitates calculation.

[0075] Step 1024: Take the absolute value of the normalized gradient value in the gradient image data to obtain the contour of each element.

[0076] In this embodiment, the absolute value of the gradient value after normalization of each pixel in the gradient image data is taken, and the absolute value of the gradient of each pixel represents a frame of image data, which is recorded as the contour of each element.

[0077] Step 103: Detect contour points that represent color changes in the contour.

[0078] In the original image data, find pixels with obvious color changes along the contour (image data) and record them as contour points.

[0079] Contour points with significant color changes can be used as the basis for reducing details in low-polygon, so that the color changes still conform to the original image data to a certain extent after detail reduction, thus reducing the degree of distortion.

[0080] In one embodiment of the present invention, step 103 may include the following steps:

[0081] Step 1031: Calculate the histogram for the grayscale image data.

[0082] In this embodiment, grayscale image data corresponding to the original image data can be queried in the cache. The grayscale image data contains the grayscale values ​​of each pixel in the original image data.

[0083] For grayscale image data, a histogram can be calculated. The histogram describes the grayscale distribution in the original image data and can intuitively show the proportion of each grayscale level in the original image data. It describes the number of pixels at each grayscale level, but does not include the position information of these pixels in the original image data. Therefore, it is not affected by the rotation and translation of the original image data and can be used as a feature of the original image data.

[0084] Generally, histograms are represented by coordinates, with the horizontal axis representing the gray level and the vertical axis representing the probability of that gray level appearing.

[0085] Step 1032: Calculate the cumulative distribution probability of the histogram.

[0086] In this embodiment, the cumulative distribution function can be used to calculate the cumulative distribution probability of the histogram.

[0087] The cumulative distribution function, also called the distribution function, is the integral of the probability density function and can completely describe the probability distribution of a real random variable X (i.e., its histogram).

[0088] In the process of finding contour points, to improve the accuracy of detecting color changes and increase the contrast of the original image data, two conditions must be ensured:

[0089] 1. No matter how pixels are mapped, the original size relationship must be maintained. Brighter areas will remain brighter, and darker areas will remain darker.

[0090] 2. If the original image data is eight bits, then the value range of the pixel mapping relationship should be between 0 and 255.

[0091] Based on the above two conditions, the cumulative distribution function is a monotonically increasing function (controlling the magnitude relationship) and its range is from 0 to 1 (controlling the problem of exceeding the limit), which meets the above two conditions.

[0092] Step 1033: Map the original image data to reference image data based on the cumulative distribution probability.

[0093] Since the original image data is composed of individual pixels, the similarity of each pixel can be equalized by solving the discrete form of the cumulative distribution function, thereby improving the contrast of the original image data.

[0094] In this process, a function can be generated based on the cumulative distribution probability to map the color values ​​of each pixel, denoted as the mapping relationship. The color values ​​(such as red component, green component, and blue component) of each pixel in the original image data are then mapped according to the mapping relationship. The merge() method is used to merge the mapped channels to obtain the reference image data.

[0095] For example, the mapping relationship is as follows:

[0096]

[0097] Among them, s kThis refers to the value of the current gray level after being mapped by the cumulative distribution function, where n is the sum of the pixels in the original image data. j It is the number of pixels at the current gray level, k = 0, 1, 2, ..., L-1, where L is the total number of gray levels in the grayscale image data.

[0098] In this embodiment, using histograms to equalize the colors of the original image data can improve the color contrast of the original image data. It transforms the original image data into one with the same number of pixels at each gray level (i.e., the output histogram is flat), which can produce an image with a balanced gray level distribution probability.

[0099] In one example, suppose the color values ​​of a certain channel of the original image data are as follows:

[0100] 255 128 200 50 50 200 255 50 255 200 128 128 200 200 255 50

[0101] The obtained statistics are as follows, and a mapping is performed:

[0102]

[0103]

[0104] Taking 50 as an example, the mapped color value is calculated as follows: 0.25 * (255 - 0) = 63.75

[0105] Therefore, the color values ​​of this graph in the original image data are mapped as follows:

[0106] 255 112 191 64 64 191 255 64 255 191 112 112 191 191 255 64

[0107] Step 1034: In the reference image data, select pixels whose color values ​​meet the preset change conditions along the contour as contour points representing color changes.

[0108] The system iterates through the reference image data to select pixels whose color values ​​meet preset change conditions (such as the difference between the color values ​​of adjacent pixels being greater than a certain threshold) along the contour, and uses these pixels as contour points to represent color changes.

[0109] Furthermore, a target value, such as 50000, can be preset. This target value can be a default empirical value or a stylized intensity set by the user and mapped to the target value. This embodiment does not impose any restrictions on this.

[0110] Therefore, in the reference image data, pixels that meet the preset change conditions and whose number is the target value are randomly selected along the contour as contour points representing the color change.

[0111] The larger the target value, the more contour points there are, and the less detail is removed when achieving the low-polygon style, making it closer to the original image data.

[0112] The smaller the target value, the fewer the number of contour points, and the more details are removed when implementing the Low Polygon style, resulting in a style that is further removed from the original image data.

[0113] In another embodiment of the present invention, step 103 may further include the following steps:

[0114] Step 1035: Construct a point pair from any two contour points.

[0115] In this embodiment, any contour point can be constructed as a point pair with any other contour point.

[0116] Step 1036: For any pair of points, calculate the distance between the contour points in the pair.

[0117] For any pair of points, the distance between the two contour points in the pair can be calculated using methods such as Euclidean distance.

[0118] Step 1037: If the distance is greater than or equal to the preset first threshold, then retain the contour points in the point pair.

[0119] Step 1038: If the distance is less than the preset first threshold, then filter out any contour point in the point pair.

[0120] In this embodiment, the distance can be compared with a preset first threshold.

[0121] If the distance is greater than or equal to the first threshold, it means that the distance is large, that is, the spacing between the two contour points in the point pair is large, the low-polygon style is strong, and the contour points in the point pair can be preserved.

[0122] If the distance is less than the first threshold, it means that the distance is small, that is, the spacing between the two contour points in the point pair is small, the style of the low-polygon is weak, and either contour point in the point pair can be deleted.

[0123] Step 104: Construct multiple color blocks based on contour points in the original image data.

[0124] In this embodiment, for all contour points in the original image data, at least three contour points are connected according to the low-polygon stylization specification to construct a color block. This color block is generally a convex polygon so that it can be rendered by rendering engines such as OpenGL (Open Graphics Library) and OpenGL ES (OpenGL for Embedded Systems).

[0125] Furthermore, the color blocks are ordered, thus forming a sequence that conforms to the rendering specifications of rendering engines such as OpenGL and OpenGL ES.

[0126] In a practical implementation, contour points can be traversed in the original image data, and three adjacent contour points can be connected to construct multiple color blocks in the shape of triangles, where the triangles are adjacent to each other and do not overlap.

[0127] In this system, the circumcircle of each color block does not contain any other contour points. The Thiessen polygons (Dirichlet diagrams, also known as Voronoi diagrams) corresponding to the contour points that serve as vertices of the color blocks have a common vertex, which is the center of the circumcircle of the color block.

[0128] In a Thiessen polygon, the Euclidean distance between any two contour points p and q is denoted as dist(p,q).

[0129] Let P = {p1, p2, ..., p} n Let P be any n distinct contour points on the plane, which are also the base points. The Voronoi diagram corresponding to P is a subregion of the plane—the entire plane is thus divided into n units, which have the following properties:

[0130] Any contour point q is located at contour point p i In the corresponding unit, if and only if for any p j ∈P j ,j≠i, all have dist(q,p) i ) < dist(q,p j At this point, the Voronoi diagram corresponding to P is denoted as Vor(P).

[0131] The term "Vor(P)" or "Voronoi diagram" indicates the edges and vertices that make up the subregion partition. In Vor(P), the edges and vertices are related to the base point p. i The corresponding unit is denoted as V(p) i — Referred to as p i The corresponding Voronoi unit.

[0132] The methods for constructing color blocks differ depending on the rendering engine. For example, for OpenGL, there are generally three ways to draw a series of triangles using color blocks:

[0133] 1. GL_TRIANGLES

[0134] Draw a triangle with three outline points as a group, and the triangles are independent of each other.

[0135] 2. GL_TRIANGLE_STRIP

[0136] Starting from the third contour point, each point is combined with the two preceding contour points to draw a triangle, forming a linear continuous series of triangles.

[0137] This order is to ensure that the triangles are drawn in the same direction, so that the sequence of triangles can correctly form part of the surface.

[0138] 3. GL_TRIANGLE_FAN.

[0139] Starting from the third contour point, each point is combined with the previous contour point and the first contour point to draw a triangle, i.e., a continuous sector triangle.

[0140] In one embodiment of the present invention, the above-described method of constructing color blocks may further include the following steps:

[0141] Step 1041: Determine the baseline.

[0142] In this embodiment, color blocks can be constructed recursively multiple times, and at least one baseline is determined in each recursion.

[0143] The initial baseline is the line connecting any contour point in the original image data to the other contour point closest to that contour point. That is, initially, any contour point is randomly selected from the collected discrete contour points, and the contour point closest to that contour point (such as Euclidean distance) is found. The connection between these two contour points is used as the initial baseline.

[0144] Step 1042: Locate the contour point closest to the baseline on the right side of the baseline.

[0145] Based on the established baseline, the distance between any contour point and the baseline is calculated to the right of the baseline. That is, the distance when the contour point is vertically projected onto the baseline. The distances between any contour point and the baseline are compared to find the contour point closest to the baseline.

[0146] Step 1043: Connect the two contour points on the baseline with the found contour points to obtain a color block in the shape of a triangle.

[0147] If the contour point closest to the baseline has been found in this recursion, then the two contour points on the baseline can be connected to the contour point found in this recursion to obtain a color block in the shape of a triangle.

[0148] Step 1044: Determine whether all contour points in the original image data have been traversed; if yes, proceed to step 1045; otherwise, proceed to step 1046.

[0149] Step 1045: Output color blocks.

[0150] Step 1046: Use the line connecting the found contour points as the new baseline and return to step 1041.

[0151] In each recursion, if the color block construction is completed, it can be determined whether all contour points in the original image data have been traversed. If all contour points in the original image data have been traversed, all constructed color blocks can be output. If all contour points in the original image data have not been traversed, the line connecting the contour point found in this recursion with two contour points on the baseline can be set as the new baseline, and the next recursion can be entered until all contour points in the original image data have been traversed.

[0152] Step 105: Using the colors of each element in the original image data as a reference, fill the color blocks with a uniform color to obtain the target image data.

[0153] In the actual implementation, the color block is not initially filled with color. At this time, as shown... Figure 2 As shown, the colors of each element in the original image data can be used as a reference to fill each color block with color, thereby generating the target image data. This allows the color blocks to maintain the general outline of each element in the original image data, as well as the general color of each element in the original image data relative to the position of the color block. This ensures that the content of the target image data maintains the general texture of the original image data and also achieves the style of a low-polygon.

[0154] Furthermore, while the same color may be used to fill the same color block, different colors may be used to fill different color blocks.

[0155] In one embodiment of the present invention, step 105 may include the following steps:

[0156] Step 1051: For each color block, query the pixel points in the original image data that represent each element within the color block.

[0157] In this embodiment, all color blocks are mapped back to the original image data, so the pixels located in the color blocks can be queried in the original image data, and these pixels represent the various elements in the original image data.

[0158] Step 1052: In the color block, cluster the pixels according to their color values ​​to obtain multiple candidate clusters.

[0159] Within each color block, the color value of a pixel can be used as its feature, thereby clustering the pixels based on their color values ​​to obtain multiple clusters, which are denoted as candidate clusters.

[0160] In one embodiment of the present invention, step 1052 may further include the following steps:

[0161] Step 10521: Initialize multiple candidate clusters.

[0162] During clustering, multiple clusters can be initialized as candidate clusters. The number of candidate clusters can be a default empirical value. Each candidate cluster has a center point. The center point can be initially set randomly, or it can be a point that is as far apart as possible from each other. Alternatively, the color values ​​of the pixels can be clustered using a hierarchical clustering algorithm or the Canopy algorithm to obtain multiple reference clusters. Then, a point can be selected from each reference cluster as the center point. This center point can be the center of the reference cluster, or the point closest to the center of the reference cluster, etc. This embodiment does not impose any restrictions on this.

[0163] Step 10522: Calculate the difference between the color value of the pixel and the center point.

[0164] In each round of clustering, each pixel is treated as a point in a candidate cluster, and its features (i.e., color value) are used to calculate the difference (i.e., distance) between the point and the center point, such as Euclidean distance, cosine distance, etc.

[0165] Step 10523: Draw the pixel into the center point with the smallest difference.

[0166] For a given pixel, the difference (i.e., distance) between the pixel and the center point of each candidate cluster can be compared. The candidate cluster with the smallest difference (i.e., distance) is selected as the candidate cluster to which the pixel belongs, thus assigning the pixel to the candidate cluster with the smallest difference (i.e., distance).

[0167] Step 10524: In each candidate cluster, calculate the average color value of all pixels and use it as the new center point.

[0168] After each pixel is reassigned to a candidate cluster, each candidate cluster contains multiple pixels. The average color value of the multiple pixels is calculated and assigned to the center point of the candidate cluster, thereby updating the center point of the candidate cluster.

[0169] Step 10525: Determine whether the change in the center point is less than or equal to the preset second threshold; if yes, proceed to step 10526; otherwise, return to proceed to steps 10522-10525.

[0170] Step 10526: Determine if the candidate clusters converge.

[0171] For the same candidate cluster, the difference between the center point before and after the update can be calculated as the change magnitude during the update, and the change magnitude during the update can be compared with a preset second threshold.

[0172] If the change during the update is less than or equal to the second threshold, it indicates that the change in the center point update is small, and the candidate cluster can be confirmed to have converged, thus completing the clustering.

[0173] If the change during the update is greater than the second threshold, it indicates that the change in the center point update is large, and it can be confirmed that the candidate cluster has not converged. Enter the next round of clustering and repeat steps 10522-10525 until the candidate cluster converges.

[0174] Step 1053: Select one of the candidate clusters from the multiple candidate clusters as the target cluster.

[0175] In this embodiment, one of the candidate clusters can be selected from multiple candidate clusters according to the needs of the business, and it is denoted as the target cluster.

[0176] Generally, the candidate cluster with the highest number of pixels is selected as the target cluster. This makes the color value filled in the color block closer to the color value in the original image data.

[0177] Of course, if the difference in color value between adjacent color blocks is taken into account, and the difference between color blocks is highlighted, the candidate cluster with the highest number of pixels can be selected as the target cluster under the condition that the color values ​​of adjacent color blocks are different. This embodiment does not limit this.

[0178] Step 1054: Fill the color values ​​represented by the target cluster into the color blocks to obtain the target image data.

[0179] For a target cluster, its characteristic color value can be filled into a color block to achieve coloring of the color block. After each color block is colored, the target image data is generated.

[0180] Generally, the color value representing the center point of the target cluster can be filled into the color block to obtain the target image data.

[0181] When the color blocks in the target image data are triangular or similar shapes, a rendering engine can be used to accelerate the rendering process. Therefore, when drawing color blocks, in addition to defining geometric coordinates (vertex coordinates) for each vertex (i.e., contour point), texture coordinates must also be defined. After various transformations, the geometric coordinates determine the position of the vertex on the screen, while the texture coordinates determine which texel in the texture image is assigned to that vertex.

[0182] Texture images are square arrays. Texture coordinates are typically defined in one, two, three, or four dimensions, referred to as s, t, r, and q coordinates. One-dimensional textures are commonly represented by s coordinates, while two-dimensional textures are commonly represented by (s, t) coordinates. The r coordinate is currently ignored. The q coordinate, like the w coordinate, is generally 1 and is mainly used to establish homogeneous coordinates. The OpenGL coordinate definition function is:

[0183] void gltexCoord{1234}{sifd}[v](TYPE coords)

[0184] This sets the current texture coordinates, and all vertices generated by subsequent calls to glVertex*() will be assigned these current texture coordinates. For gltexCoord1*(), the s coordinate is set to the given value, t and r are set to 0, and q is set to 1; gltexCoord2*() sets the s and t coordinate values, r to 0, and q to 1; for gltexCoord3*(), q is set to 1, and other coordinates are set to the given values; gltexCoord4*() provides all the coordinates.

[0185] In this embodiment, OpenGL ES is used as an example to explain the process of drawing color blocks. This process is a programmable pipeline and specifically includes the following operations:

[0186] 1. VBO / VAO (Vertex Buffer / Arrays Objects)

[0187] VBO / VAO is vertex information provided by the CPU to the GPU, including vertex coordinates, color (only the color of the vertex, unrelated to the color of the texture), texture coordinates (used for texture mapping), etc.

[0188] 2. Vertex Shader

[0189] A vertex shader is a program that processes vertex information provided by a VBO / VAO. The vertex shader is executed once for each vertex provided by the VBO / VAO. Uniforms (a variable type) remain consistent across all vertices, while attributes are different for each vertex (which can be understood as input vertex properties). Each execution of the VertexShader outputs a Varying (variable variable) and a gl_position.

[0190] The inputs to the vertex shader include:

[0191] 2.1 Shader Program: The source code or executable file of the vertex shader program that describes the operations performed on the vertices.

[0192] 2.2 Vertex Shader Input (or Attributes): Data for each vertex provided by the vertex array.

[0193] 2.3 Uniform variables: Immutable data used by vertex / fragment shaders

[0194] 2.4 Samplers: A special uniform variable type representing the textures used by the vertex shader.

[0195] VertexShader is the stage that vertex shader programming can operate on, used to control the transformation process of vertex coordinates, while fragment shader controls the calculation process of color for each pixel.

[0196] 3. Primitive Assembly:

[0197] The next stage after the vertex shader is primitive assembly. Primitives are geometric objects such as triangles, lines, or points. In this stage, the vertices output by the vertex shader are combined into primitives.

[0198] The vertex data is used to reconstruct the mesh structure based on the Primitive (original link relationship). The mesh consists of vertices and indices. At this stage, the vertices are linked together according to the indices to form three different primitives: points, lines, and faces. Then, triangles that exceed the screen are clipped.

[0199] For example, if one of the three vertices of a triangle (grid) is outside the screen and the other two are inside the screen, what is actually seen on the screen should be a quadrilateral. This quadrilateral can be cut into two smaller triangles (grids).

[0200] In short, the points obtained after the vertex shader calculation are combined into points, lines, and faces (triangles) based on their connection relationships.

[0201] 4. Rasterization

[0202] Rasterization is the process of converting primitives into a set of two-dimensional fragments, which are then processed by a fragment shader (the input to the fragment shader). These two-dimensional fragments represent pixels that can be drawn on the screen, and the mechanism used to generate each fragment value from the vertex shader output assigned to each primitive vertex is called interpolation.

[0203] After primitives are assembled, vertices can be understood as becoming graphics. During rasterization, the pixels (texture coordinates v_texCoord, color, etc.) of that graphic region can be interpolated based on the shape of the graphic. Note that at this point, the pixels are not the pixels on the screen and do not have color. The subsequent fragment shader performs the coloring work.

[0204] 5. FragmentShader

[0205] Fragment shaders provide a general programmable approach to operations on fragments (pixels). The fragment shader is executed once for each fragment in the rasterization output. This shader is executed for each fragment generated during the rasterization stage, generating one or more (multi-rendering) color values ​​as output.

[0206] 6. Per-Fragment Operations

[0207] During this phase, each segment will perform the following 5 operations:

[0208] 6.1 PixelOwnership Test

[0209] Determine whether the pixel at position (x, y) in the framebuffer belongs to the current context.

[0210] For example, if a display framebuffer window is obscured by another window, the window system can determine that the obscured pixels do not belong to the context of this OpenGL and therefore do not display those pixels.

[0211] 6.2 Scissor Test:

[0212] If the segment is outside the clipping area, it is discarded.

[0213] 6.3 Stencil Test and Depth Test:

[0214] If the shape returned by the fragment shader is not the shape in the template, it is discarded.

[0215] If the depth returned by the fragment shader is less than the depth in the buffer, it is discarded.

[0216] 6.4 Blending:

[0217] The newly generated fragment color values ​​are combined with the color values ​​stored in the frame buffer to produce a new RGBA (Red, Green, Blue, and Alpha color space).

[0218] 6.5. Dithering:

[0219] At the end of the fragment-by-fragment operation phase, a fragment is either rejected or its color, depth, or stencil value is written to a location somewhere in the framebuffer (x, y). The color, depth, and stencil value written depends on the corresponding write mask. The write mask allows for more precise control over the color, depth, and stencil values ​​written to the relevant buffer. For example, a write mask can be set for the color buffer so that no red value can be written to it.

[0220] Finally, the generated fragments are placed in the frame buffer (front buffer, back buffer, or FBO (Frame Buffer Object)). If it is not an FBO, the fragments in the screen drawing buffer are used to generate pixels on the screen.

[0221] In this embodiment, the original image data is acquired; the contours of each element in the original image data are detected; contour points representing color changes are detected in the contours; multiple color blocks are constructed in the original image data based on the contour points; and the color blocks are filled with a uniform color using the colors of each element in the original image data as a reference, thus obtaining the target image data. This embodiment uses contours and color changes to construct color blocks and fills the color blocks with color with reference to the colors of the original image data to achieve a low-polygon style. This process does not rely on professional editing tools, has a low technical threshold, and does not rely on neural networks. The entire process is simple to operate and has low computational load, which can greatly reduce resource consumption and improve the speed and efficiency of stylization. Furthermore, while maintaining the approximate contours of each element in the original image data, the color blocks also maintain the approximate colors of each element in the original image data relative to the position of the color block. This ensures that the content of the target image data maintains the approximate texture of the original image data, and the contours, color changes, and color blocks have strong resolution. The changes in the structure and color of the image during stylization are predictable, making it easier for users to find suitable stylization effects.

[0222] Example 2

[0223] Figure 3This is a flowchart of a stylization method provided in Embodiment 2 of the present invention. This embodiment is applicable to situations where color blocks are constructed based on contours and color changes to achieve a low-polygon style for video data. This method can be executed by a stylization device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 3 As shown, the method includes:

[0224] Step 301: Obtain the original video data that introduces the game.

[0225] In this embodiment, the computer program for stylizing low-polygons is relatively simple. Therefore, the computer program for stylizing low-polygons can be deployed on the server side to provide low-polygon stylization services to users on the local area network and / or the public network via API (Application Programming Interface), or it can be deployed on the client side as a plugin, hard-coded, etc. to provide low-polygon stylization services to client users. This embodiment does not limit this.

[0226] If the computer program that implements the stylization of low-polygon is deployed on the server, then the user can call the API provided by the server in the client to upload the video data to be stylized to the server, or provide the network address of the video data to be stylized to the server, and the server will download the video data to be stylized based on the network address.

[0227] If the computer program that implements the stylization of low-polygons is deployed on the server, then users can select video data located in the local directory on the client and wait for stylization, or call the camera to capture video data and wait for stylization.

[0228] For ease of distinction, the video data to be stylized is recorded as the original video data, the content of which is mainly used to introduce the game and thus promote the game.

[0229] The types of games can include MOBA (Multiplayer Online Battle Arena), RPG (Role-playing game), SLG (Simulation Game), etc., and this embodiment does not limit them.

[0230] From a human perspective, the content of raw video data can be divided into two main forms: game content and real-life storylines. Game content can include introductions to the user's control of the game, introductions by a spokesperson, or introductions by a spokesperson wearing in-game costumes. Storylines can be further categorized as follows:

[0231] 1. Pseudo-food sharing

[0232] The original video data included some food-related materials to attract users' attention and incorporate gameplay elements of eating food while playing games.

[0233] 2. Themes closely related to users' daily lives

[0234] The content of the raw video data closely reflects users' current lifestyles, integrating games into various aspects of life, such as playing games while eating or buying snacks. The first half of this type of material primarily features dialogue between two people, while the second half includes segments showcasing the game integration.

[0235] 3. Exaggerated situational drama

[0236] The raw video data contains material from sitcoms, some of which features exaggerated storylines designed to attract users' attention.

[0237] Of course, the above-described raw video data is merely an example. When implementing this embodiment, other raw video data can be set according to actual circumstances, and this embodiment does not impose any limitations on this. Furthermore, in addition to the above-described raw video data, those skilled in the art can use other raw video data as needed, and this embodiment does not impose any limitations on this either.

[0238] Step 302: Detect the contours of each element in the original image data.

[0239] In the specific implementation, the original video data contains multiple frames of image data, referred to as the original image data. In each frame of image data, the contours of each element can be detected.

[0240] When detecting contours, grayscale values ​​are calculated from the original image data to obtain grayscale image data; gradient values ​​are calculated from the grayscale image data to obtain gradient image data; normalization is performed on the gradient values ​​in the gradient image data; and the absolute value of the normalized gradient values ​​in the gradient image data is taken to obtain the contour of each element.

[0241] Furthermore, when calculating the gradient value, a first convolution kernel and a second convolution kernel are loaded respectively; the first convolution kernel is used to perform a first convolution operation on the grayscale image data in the horizontal direction to obtain the gradient value in the horizontal direction; the second convolution kernel is used to perform a second convolution operation on the grayscale image data in the vertical direction to obtain the gradient value in the vertical direction; the gradient value in the horizontal direction and the gradient value in the vertical direction are fused together to obtain the gradient image data.

[0242] Step 303: Detect contour points that represent color changes in the contour.

[0243] When detecting contour points, a histogram is calculated for the grayscale image data, which contains the grayscale values ​​of the original image data; the cumulative distribution probability is calculated for the histogram; the original image data is mapped to reference image data based on the cumulative distribution probability; in the reference image data, pixels whose color values ​​meet the preset change conditions are selected along the contour as contour points representing color changes.

[0244] In addition, before filtering out contour points, any two contour points can be constructed into a point pair; for any point pair, the distance between the contour points in the point pair is calculated; if the distance is greater than or equal to a preset first threshold, the contour points in the point pair are retained; if the distance is less than the preset first threshold, any contour point in the point pair is filtered out.

[0245] The contour points detected between different frames of original image data will have certain fluctuations, which will cause the color blocks to flicker noticeably. In order to reduce the flicker, in this embodiment, the contour points between frames can be smoothed.

[0246] In one smoothing method, contour points belonging to the same position are found in two adjacent frames of original image data. If the number of contour points is determined and they have been assigned numbers, then contour points with the same number are contour points with the same position. Alternatively, if the position of the contour point in the previous frame is determined, the contour point closest to that position is found in the next frame and determined to be the contour point with the same position, and so on.

[0247] For contour points with the same position, the adjusted contour points in the previous frame of the original image data are linearly fused with the original contour points in the next frame of the original image data to obtain the adjusted contour points in the next frame of the original image data.

[0248] That is, the position of the adjusted contour point in the previous frame of original image data is assigned a first weight, and the position of the original contour point in the next frame of original image data is assigned a second weight, wherein the second weight is greater than the first weight, so that the position of the adjusted contour point in the next frame of original image data is biased towards the position of the original contour point in the next frame of original image data.

[0249] Calculate the first product between the adjusted contour point position in the previous frame of original image data and the first weight, calculate the second product between the original contour point position in the next frame of original image data and the second weight, and add the first product and the second product together to obtain the adjusted contour point in the next frame of original image data.

[0250] Step 304: Construct multiple adjacent color blocks in the original image data based on the contour points.

[0251] When constructing color blocks, the contour points are traversed in the original image data, and three adjacent contour points are connected to construct multiple color blocks with triangular shapes, where the triangles are adjacent to each other and do not overlap. Among them, the circumcircle of each color block does not contain other contour points, and the Thiessen polygons corresponding to the contour points that serve as vertices of the color block have a common vertex, which is the center of the circumcircle of the color block.

[0252] Furthermore, a baseline can be determined. The baseline is initially the line connecting any contour point in the original image data to the nearest other contour point. The nearest contour point to the baseline is found on the right side of the baseline. The two contour points on the baseline are connected to the found contour points to obtain triangular color blocks. It is determined whether all contour points in the original image data have been traversed. If yes, the color block is output. If no, the line containing the found contour points is used as the new baseline, and the process returns to determine the baseline.

[0253] Step 305: Using the colors of each element in the original image data as a reference, fill the color blocks with a uniform color to obtain the target image data.

[0254] When filling in the color, for each color block, the pixel that represents each element in the color block is queried in the original image data; in the color block, the pixels are clustered according to the color value of the pixels to obtain multiple candidate clusters; one of the candidate clusters is selected as the target cluster; the color value represented by the target cluster is filled into the color block to obtain the target image data.

[0255] During clustering, multiple candidate clusters are initialized, each with a center point; the difference between the color value of a pixel and the center point is calculated; the pixel is assigned to the center point with the smallest difference; in each candidate cluster, the average color value of all pixels is calculated and used as the new center point; it is determined whether the change in the center point is less than or equal to a preset second threshold; if so, the candidate cluster is considered to have converged; otherwise, the process returns to calculate the difference between the color value of the pixel and the center point.

[0256] Accordingly, when filling in the color, the color value represented by the center point of the target cluster is filled into the color block to obtain the target image data.

[0257] In this embodiment, since the application of low-polygon style for single-frame raw image data is basically similar to that in Embodiment 1, the description is relatively simple. For relevant details, please refer to the description in Embodiment 1. This embodiment will not be described in detail here.

[0258] Step 306: Replace the original image data with the target image data in the original video data to obtain the target video data.

[0259] In the original video data, the target image data can be replaced with the corresponding original image data to obtain the target video data.

[0260] Subsequently, game-related advertising elements are added to the target video data as advertising video data. The advertising video data is then published on designated channels (such as news, short videos, novel reading, sports and health, etc.) so that when the client accesses the channel, the advertising video data is pushed to the client for playback. When the user is interested in the target game, they can download the target game from the platform that distributes the game.

[0261] Furthermore, advertising element data can include icons (logos), banner information, ending cards (ECs), etc.

[0262] The icon (Logo) is the identifier of the business object itself, and it can be a text icon (containing the name of the business object) or a graphic icon.

[0263] Banner information is generally rectangular and is usually located at the top and / or bottom of image data. It can record information about the business object itself (such as a scene in a game, a character in the game, or a name) or information to attract users to purchase or download the business object (such as a gift code).

[0264] The end segment EC contains the identifier of the download business object, such as the business object's own information (e.g., in-game graphics, characters, and names), and the method of purchasing or downloading the business object (e.g., the icon of the application distribution platform, the name and icon of the application distribution platform, the name and icon of the shopping platform, etc.).

[0265] In this embodiment, the content to be acquired is the original video data introducing the game, which contains multiple frames of original image data; the contours of each element are detected in the original image data; contour points representing color changes are detected in the contours; multiple adjacent color blocks are constructed in the original image data based on the contour points; the color blocks are filled with a uniform color using the colors of each element in the original image data as a reference to obtain the target image data; the target image data is then used to replace the original image data in the original video data to obtain the target video data. This implementation uses contour and color variations to construct color blocks, and fills the color blocks with color based on the original image data to achieve a low-polygon style. This process does not rely on professional editing tools, has a low technical threshold, and does not depend on neural networks. The entire process is simple to operate and computationally intensive, significantly reducing resource consumption and increasing stylization speed and efficiency. Furthermore, while maintaining the approximate contours of each element in the original image data, the color blocks also maintain the approximate colors of each element in relation to the position of the color block. This ensures that the target image data retains the approximate texture of the original image data, with strong resolution in contours, color variations, and color blocks. The changes in the structure and color of the image during stylization are predictable, making it easier for users to find suitable stylization effects.

[0266] Example 3

[0267] Figure 4 This is a schematic diagram of the structure of a stylization device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes:

[0268] The raw image data acquisition module 401 is used to acquire raw image data;

[0269] Contour detection module 402 is used to detect the contours of each element in the original image data;

[0270] Contour point detection module 403 is used to detect contour points representing color changes in the contour.

[0271] Color block construction module 404 is used to construct multiple color blocks in the original image data based on the contour points;

[0272] The color filling module 405 is used to fill the color block with a uniform color with reference to the colors of each element in the original image data to obtain the target image data.

[0273] In one embodiment of the present invention, the contour detection module 402 is further configured to:

[0274] The grayscale values ​​are calculated from the original image data to obtain grayscale image data;

[0275] The gradient value is calculated on the grayscale image data to obtain gradient image data;

[0276] Normalization is performed on the gradient values ​​in the gradient image data;

[0277] The absolute value of the gradient value after normalization is taken in the gradient image data to obtain the contour of each element.

[0278] In one embodiment of the present invention, the contour detection module 402 is further configured to:

[0279] Load the first convolution kernel and the second convolution kernel respectively;

[0280] The first convolution kernel is used to perform a first convolution operation on the grayscale image data in the horizontal direction to obtain the gradient value in the horizontal direction;

[0281] The second convolution kernel is used to perform a second convolution operation on the grayscale image data in the vertical direction to obtain the gradient value in the vertical direction;

[0282] The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are fused together to obtain the gradient value of the grayscale image data, thus obtaining gradient image data.

[0283] In one embodiment of the present invention, the contour point detection module 403 is further configured to:

[0284] Calculate a histogram for grayscale image data, wherein the grayscale image data includes the grayscale values ​​of the original image data;

[0285] Calculate the cumulative distribution probability for the histogram;

[0286] The original image data is mapped to reference image data based on the cumulative distribution probability.

[0287] In the reference image data, pixels whose color values ​​meet preset change conditions are selected along the contour and used as contour points representing color changes.

[0288] In one embodiment of the present invention, the contour point detection module 403 is further configured to:

[0289] Construct a point pair from any two of the aforementioned contour points;

[0290] For any given pair of points, calculate the distance between the contour points in the pair;

[0291] If the distance is greater than or equal to a preset first threshold, then the contour points in the point pair are retained;

[0292] If the distance is less than a preset first threshold, then any of the contour points in the point pair are filtered out.

[0293] In one embodiment of the present invention, the color block construction module 404 is further configured to:

[0294] Traverse the contour points in the original image data, connect three adjacent contour points, and construct multiple color blocks in the shape of triangles, wherein the triangles are adjacent to each other and do not overlap.

[0295] In this context, the circumcircle of each color block does not contain any other contour points, and the Thiessen polygons corresponding to the contour points that serve as vertices of the color blocks have a common vertex, which is the center of the circumcircle of the color block.

[0296] In one embodiment of the present invention, the color block construction module 404 is further configured to:

[0297] Determine a baseline, which is initially a line connecting any contour point in the original image data to the other contour point closest to that contour point.

[0298] Locate the contour point that is closest to the baseline on the right side of the baseline;

[0299] Connect the two contour points on the baseline to the found contour points to obtain a color block in the shape of a triangle;

[0300] Determine whether all contour points in the original image data have been traversed; if yes, output the color block; if no, take the line containing the found contour points as the new baseline and return to execute the baseline determination.

[0301] In one embodiment of the present invention, the color filling module 405 is further configured to:

[0302] For each color patch, query the original image data for the pixel points that represent each element located within the color patch;

[0303] Within the color block, the pixels are clustered based on their color values ​​to obtain multiple candidate clusters;

[0304] Select one of the candidate clusters from the plurality of candidate clusters as the target cluster;

[0305] The color values ​​represented by the target cluster are filled into the color blocks to obtain the target image data.

[0306] In one embodiment of the present invention, the color filling module 405 is further configured to:

[0307] The pixel points are clustered based on their color values ​​to obtain multiple candidate clusters, including:

[0308] Initialize multiple candidate clusters, each candidate cluster having a center point;

[0309] Calculate the difference between the color value of the pixel and the center point;

[0310] The pixel is assigned to the center point with the smallest difference.

[0311] In each candidate cluster, the average color value of all the pixels is calculated and used as the new center point;

[0312] Determine whether the change in the center point is less than or equal to a preset second threshold; if yes, then determine that the candidate cluster has converged; if no, then return to perform the calculation of the difference between the color value of the pixel and the center point.

[0313] In one embodiment of the present invention, the color filling module 405 is further configured to:

[0314] The color value representing the center point of the target cluster is filled into the color block to obtain the target image data.

[0315] The stylization apparatus provided in the embodiments of the present invention can execute the stylization method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the stylization method.

[0316] Example 4

[0317] Figure 5 This is a schematic diagram of the structure of a stylization device provided in Embodiment 4 of the present invention. Figure 5 As shown, the device includes:

[0318] The raw video data acquisition module 501 is used to acquire raw video data containing game introduction content, wherein the raw video data contains multiple frames of raw image data;

[0319] Contour detection module 502 is used to detect the contours of each element in the original image data;

[0320] Contour point detection module 503 is used to detect contour points representing color changes in the contour.

[0321] The color block construction module 504 is used to construct multiple adjacent color blocks in the original image data based on the contour points;

[0322] Color filling module 505 is used to fill the color block with a uniform color with reference to the color of each element in the original image data to obtain target image data;

[0323] The target video data generation module 506 is used to replace the original image data with the target image data in the original video data to obtain target video data.

[0324] In one embodiment of the present invention, it further includes:

[0325] The contour point finding module is used to find contour points that belong to the same position in two adjacent frames of the original image data;

[0326] The contour point smoothing module is used to linearly fuse the adjusted contour points in the previous frame of the original image data with the original contour points in the next frame of the original image data for contour points with the same position, so as to obtain the adjusted contour points in the next frame of the original image data.

[0327] In one embodiment of the present invention, it further includes:

[0328] An advertising video data generation module is used to add game-related advertising element data to the target video data as advertising video data.

[0329] The advertising video data publishing module is used to publish the advertising video data on a designated channel, so that when the client accesses the channel, the advertising video data is pushed to the client for playback.

[0330] The stylization apparatus provided in the embodiments of the present invention can execute the stylization method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the stylization method.

[0331] Example 5

[0332] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0333] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0334] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0335] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as stylization methods.

[0336] In some embodiments, the stylization method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the stylization method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the stylization method by any other suitable means (e.g., by means of firmware).

[0337] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0338] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0339] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0340] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0341] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0342] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0343] Example 6

[0344] This invention also provides a computer program product comprising a computer program that, when executed by a processor, implements the stylization method provided in any embodiment of this invention.

[0345] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0346] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0347] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A stylization method, characterized in that, include: Obtain the raw image data; The contours of each element are detected in the original image data; Detect contour points representing color changes within the contour; Traverse the contour points in the original image data to determine the baseline, which is initially a line connecting any contour point in the original image data to the other contour point that is closest to any contour point. Find any of the contour points that are closest to the baseline to the right of the baseline; Connect the two contour points on the baseline to the found contour points to obtain a color block in the shape of a triangle; Determine whether all contour points in the original image data have been traversed; if so, output the color block. If not, the line connecting the found contour points will be used as the new baseline, and the process of determining the baseline will be returned. Using the colors of each element in the original image data as a reference, the color blocks are filled with a uniform color to obtain the target image data; The step of detecting the contours of each element in the original image data includes: The grayscale values ​​are calculated from the original image data to obtain grayscale image data; The gradient value is calculated on the grayscale image data to obtain gradient image data; Normalization is performed on the gradient values ​​in the gradient image data; The absolute value of the gradient value after normalization is taken in the gradient image data to obtain the contour of each element. The step of detecting contour points representing color changes in the contour includes: Calculate a histogram for grayscale image data, wherein the grayscale image data includes the grayscale values ​​of the original image data; Calculate the cumulative distribution probability for the histogram; The original image data is mapped to reference image data based on the cumulative distribution probability. In the reference image data, pixels whose color values ​​meet preset change conditions are selected along the contour and used as contour points representing color changes. The step of filling the color blocks with a uniform color using the colors of each element in the original image data as a reference to obtain the target image data includes: Map all the color blocks back to the original image data. For each color block, query the pixel located in the original image data. The pixel represents each element in the original image data. Within the color block, the pixels are clustered based on their color values ​​to obtain multiple candidate clusters; Select one of the candidate clusters from the plurality of candidate clusters as the target cluster; The color values ​​represented by the target cluster are filled into the color blocks to obtain the target image data.

2. The method according to claim 1, characterized in that, The step of calculating gradient values ​​from the grayscale image data to obtain gradient image data includes: Load the first convolution kernel and the second convolution kernel respectively; The first convolution kernel is used to perform a first convolution operation on the grayscale image data in the horizontal direction to obtain the gradient value in the horizontal direction; The second convolution kernel is used to perform a second convolution operation on the grayscale image data in the vertical direction to obtain the gradient value in the vertical direction; The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are fused together to obtain the gradient value of the grayscale image data, thus obtaining gradient image data.

3. The method according to claim 1, characterized in that, The step of detecting contour points characterizing color changes in the contour further includes: Construct a point pair from any two of the aforementioned contour points; For any given pair of points, calculate the distance between the contour points in the pair; If the distance is greater than or equal to a preset first threshold, then the contour points in the point pair are retained; If the distance is less than a preset first threshold, then any of the contour points in the point pair are filtered out.

4. The method according to claim 1, characterized in that, The pixel points are clustered based on their color values ​​to obtain multiple candidate clusters, including: Initialize multiple candidate clusters, each candidate cluster having a center point; Calculate the difference between the color value of the pixel and the center point; The pixel is assigned to the center point with the smallest difference. In each candidate cluster, the average color value of all the pixels is calculated and used as the new center point; Determine whether the change range of the center point is less than or equal to a preset second threshold; if yes, then determine that the candidate cluster has converged; if no, then return to the process of calculating the difference between the color value of the pixel and the center point, assigning the pixel to the center point with the smallest difference, and calculating the average of the color values ​​of all the pixels in each candidate cluster as the new center point. The step of filling the color patch with the color value represented by the target cluster to obtain target image data includes: The color value representing the center point of the target cluster is filled into the color block to obtain the target image data.

5. A stylization method, characterized in that, include: The content to be acquired is the original video data introducing the game, which contains multiple frames of original image data; The contours of each element are detected in the original image data; Detect contour points representing color changes within the contour; Traverse the contour points in the original image data to determine the baseline, which is initially a line connecting any contour point in the original image data to the other contour point that is closest to any contour point. Find any of the contour points that are closest to the baseline to the right of the baseline; Connect the two contour points on the baseline to the found contour points to obtain a color block in the shape of a triangle; Determine whether all contour points in the original image data have been traversed; if so, output the color block. If not, the line connecting the found contour points will be used as the new baseline, and the process of determining the baseline will be returned. Using the colors of each element in the original image data as a reference, the color blocks are filled with a uniform color to obtain the target image data; The target image data is replaced in the original video data to obtain the target video data; The step of detecting the contours of each element in the original image data includes: The grayscale values ​​are calculated from the original image data to obtain grayscale image data; The gradient value is calculated on the grayscale image data to obtain gradient image data; Normalization is performed on the gradient values ​​in the gradient image data; The absolute value of the gradient value after normalization is taken in the gradient image data to obtain the contour of each element. The step of detecting contour points representing color changes in the contour includes: Calculate a histogram for grayscale image data, wherein the grayscale image data includes the grayscale values ​​of the original image data; Calculate the cumulative distribution probability for the histogram; The original image data is mapped to reference image data based on the cumulative distribution probability. In the reference image data, pixels whose color values ​​meet preset change conditions are selected along the contour and used as contour points representing color changes. The step of filling the color blocks with a uniform color using the colors of each element in the original image data as a reference to obtain the target image data includes: Map all the color blocks back to the original image data. For each color block, query the pixel located in the original image data. The pixel represents each element in the original image data. Within the color block, the pixels are clustered based on their color values ​​to obtain multiple candidate clusters; Select one of the candidate clusters from the plurality of candidate clusters as the target cluster; The color values ​​represented by the target cluster are filled into the color blocks to obtain the target image data.

6. The method according to claim 5, characterized in that, Also includes: In two adjacent frames of the original image data, find the contour points that are at the same location. For contour points with the same position, the adjusted contour points in the previous frame of the original image data are linearly fused with the original contour points in the next frame of the original image data to obtain the adjusted contour points in the next frame of the original image data.

7. The method according to claim 5 or 6, characterized in that, Also includes: Add game-related advertising element data to the target video data to create advertising video data. The advertising video data is published on a designated channel so that when a client accesses the channel, the advertising video data is pushed to the client for playback.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the stylization method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that enables a processor to implement the stylization method according to any one of claims 1-7 when executed.

Images