Image processing method and device, storage medium and electronic equipment

Abstract

The application discloses an image processing method and device, a storage medium and an electronic device. The method comprises the following steps: acquiring a line drawing image to be processed, wherein the line drawing image comprises at least one object; determining at least one contour line corresponding to the object and an initial height level corresponding to each contour line; determining a target height value of each pixel point in each contour line according to the initial height level and the contour line; and generating a shadow rendering image of the object according to the target height value and the line drawing image. Thus, the shadow effect of various types of objects in the line drawing image can be accurately drawn, the application range is wide, and the shadow processing effect is good.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the booming development of the game and comic industries, the update speed of game and comic products is gradually accelerating. The update of game and comic products mainly includes two processes: drawing line art images and coloring the line art images.

[0003] When coloring line art, considering the lighting and shadow effects in the scene, shadow areas are usually identified and colored to give the line art a shadow effect. Currently, pre-trained neural network models are mainly used to automatically identify and color shadow areas in line art images. However, because neural network models need to learn from a large number of line art samples for different object types, such as faces and cars, the trained neural network models can only be used to identify and color shadow areas for known object types. They cannot handle unknown object types, resulting in limited applicability and poor shadow processing effects. Summary of the Invention

[0004] This application provides an image processing method, apparatus, storage medium, and electronic device that can be applied to the rendering of shadow effects for various objects, thereby improving the shadow processing effect.

[0005] This application provides an image processing method, including:

[0006] Obtain a line art image to be processed, wherein the line art image includes at least one object;

[0007] Determine at least one outline corresponding to the object, and the initial height level corresponding to each outline;

[0008] Based on the initial height level and the contour lines, determine the target height value of each pixel in each contour line;

[0009] Based on the target height value and the line drawing image, a shadow rendering image of the object is generated.

[0010] This application also provides an image processing apparatus, including:

[0011] An acquisition module is used to acquire a line drawing image to be processed, wherein the line drawing image includes at least one object;

[0012] The first determining module is used to determine at least one outline corresponding to the object, and the initial height level corresponding to each outline;

[0013] The second determining module is used to determine the target height value of each pixel in each contour line based on the initial height level and the contour line;

[0014] The generation module is used to generate a shadow rendering image of the object based on the target height value and the line drawing image.

[0015] This application also provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded by a processor to execute any of the above-described image processing methods.

[0016] This application also provides an electronic device, including a coupled memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to perform any of the above-described image processing methods.

[0017] The image processing method, apparatus, storage medium, and electronic device provided in this application acquire a line drawing image to be processed, which includes at least one object. The method determines at least one contour line corresponding to the object and the initial height level corresponding to each contour line. Based on the initial height level and the contour line, the method determines the target height value of each pixel in each contour line. Based on the target height value and the line drawing image, the method generates a shadow rendering image of the object. This allows for accurate rendering of shadow effects for various types of objects in the line drawing image, with a wide range of applications and good shadow processing effect. Attached Figure Description

[0018] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0019] Figure 1 This is a schematic flowchart of the image processing method provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram illustrating the line drawing image provided in an embodiment of this application.

[0021] Figure 3 This is another schematic flowchart of the image processing method provided in the embodiments of this application.

[0022] Figure 4 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application.

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0024] Figure 6 Another structural schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the embodiments described in this application are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that the terms "first," "second," etc., used in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and / or "including" as used herein specify the presence of the stated features, integers, steps, operations, units, and / or components, without excluding the presence or addition of one or more other features, integers, steps, operations, units, components, and / or combinations thereof.

[0027] This application provides an image processing method, apparatus, storage medium, and electronic device.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This image processing method is applied to electronic devices such as servers or terminals, and these devices are suitable for scenarios such as animation production, comic book production, and game development. The specific flow of this image processing method may include the following steps S101-S104, wherein:

[0029] S101. Obtain the line art image to be processed, which includes at least one object.

[0030] Line art images refer to uncolored images composed of lines. The objects in a line art image can be people, animals, or other types of objects. Line art images can be created using drawing software or photographed from paper line art drawings. Unlike RGB three-channel color images, line art images are generally single-channel images. Typically, the lines in a line art image are black, and the rest is white; that is, the pixels only have two colors: black and white. Black pixels (which can be considered non-blank pixels) can have a pixel value of 0, and white pixels (which can be considered blank pixels) can have a pixel value of 255 or 1. In some implementations, line art images can also be grayscale images converted from line drawings drawn with colored lines. In this case, the pixel values ​​range from 0 to 1 or from 0 to 255. Pixel values ​​of 1 or 255 represent blank pixels, and other pixel values ​​represent non-blank pixels.

[0031] S102. Determine at least one outline corresponding to the object, and the initial height level corresponding to each outline.

[0032] In this line art image, the shapes of each object are represented by outlines, which include outer and inner outlines. The lines in the line art image can be considered the outlines of each object. Please see [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating a line drawing image provided in an embodiment of this application. The object in the line drawing image is person A, and the black lines on it are the outline of person A. Since the line drawing image is in a two-dimensional planar coordinate system (such as the X-axis and Y-axis), while shadow rendering of the object needs to be performed in a world coordinate system (such as the X-axis, Y-axis, and Z-axis), the Z-axis can simulate the optical axis of a camera. The magnitude of the Z-axis value affects the final shadow effect. Therefore, an initial Z-axis value can be set for each pixel in the line drawing image, and different initial height levels correspond to different Z-axis values.

[0033] It should be noted that since line art images are generally drawn manually, there is a certain degree of error. Moreover, since the image is presented based on pixels, the outlines of various objects on the line art image may have visible or invisible discontinuities. That is, there are gaps between two adjacent pixels on the outline. In this case, if these pixels are directly connected by straight line segments to form the outline, it may result in a non-smooth polyline. The outline is unnatural and not conducive to the generation of subsequent shadow effects. To obtain a natural outline, line fitting can be performed based on these discontinuous pixels to obtain a smooth curve as the outline.

[0034] In some implementations, please refer to Figure 3 The above step S102 may specifically include the following steps S1021-S1023, wherein:

[0035] S1021. Perform an outer contour scan on the current object to obtain multiple target pixels and record the number of scans.

[0036] In this context, target pixels are non-blank pixels, meaning pixels whose pixel value is not 255 or 1. To obtain all the contour lines of an object, an approximation method can be used, scanning the outer contour of the object pixel by pixel from top to bottom and left to right, and fitting the corresponding contour lines based on the target pixels obtained in each scan.

[0037] Specifically, for the first scan, the outer contour of the object corresponds to the contour line of the outermost pixels whose pixel value is not 255 or 1. For subsequent scans, the outer contour of the object corresponds to the contour line of the outermost pixels among the remaining pixels after removing the target pixels obtained from previous scans. In other words, the number of pixels scanned in each scan is continuously decreasing. The target pixels obtained from each outer edge scan are the leftmost and rightmost pixels in each row of the remaining pixels whose pixel value is not 255 or 1. The pixels with values ​​of 255 or 1, and the top and bottom two pixels in each column whose values ​​are not 255 or 1. Of course, if there is only one pixel in a row whose value is not 255 or 1, then the leftmost and rightmost pixels are the same pixel. If there is only one pixel in a column whose value is not 255 or 1, then the top and bottom pixels are also the same pixel. If all the remaining pixels in a row or column are pixels with values ​​of 255 or 1, then the scan result for that row or column is empty.

[0038] S1022. Determine at least one contour line corresponding to the target pixel, and determine the initial height level corresponding to the contour line based on the number of scans.

[0039] In some implementations, the step of "determining at least one contour line corresponding to the target pixel" may specifically include:

[0040] Obtain the image coordinates of each target pixel;

[0041] Determine the coordinate distance between two adjacent target pixels based on image coordinates;

[0042] Line fitting is performed on the target pixel based on the coordinate distance and image coordinates to obtain at least one contour line.

[0043] In this context, "adjacent" refers to adjacent coordinate positions. Image coordinates are generally expressed in the form of (x, y), where x represents the coordinate value on the X-axis and y represents the coordinate value on the Y-axis. Since each outer contour scan may result in the outermost target pixels corresponding to multiple contour lines or a single contour line, and considering that the distance between adjacent target pixels on the same contour line is generally small, while the distance between any two target pixels on different contour lines is generally large, when performing line fitting, the target pixels can be grouped according to the coordinate distance between adjacent target pixels. Each group corresponds to one contour line. Adjacent target pixels with a large coordinate distance, such as greater than a set interval threshold, can be grouped together, thus obtaining at least one group.

[0044] When performing line fitting, it can be done in groups. During the fitting process, a parameter S can be introduced first, defined as:

[0045]

[0046] Where, Δx(j) = x(j) - x(j-1);

[0047] Δy(j) = y(j) - y(j-1);

[0048] Where i∈[2,n], j∈[2,n], n is the number of target pixels in each group, S (i) x is the length of the line (usually the length of a polyline) connecting the first target pixel to the i-th target pixel. (j) Let be the x-axis coordinate of the j-th target pixel, and y be the x-axis coordinate of the j-th target (j) Let be the Y-axis coordinate of the j-th target pixel.

[0049] During the fitting process, the target pixels in each group are treated as discrete points. The above formula is used to iterate through the data to obtain multiple S values. Each target pixel corresponds to one S value. Then, the parametric equations x = h(s) and y = g(s) are established. Since the S values ​​are an increasing sequence, h and g are easy to fit. Based on the fitting functions h and g, any S value can be input to obtain the corresponding x and y values. Then, the curve fitting tool can be used to form the corresponding contour line.

[0050] In some implementations, the step of "determining the initial height level corresponding to the contour line based on the number of scans" may specifically include:

[0051] Use the number of scans as the initial height level for the corresponding contour.

[0052] This process involves stacking the heights of corresponding contour lines layer by layer according to the scanning sequence. For at least one contour line obtained in the first scan, its initial height level can be set to 1, i.e., the first layer contour line. Subsequent scans will stack the height of at least one contour line on top of the previous layer, meaning the initial height level increases by 1 sequentially, such as the second layer contour line, the third layer contour line, and so on. It's easy to understand that the outer contour lines obtained in each scan can be closed or open. A closed line is one where the starting and ending points are the same target pixel, while an open line is one where the starting and ending points are different target pixels.

[0053] S1023. Take the remaining pixels in the object other than the target pixel as the current object, and return to execute the above step S1021 until the preset conditions are met.

[0054] The preset conditions can be that the current object has only one target pixel or no target pixel. When the preset conditions are met, it is considered that the object has completed the scanning of all contour lines.

[0055] S103. Determine the target height value of each pixel in each contour line based on the initial height level and contour lines.

[0056] The target height value can be seen as the coordinate value of the corresponding pixel on the Z-axis.

[0057] In some implementations, please refer to [link / reference]. Figure 3 The above step S103 may specifically include the following steps S1031-S1034, wherein:

[0058] S1031. Determine the line length of each contour line.

[0059] The line length, referred to as the S value, is the length of the line formed by connecting all target pixels in each group. For example, when there are n target pixels, the line length is S. (n) .

[0060] S1032. Determine the minimum distance between the contour lines corresponding to two adjacent initial height levels.

[0061] The minimum distance is the minimum perpendicular distance between two adjacent contour lines. In general, many perpendicular distances can be calculated between two contour lines (curves).

[0062] It should be noted that the above steps S1031 and S1032 can be performed simultaneously or sequentially, and there is no restriction here.

[0063] S1033. Determine the target height level for each contour line based on the line length, minimum distance, and initial height level.

[0064] Since the initial height level is set only based on the scanning order, it is difficult to reflect the actual height value (z-axis coordinate value) of the contour line. The initial height level needs to be adjusted according to the line length and minimum distance to obtain the target height level.

[0065] In some implementations, step S1033 may specifically include:

[0066] When the line length is less than the preset length threshold, the initial height level corresponding to the outline will be lowered.

[0067] When the minimum distance is less than the preset distance threshold, the initial height level with the larger value among the two initial height levels will be adjusted to be lowered.

[0068] The initial height level after the descent adjustment is used as the target height level of the corresponding contour line, and the remaining unadjusted initial height level is used as the target height level of the corresponding contour line.

[0069] The preset length threshold, preset distance threshold, and adjustment range are all manually set. Since the initial height level of the contour line has been increased by 1 increments according to the scanning order, the adjustment range for the height level can be 2 levels, which means subtracting 2 from the previously obtained initial height level value, thus changing the corresponding contour line's level from increasing by 1 level to decreasing by 1 level.

[0070] For example, for the contour obtained in the third scan, its initial height level is 3. Compared to the contour obtained in the second scan, the height level should have increased by one level. However, if the line length of the contour is less than a preset distance threshold, or if the minimum perpendicular distance between the contour and the contour obtained in the second scan is less than a preset distance threshold, then its initial height level is reduced by 2, from 3 to 1. Thus, its height level decreases by one level compared to the contour obtained in the second scan. Conversely, if the line length of the contour obtained in the third scan is greater than or equal to a preset length threshold, or if the minimum perpendicular distance between the contour and the contour obtained in the second scan is greater than or equal to a preset distance threshold, then the initial height level is not adjusted, thus its height level increases by one level compared to the contour obtained in the second scan. It should be noted that if multiple contours are obtained in a single scan, some contours may need to increase their height level by one level, while others may need to decrease their height level by one level.

[0071] S1034. Determine the target height value of each pixel in the corresponding contour line according to the target height level.

[0072] In some implementations, step S1034 may specifically include:

[0073] Determine the product between the preset height increment and the target height level;

[0074] Determine the sum between the product and the preset starting height value, and use this sum as the initial height value of each pixel in the corresponding contour line;

[0075] The initial height value is smoothed to obtain the target height value of each pixel in the corresponding contour line.

[0076] The preset height increment and preset starting height value can be manually set. For example, the preset height increment can be 1, meaning the height value increases by 1 for each additional height level. The preset starting height value can be set to 2 by default. The initial height value can be obtained by incrementing the preset starting height value according to the height levels. For example, the initial height value of each pixel in the first contour line is 2, the initial height level of each pixel in the second contour line is 3, and the initial height level of each pixel in the third contour line is 4. Since the initial height values ​​are discrete values, they can be smoothed to avoid unnatural shadow effects generated based on them.

[0077] Specifically, smoothing can be achieved using partial derivatives ddx and ddy. The specific smoothing formula is as follows:

[0078]

[0079] Where ρ is a single variable parameter, f() is the normal distribution function, and P(x) i ,y j ) represents the initial height value of the pixel in the i-th row and j-th column. The value of i is the range of values ​​in the row where the pixel is located in the corresponding outline, and the value of j is the range of values ​​in the column where the pixel is located in the corresponding outline. d is the distance to the four non-blank pixels (top, bottom, left, right) that are closest to the pixel in the i-th row and j-th column.

[0080] S104. Generate the object's shadow rendering image based on the target height value and the line drawing image.

[0081] Among them, the shadow rendering image refers to the rendering image that gives the object a shadow effect.

[0082] In some implementations, step S104 may specifically include:

[0083] S1041. Determine the normal information of each pixel in the contour line based on the target height value.

[0084] For each pixel on the contour line, four faces can be formed by the pixel and the four nearest non-blank pixels above, below, left, and right of the pixel. The normal vectors of these four faces are calculated, and the average value of the normal vectors is used as the normal information of the pixel.

[0085] S1042. Generate the object's shadow information based on the normal information, target height value, and preset lighting parameters.

[0086] In some implementations, step S1042 may specifically include:

[0087] Generate the occlusion diagram corresponding to the object based on the normal information;

[0088] Generate an occlusion map corresponding to the object based on the target height value;

[0089] Based on the occlusion map, occlusion map, and preset lighting parameters, generate the object's shadow information.

[0090] In this context, a occlusion map can be obtained by using the partial derivative of the normal vector of a pixel as the pixel value (ranging from 0 to 1), while an occlusion map can be obtained by using the target height value of a pixel as the pixel value. It should be noted that since the object is the entire image region enclosed by various contour lines, it typically includes blank pixels (i.e., pixels with a value of 255 or 1) and non-blank pixels (i.e., pixels with a value other than 255 or 1). For non-blank pixels, their normal information can be set to 0, meaning the corresponding pixel value in the occlusion map is 0. Similarly, the target height value of blank pixels can be set to 0, meaning the corresponding pixel value in the occlusion map is 0.

[0091] When generating shadow information, the projection vector of each pixel can be calculated first based on the occlusion map and preset lighting parameters. Then, the shadow information corresponding to the pixel is calculated based on the projection vector and the occlusion map. The preset lighting parameters determine lighting information such as lighting direction, light intensity, and softness / hardness. These parameters can include the position of the point light source. Typically, each pixel in the object receives light rays emitted from the point light source in a different direction. Users can set the required lighting parameters according to their actual needs.

[0092] For any given pixel, the formula for calculating its projection vector can be as follows:

[0093]

[0094] Plane equation: N·Q+d=0;

[0095] Where N is the plane normal (normalized) of the projection plane, Q is any point on the projection plane, d is the distance parameter in the plane equation, and S is the coordinates of the point light source. x ,q' y and q' z Let n be the X-axis, Y-axis, and Z-axis components (i.e., the components of the projection vector) of the coordinates of any point on the projection plane. x n y and n z These are the Y-axis components, Y-axis components, and Z-axis components of the plane normal N, respectively. x s y and s z These are the X-axis, Y-axis, and Z-axis components of the point light source coordinates, respectively. xP y and P z These are the X-axis, Y-axis, and Z-axis components of the pixel coordinates (the Z-axis component is also the pixel value of the pixel in the occlusion image).

[0096] After obtaining the components of the projection vector of each pixel on each coordinate axis, the components of the projection vector can be multiplied by the pixel value of the corresponding pixel on the occlusion map. The product is the shadow information corresponding to that pixel. Finally, after calculating the shadow information corresponding to each pixel in the object, all the shadow information can be used as the shadow information of the object.

[0097] S1043. Generate a shadow rendering image of the object based on the shadow information and the line drawing image.

[0098] In this method, the shadow information can be used as the second layer, and the line drawing image can be used as the first layer. The two layers can be overlaid to obtain the overlaid image, which is the shadow rendering image.

[0099] As can be seen from the above, the image processing method provided in this application embodiment obtains a line drawing image to be processed, which includes at least one object. It determines at least one contour line corresponding to the object and an initial height level corresponding to each contour line. Based on the initial height level and the contour line, it determines the target height value of each pixel in each contour line. Based on the target height value and the line drawing image, it generates a shadow rendering image of the object. Thus, it can accurately and automatically draw the shadow effect of various types of objects in the line drawing image. It has a wide range of applications and good shadow processing effect.

[0100] Based on the methods described in the above embodiments, this embodiment will be further described from the perspective of an image processing device. The image processing device can be implemented as an independent entity and can be applied to electronic devices such as servers or terminals. The terminal can include mobile phones, tablets, personal PCs, etc., and the server can be a server that can provide animation production, animation production, and game production services.

[0101] Please see Figure 4 , Figure 4 This application provides a specific description of an image processing apparatus, which may include: an acquisition module 10, a first determination module 20, a second determination module 30, and a generation module 40, wherein:

[0102] (1) Get module 10

[0103] Acquisition module 10 is used to acquire a line drawing image to be processed, which includes at least one object.

[0104] (2) First Determination Module 20

[0105] The first determining module 20 is used to determine at least one outline corresponding to the object and the initial height level corresponding to each outline.

[0106] In some implementations, the first determining module 20 is specifically used for:

[0107] Perform an outer contour scan on the current object to obtain multiple target pixels and record the number of scans.

[0108] Determine at least one contour line corresponding to the target pixel, and determine the initial height level corresponding to the contour line based on the number of scans.

[0109] The remaining pixels in the object, excluding the target pixel, are taken as the current object, and the process of performing an outer contour scan on the current object is returned until the preset conditions are met.

[0110] Furthermore, the aforementioned first determining module 20 is used for:

[0111] Obtain the image coordinates of each target pixel;

[0112] Determine the coordinate distance between two adjacent target pixels based on the image coordinates;

[0113] Based on the coordinate distance and the image coordinates, a line is fitted to the target pixel to obtain at least one contour line.

[0114] Furthermore, the aforementioned first determining module 20 is used for:

[0115] Use the number of scans as the initial height level for the corresponding contour.

[0116] (3) Second determination module 30

[0117] The second determining module 30 is used to determine the target height value of each pixel in each contour line based on the initial height level and the contour line.

[0118] In some implementations, the second determining module 30 is specifically used for:

[0119] Determine the length of each line of the outline;

[0120] Determine the minimum distance between the contour lines corresponding to two adjacent initial height levels;

[0121] Based on the line length, the minimum distance, and the initial height level, determine the target height level corresponding to each contour line;

[0122] The target height value of each pixel in the corresponding contour line is determined based on the target height level.

[0123] In some implementations, the second determining module 30 is specifically used for:

[0124] When the length of the line is less than the preset length threshold, the initial height level corresponding to the outline will be lowered.

[0125] When the minimum distance is less than the preset distance threshold, the initial height level with the larger value among the two initial height levels will be adjusted downwards.

[0126] The initial height level after the descent adjustment is used as the target height level of the corresponding contour line, and the remaining unadjusted initial height level is used as the target height level of the corresponding contour line.

[0127] In some implementations, the second determining module 30 is specifically used for:

[0128] Determine the product between the preset height increment and the target height level;

[0129] Determine the sum between the product and the preset starting height value, and use this sum as the initial height value of each pixel in the corresponding contour line;

[0130] The initial height value is smoothed to obtain the target height value of each pixel in the corresponding contour line.

[0131] (4) Generation module 40

[0132] The generation module 40 is used to generate a shadow rendering image of the object based on the target height value and the line drawing image.

[0133] In some implementations, the generation module 40 is specifically used for:

[0134] Determine the normal information of each pixel in the contour line based on the target height value;

[0135] The shadow information of the object is generated based on the normal information, the target height value, and the preset lighting parameters.

[0136] Based on the shadow information and the line drawing image, generate a shadow rendering image of the object.

[0137] In some implementations, the generation module 40 is further configured to:

[0138] Generate the occlusion diagram corresponding to the object based on the normal information;

[0139] Generate an occlusion map corresponding to the object based on the target height value;

[0140] Based on the occlusion map, the occlusion map, and the preset lighting parameters, the shadow information of the object is generated.

[0141] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0142] In addition, this application also provides an electronic device, which may be a smartphone, tablet computer, or other similar device. Figure 5 As shown, the electronic device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are electrically connected.

[0143] The processor 201 is the control center of the electronic device 200. It connects various parts of the electronic device through various interfaces and lines. By running or loading the application program stored in the memory 202 and calling the data stored in the memory 202, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0144] In this embodiment, the processor 201 in the electronic device 200 loads the instructions corresponding to the processes of one or more applications into the memory 202 according to the following steps, and the processor 201 runs the applications stored in the memory 202 to realize various functions.

[0145] Figure 6 A specific structural block diagram of an electronic device provided in an embodiment of this application is shown. This electronic device can be used to implement the image processing method provided in the above embodiments. The electronic device may include a smartphone or a server.

[0146] The electronic device may include a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a radio frequency (RF) circuit 303, a power supply 304, an input unit 305, and a display unit 306, etc. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0147] The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 302, and by calling data stored in the memory 302, thereby providing overall monitoring of the electronic device. Optionally, the processor may include one or more processing cores; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor.

[0148] The memory 302 can be used to store software programs (computer programs) and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0149] RF circuit 303 can be used for signal reception and transmission during information transmission and reception. Specifically, it receives downlink information from the base station and hands it over to one or more processors 301 for processing; additionally, it transmits uplink data to the base station. Typically, RF circuit 303 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 303 can also communicate wirelessly with networks and other devices. This wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile Communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, and SMS (Short Messaging Service).

[0150] The electronic device also includes a power supply 304 (such as a battery) that supplies power to various components. Preferably, the power supply 304 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. The power supply 304 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0151] The electronic device may also include an input unit 305, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in one embodiment, the input unit 305 may include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display or touchpad, can collect user touch operations on or near it (e.g., user operations using fingers, styluses, or any suitable object or accessory on or near the touch-sensitive surface) and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 301, and can receive and execute commands from the processor 301. Furthermore, various types of touch-sensitive surfaces, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touch-sensitive surface. In addition to the touch-sensitive surface, the input unit 305 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0152] The electronic device may further include a display unit 306, which can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit 306 includes multiple hardware display processing units, a video frame processing module, a display screen, etc. The multiple hardware display processing units and the video frame processing module can be integrated into a processing chip. The display screen may include a display panel, optionally configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED), or similar form. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor 301 to determine the type of touch event. Subsequently, the processor 301 provides corresponding visual output on the display panel according to the type of touch event. Although in the figures, the touch-sensitive surface and the display panel are implemented as two separate components to achieve input and output functions, in some embodiments, the touch-sensitive surface and the display panel can be integrated to achieve input and output functions.

[0153] Although not shown, the electronic device may also include a camera, Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 302 according to the following instructions, and the processor 301 runs the applications stored in the memory 302 to realize various functions, as follows:

[0154] Obtain a line art image to be processed, which includes at least one object;

[0155] Determine at least one outline corresponding to the object, and the initial height level corresponding to each outline;

[0156] Based on the initial height level and the contour line, determine the target height value of each pixel in each contour line;

[0157] Generate a shadow rendering image of the object based on the target height value and the line drawing image.

[0158] In some implementations, determining at least one contour line corresponding to the object, and the initial height level corresponding to each contour line, includes:

[0159] Perform an outer contour scan on the current object to obtain multiple target pixels and record the number of scans.

[0160] Determine at least one contour line corresponding to the target pixel, and determine the initial height level corresponding to the contour line based on the number of scans.

[0161] The remaining pixels in the object, excluding the target pixel, are taken as the current object, and the step of scanning the outer contour of the current object is returned until the preset conditions are met.

[0162] In some implementations, determining at least one contour line corresponding to the target pixel includes:

[0163] Obtain the image coordinates of each target pixel;

[0164] Determine the coordinate distance between two adjacent target pixels based on the image coordinates;

[0165] Based on the coordinate distance and the image coordinates, a line is fitted to the target pixel to obtain at least one contour line.

[0166] In some implementations, determining the initial height level corresponding to the contour line based on the number of scans includes:

[0167] Use the number of scans as the initial height level for the corresponding contour.

[0168] In some implementations, determining the target height value of each pixel in each contour line based on the initial height level and the contour line includes:

[0169] Determine the length of each line of the outline;

[0170] Determine the minimum distance between the contour lines corresponding to two adjacent initial height levels;

[0171] Based on the line length, the minimum distance, and the initial height level, determine the target height level corresponding to each contour line;

[0172] The target height value of each pixel in the corresponding contour line is determined based on the target height level.

[0173] In some implementations, determining the target height level for each contour line based on the line length, the minimum distance, and the initial height level includes:

[0174] When the length of the line is less than the preset length threshold, the initial height level corresponding to the outline will be lowered.

[0175] When the minimum distance is less than the preset distance threshold, the initial height level with the larger value among the two initial height levels will be adjusted downwards.

[0176] The initial height level after the descent adjustment is used as the target height level of the corresponding contour line, and the remaining unadjusted initial height level is used as the target height level of the corresponding contour line.

[0177] In some implementations, determining the target height value of each pixel in the corresponding contour line based on the target height level includes:

[0178] Determine the product between the preset height increment and the target height level;

[0179] Determine the sum between the product and the preset starting height value, and use this sum as the initial height value of each pixel in the corresponding contour line;

[0180] The initial height value is smoothed to obtain the target height value of each pixel in the corresponding contour line.

[0181] In some implementations, generating a shadow rendering image of the object based on the target height value and the line drawing image includes:

[0182] Determine the normal information of each pixel in the contour line based on the target height value;

[0183] The shadow information of the object is generated based on the normal information, the target height value, and the preset lighting parameters.

[0184] Based on the shadow information and the line drawing image, generate a shadow rendering image of the object.

[0185] In some implementations, generating the object's shadow information based on the normal information, the target height value, and preset lighting parameters includes:

[0186] Generate the occlusion diagram corresponding to the object based on the normal information;

[0187] Generate an occlusion map corresponding to the object based on the target height value;

[0188] Based on the occlusion map, the occlusion map, and the preset lighting parameters, the shadow information of the object is generated.

[0189] This electronic device can implement the steps of any embodiment of the image processing method provided in this application. Therefore, it can achieve the beneficial effects that any image processing method provided in this application can achieve. For details, please refer to the previous embodiments, which will not be repeated here.

[0190] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions (computer programs) or by instructions (computer programs) controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0191] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any embodiment of the image processing method provided in this application. For example, the computer program can execute the following steps:

[0192] Obtain a line art image to be processed, which includes at least one object;

[0193] Determine at least one outline corresponding to the object, and the initial height level corresponding to each outline;

[0194] Based on the initial height level and the contour line, determine the target height value of each pixel in each contour line;

[0195] Generate a shadow rendering image of the object based on the target height value and the line drawing image.

[0196] In some implementations, determining at least one contour line corresponding to the object, and the initial height level corresponding to each contour line, includes:

[0197] Perform an outer contour scan on the current object to obtain multiple target pixels and record the number of scans.

[0198] Determine at least one contour line corresponding to the target pixel, and determine the initial height level corresponding to the contour line based on the number of scans.

[0199] The remaining pixels in the object, excluding the target pixel, are taken as the current object, and the process of performing an outer contour scan on the current object is returned until the preset conditions are met.

[0200] In some implementations, determining at least one contour line corresponding to the target pixel includes:

[0201] Obtain the image coordinates of each target pixel;

[0202] Determine the coordinate distance between two adjacent target pixels based on the image coordinates;

[0203] Based on the coordinate distance and the image coordinates, a line is fitted to the target pixel to obtain at least one contour line.

[0204] In some implementations, determining the initial height level corresponding to the contour line based on the number of scans includes:

[0205] Use the number of scans as the initial height level for the corresponding contour.

[0206] In some implementations, determining the target height value of each pixel in each contour line based on the initial height level and the contour line includes:

[0207] Determine the length of each line of the outline;

[0208] Determine the minimum distance between the contour lines corresponding to two adjacent initial height levels;

[0209] Based on the line length, the minimum distance, and the initial height level, determine the target height level corresponding to each contour line;

[0210] The target height value of each pixel in the corresponding contour line is determined based on the target height level.

[0211] In some implementations, determining the target height level for each contour line based on the line length, the minimum distance, and the initial height level includes:

[0212] When the length of the line is less than the preset length threshold, the initial height level corresponding to the outline will be lowered.

[0213] When the minimum distance is less than the preset distance threshold, the initial height level with the larger value among the two initial height levels will be adjusted downwards.

[0214] The initial height level after the descent adjustment is used as the target height level of the corresponding contour line, and the remaining unadjusted initial height level is used as the target height level of the corresponding contour line.

[0215] In some implementations, determining the target height value of each pixel in the corresponding contour line based on the target height level includes:

[0216] Determine the product between the preset height increment and the target height level;

[0217] Determine the sum between the product and the preset starting height value, and use this sum as the initial height value of each pixel in the corresponding contour line;

[0218] The initial height value is smoothed to obtain the target height value of each pixel in the corresponding contour line.

[0219] In some implementations, generating a shadow rendering image of the object based on the target height value and the line drawing image includes:

[0220] Determine the normal information of each pixel in the contour line based on the target height value;

[0221] The shadow information of the object is generated based on the normal information, the target height value, and the preset lighting parameters.

[0222] Based on the shadow information and the line drawing image, generate a shadow rendering image of the object.

[0223] In some implementations, generating the object's shadow information based on the normal information, the target height value, and preset lighting parameters includes:

[0224] Generate the occlusion diagram corresponding to the object based on the normal information;

[0225] Generate an occlusion map corresponding to the object based on the target height value;

[0226] Based on the occlusion map, the occlusion map, and the preset lighting parameters, the shadow information of the object is generated.

[0227] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0228] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0229] Since the computer program stored in the storage medium can execute the steps in any of the image processing method embodiments provided in this application, the beneficial effects that any of the image processing methods provided in this application can achieve can be realized. For details, please refer to the preceding embodiments, which will not be repeated here.

[0230] The foregoing has provided a detailed description of an image processing method, apparatus, electronic device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized in that, include: Obtain a line art image to be processed, wherein the line art image includes at least one object; Determine at least one outline corresponding to the object, and the initial height level corresponding to each outline; Based on the initial height level and the contour lines, determine the target height value of each pixel in each contour line; Based on the target height value and the line drawing image, generate a shadow rendering image of the object; The step of generating the shadow rendering image of the object based on the target height value and the line drawing image includes: The normal information of each pixel in the contour line is determined based on the target height value; The shadow information of the object is generated based on the normal information, the target height value, and the preset lighting parameters. Based on the shadow information and the line drawing image, a shadow rendering image of the object is generated.

2. The image processing method according to claim 1, characterized in that, Determining at least one contour line corresponding to the object, and the initial height level corresponding to each contour line, includes: Perform an outer contour scan on the current object to obtain multiple target pixels, and record the number of scans. Determine at least one contour line corresponding to the target pixel, and determine the initial height level corresponding to the contour line based on the number of scans. Take the remaining pixels in the object other than the target pixel as the current object, and return to the step of scanning the outer contour of the current object until the preset condition is met.

3. The image processing method according to claim 2, characterized in that, Determining at least one contour line corresponding to the target pixel includes: Obtain the image coordinates of each of the target pixels; The coordinate distance between two adjacent target pixels is determined based on the image coordinates; Based on the coordinate distance and the image coordinates, the target pixel is fitted with a line to obtain at least one contour line.

4. The image processing method according to claim 2, characterized in that, Determining the initial height level corresponding to the contour line based on the number of scans includes: The number of scans is used as the initial height level of the corresponding contour line.

5. The image processing method according to claim 1, characterized in that, The step of determining the target height value of each pixel in each contour line based on the initial height level and the contour line includes: Determine the line length of each of the aforementioned contour lines; Determine the minimum distance between the contour lines corresponding to two adjacent initial height levels; Based on the line length, the minimum distance, and the initial height level, determine the target height level corresponding to each contour line; The target height value of each pixel in the corresponding contour line is determined based on the target height level.

6. The image processing method according to claim 5, characterized in that, The step of determining the target height level corresponding to each contour line based on the line length, the minimum distance, and the initial height level includes: When the line length is less than a preset length threshold, the initial height level corresponding to the outline line will be adjusted downwards. When the minimum distance is less than the preset distance threshold, the initial height level with the larger value among the two initial height levels will be adjusted to be lowered. The initial height level after the descent adjustment is used as the target height level of the corresponding contour line, and the remaining unadjusted initial height level is used as the target height level of the corresponding contour line.

7. The image processing method according to claim 5, characterized in that, The step of determining the target height value of each pixel in the corresponding contour line according to the target height level includes: Determine the product between the preset height increment and the target height level; Determine the sum between the product and the preset starting height value, and use the sum as the initial height value of each pixel in the corresponding contour line; The initial height value is smoothed to obtain the target height value of each pixel in the corresponding contour line.

8. The image processing method according to claim 1, characterized in that, The step of generating the shadow information of the object based on the normal information, the target height value, and preset lighting parameters includes: Generate a occlusion diagram corresponding to the object based on the normal information; Generate an occlusion map corresponding to the object based on the target height value; The shadow information of the object is generated based on the occlusion map, the occlusion map, and the preset lighting parameters.

9. An image processing apparatus, characterized in that, include: An acquisition module is used to acquire a line drawing image to be processed, wherein the line drawing image includes at least one object; The first determining module is used to determine at least one outline corresponding to the object, and the initial height level corresponding to each outline; The second determining module is used to determine the target height value of each pixel in each contour line based on the initial height level and the contour line; A generation module is used to generate a shadow rendering image of the object based on the target height value and the line drawing image; The generation module is used for: The normal information of each pixel in the contour line is determined based on the target height value; The shadow information of the object is generated based on the normal information, the target height value, and the preset lighting parameters. Based on the shadow information and the line drawing image, a shadow rendering image of the object is generated.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted to be loaded by a processor to perform the image processing method of any one of claims 1 to 8.

11. An electronic device, characterized in that, The method includes a coupled memory and a processor, the memory storing a computer program, and the processor running the computer program in the memory to perform the steps of the image processing method according to any one of claims 1 to 8.

Images