Improved system for generating animation sketches based on deep learning

Abstract

The application discloses an improved system for generating an animation sketch based on deep learning, which comprises the following steps: firstly, an animation scene is recognized through a convolutional neural network; secondly, a transformation supplementary network model is established based on the recognition result of the scene; thirdly, a transformation area is calculated through scene separation, region segmentation and feature extraction, and then a transformation signal is generated to fill and complete the missing area, and scene superposition is performed to generate a two-dimensional sketch, so as to ensure the integrity of the image; fourthly, a corresponding three-dimensional sketch is generated through a preset 3D deep network model; and finally, the interactive process of the sketch and an animation image sequence is analyzed to obtain a discriminant coefficient under different discriminant modes in the interactive process, and then a loss degree is generated to perform corresponding optimization operation on the interactive process.

Description

Technical Field

[0001] This invention relates to the field of animation design technology, and in particular to an improved system for generating animation sketches based on deep learning. Background Technology

[0002] With its unique perspective and entertainment value, animation is loved by a wide audience, providing people with a vast virtual imagination space that transcends real life. After watching many animation works, many emerging animation companies are now investing in animation production.

[0003] However, in the process of animation production, sketches are usually generated directly from animation images or video frames into two-dimensional shapes. It is relatively rare to directly generate sketches into three-dimensional shapes. When it is necessary to modify the animation, the key frames of the sketch are edited to achieve batch modification of the animation. In the interaction process between the sketch and the animation image sequence, the incompleteness of the sketch can easily lead to the failure of batch modification of the animation.

[0004] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0005] The purpose of this invention is to propose an improved system for generating animation sketches based on deep learning. By establishing a transformation and supplementation network model to fill in missing regions of animation frame images, and then superimposing them to generate two-dimensional sketches, the integrity of the images is ensured. Furthermore, a corresponding three-dimensional sketch is generated through a preset 3D depth network model. Finally, by analyzing the interaction process between the sketches and the animation image sequence, the interaction process is optimized.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an improved system for generating animation sketches based on deep learning, comprising a server, a scene recognition unit, a repair and construction unit, and an editing and computing unit;

[0007] Once the server detects the running command, the scene recognition unit extracts frame images based on the animation video stream image sequence, and classifies and filters the animation scenes into animation character scenes and animation environment scenes based on the convolutional neural network, and then sends the scene recognition results to the repair and construction unit.

[0008] The repair building unit receives the scene recognition result, fills in the missing areas of the scene by establishing a transformation supplement network model, and then overlays the frame image according to the output result of the transformation supplement network model to generate a two-dimensional sketch. The two-dimensional sketch is then brought into a preset 3D depth network model to reconstruct the two-dimensional sketch and generate the corresponding three-dimensional sketch.

[0009] The input of the editing calculation unit is connected to the output of the repair construction unit. It generates a new animation image sequence by directly editing the 3D sketch, analyzes the interaction process between the sketch and the animation image sequence, calculates the loss degree in the interaction process, and generates an optimization signal to perform optimization operation when the loss degree exceeds a preset threshold.

[0010] Furthermore, the specific process of screening based on neural networks is as follows:

[0011] Step A1: Acquire anime frame images in real time and extract several candidate region frame images. The candidate region images are several images containing anime characters.

[0012] Step A2: Each neuron in the input layer of the neural network receives the candidate region image and passes it to each neuron in the intermediate layer. The sample candidate region image is processed and transformed to obtain the character features in the candidate region image as target type information. The target type information is then output by each neuron in the output layer. The weights of each layer are then corrected according to the gradient descent method to reduce errors and obtain the character feature recognition model.

[0013] Step A3: Filter the frame images within a preset time period using the character feature recognition model, and select the anime character scenes with anime characters, and mark them as scene set A. Then, the anime environment scenes that are not selected within the preset time period are marked as scene set B.

[0014] Furthermore, the specific working process of transforming and supplementing the network model is as follows:

[0015] Constructing a transform-complementary network model: The input information consists of scene set A and scene set B, and the output is set as the complete target background image. The specific steps are as follows:

[0016] Step B1: First, scene separation is performed on scene set A to obtain the target person image and target background image; then, the target background image is divided into equal regions with N region points; then, pixel features and contour features are extracted from all region points through convolutional and pooling layers, and feature search and matching are performed in scene set B. Then, the pixel matching difference and contour matching difference between the two in each region point i are calculated and labeled as DXi and DLi, respectively.

[0017] Step B2: When both pixel matching difference DXi and contour matching difference DLi are 0, it indicates a complete match, and the missing areas of the target background image are filled in. When both pixel matching difference DXi and contour matching difference DLi are not 0, it indicates an incomplete match. A Cartesian coordinate system is established with the ratio of pixel matching difference DXi / contour matching difference DLi as the X-axis and region point N as the Y-axis. A characteristic curve of the change in pixel matching difference DXi / contour matching difference DLi - region point N is plotted. Simultaneously, a standard threshold curve is preset in the coordinate system. The area enclosed by the line segment of the characteristic curve above the standard threshold curve and the standard threshold curve is obtained and marked as the transformation area. A transformation area threshold is preset, and the transformation area is compared with the preset transformation area threshold. When the transformation area is greater than the preset transformation area threshold, a transformation signal is generated to fill in the missing areas of the target background image.

[0018] Furthermore, the specific process of generating the transformed signal is as follows:

[0019] A corresponding correction coefficient is assigned to the transformation area, and this coefficient is combined with the transformation area to generate a weighted transformation coefficient, which is then labeled as α. Simultaneously, the matching animation environment scene corresponding to the target background image is obtained, and features of the animation environment scene at different scales are extracted. The w1-th and w2-th outputs of the network perception layer are used as the perception feature information, where w1 and w2 are both positive integers and are labeled as feature S1. The image data corresponding to feature S1 is divided into three parts in a 5:1:1 ratio: training set, validation set, and test set. A weighted transformation coefficient α is assigned to feature S1. The transformation supplementary network model is trained until the transformation area is less than the preset transformation area threshold, at which point the network training is immediately stopped.

[0020] Furthermore, the pre-set 3D depth network model operation process includes:

[0021] Step C1: The input information is a two-dimensional sketch, specifically a W*H binary sketch image I, where I∈{0,1}, where W*H are the width and height of the two-dimensional sketch, respectively; then, features are extracted at different scales, and the feature information is labeled as feature S2;

[0022] Step C2: A three-dimensional space is predefined and equally divided into multiple cubes. Points in the three-dimensional space are projected onto a two-dimensional sketch to obtain their corresponding two-dimensional positions. A 1×1 convolutional kernel is then used to extract the feature information corresponding to the two-dimensional positions. This feature information is then connected within a predefined region to form local feature information, which is labeled as feature S3. Finally, features S2 and S3 are fused to generate a one-dimensional tensor, labeled as S... 合 ;

[0023] Step C3, transfer feature S 合The corresponding image data is divided into three parts in a 3:1:1 ratio: training set, validation set, and test set. For feature S... 合 The appropriate weights are assigned, and the training process is performed to obtain the final output information, which is a 3D sketch with a high-resolution mesh.

[0024] Furthermore, the specific working process of editing the calculation unit is as follows:

[0025] Step D1, construct the impact analysis model: label the input information as set U, which contains N0 element values. Label any element value as Xj. Assign corresponding weight coefficients to the N0 element values. The weight coefficient of the labeled element value Xj is ej, and ej is greater than 0. Σej=1. Output the impact coefficient Ue of set U by establishing a formula.

[0026] Step D2: Obtain the discrimination mode of the linkage interaction. There are three discrimination modes: the first discrimination mode, the second discrimination mode, and the third discrimination mode. Then, substitute the discrimination parameters corresponding to the three discrimination modes into the influence analysis model to generate the first discrimination coefficient, the second discrimination coefficient, and the third discrimination coefficient.

[0027] Step D3: Construct a three-dimensional dynamic graph with the first discriminant coefficient as the X-axis, the second discriminant coefficient as the Y-axis, and the third discriminant coefficient as the Z-axis. Overlay the three-dimensional dynamic graph with the standard threshold dynamic graph, then obtain its horizontal and vertical screenshots and compare them to generate the total overlapping area cmj and the total non-overlapping area fmj. Set a formula to calculate the loss degree g, and compare and analyze the loss degree with the preset loss interval [G1,G2] to generate the corresponding optimization signal.

[0028] Furthermore, the specific process for generating the first discriminant coefficient is as follows:

[0029] The first discrimination mode is the process of generating a frame image into a two-dimensional sketch: obtaining the first discrimination parameters within a preset time period, the first discrimination parameters include the transformed area and the supplementary line length, and are marked as a1 and a2 respectively; wherein, the supplementary line length BC is the line segment length value for filling and completing the missing area of ​​the target background image;

[0030] Then, substitute the first discriminant parameter into the influence analysis model to generate the first discriminant coefficient.

[0031] Furthermore, the specific process for generating the second discriminant coefficient is as follows:

[0032] The second discrimination mode is the process of generating a 2D sketch into a 3D sketch: first, a formula is set to obtain the distance energy value P, then the isosurface coefficient DZ corresponding to the distance energy value P is obtained, and the distance energy value P and the isosurface coefficient are combined and assigned the corresponding transformation coefficient to generate the distance index b1; a formula is set to obtain the shape root mean square error value b2, and then the distance index b1 and the shape root mean square error value b2 are both marked as the second discrimination parameters;

[0033] Next, obtain the second discrimination parameter within the preset time period, and then substitute the second loss parameter into the impact analysis model to generate the second discrimination coefficient.

[0034] Furthermore, the specific process for generating the third discriminant coefficient is as follows:

[0035] The third discrimination mode is the process of generating a 3D sketch into a real image: The third discrimination mode is the process of generating a 3D sketch into a real image: Obtain the missing parameters in the region within a preset time. The missing parameters include color missing values, contour missing values ​​and texture missing values, and are marked as c1, c2 and c3 respectively; then perform preliminary processing on the missing parameters to generate the missing degree value c4, and collect the missing degree value c4 into the missing parameters to generate the third discrimination parameter;

[0036] Next, obtain the third discriminant parameter within the preset time period, and then substitute the third loss parameter into the impact analysis model to generate the third discriminant coefficient.

[0037] Furthermore, the specific process for preliminary processing of missing parameters is as follows:

[0038] Step Sa: Collect the duration between the start and end times of the generation process and mark it as the time threshold T. Then divide the time threshold T into m sub-time nodes, where m is greater than 0. Obtain the missing parameters of each time node. Then establish a rectangular coordinate system with time as the X-axis and missing parameters as the Y-axis. Plot the curve of missing parameter change in the rectangular coordinate system by plotting points.

[0039] In step Sb, a standard threshold curve is preset in the rectangular coordinate system. The missing parameter change curve is compared with the preset change threshold curve to obtain the area enclosed by the line segment of the missing parameter change curve above the standard threshold curve and the standard threshold curve. A circle is drawn with the point on the preset change threshold curve as the center and r as the radius. When the circle completely includes the area, the correction coefficient WX1 for the missing parameter is assigned; otherwise, the correction coefficient WX2 for the missing parameter is assigned. The missing parameter is then multiplied by the corresponding correction coefficient to generate the missing degree value c4.

[0040] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0041] This invention first identifies animation scenes using a convolutional neural network. Based on this, a transformation and supplementation network model is established based on the scene recognition results. The transformation area is calculated through scene separation, region segmentation, and feature extraction. Then, a transformation signal is generated to fill in the missing areas and superimpose them to generate a two-dimensional sketch to ensure the integrity of the image. Then, a corresponding three-dimensional sketch is generated through a preset 3D depth network model. Furthermore, by analyzing the interaction process between the sketch and the animation image sequence, the discrimination coefficients under different discrimination modes during the interaction process are obtained, and a loss value is generated to optimize the interaction process accordingly. Attached Figure Description

[0042] Figure 1 A schematic diagram of the structure of the present invention is shown;

[0043] Figure 2 A schematic diagram of the process of the present invention is shown. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1: As Figure 1-2 As shown, the improved system for anime sketch generation based on deep learning includes a server, a scene recognition unit, a repair and construction unit, and an editing and computing unit.

[0046] The specific work steps are as follows:

[0047] S1. When the server detects the running command, the scene recognition unit extracts frame images based on the animation video stream image sequence, and classifies and filters the animation scene into animation character scenes and animation environment scenes based on the convolutional neural network, and then sends the scene recognition results to the repair and construction unit.

[0048] S1-1, The specific process of screening based on neural networks is as follows:

[0049] Step A1: Acquire anime frame images in real time and extract several candidate region frame images. The candidate region images are several images containing anime characters.

[0050] Step A2: Each neuron in the input layer of the neural network receives the candidate region image and passes it to each neuron in the intermediate layer. The sample candidate region image is processed and transformed to obtain the character features in the candidate region image as target type information. The target type information is then output by each neuron in the output layer. The output target type information is compared with the target type information expected to be output by the sample. If the actual output does not match the expected output, the error backpropagation stage begins. The error passes through the output layer of the neural network and is corrected by the weights of each layer in the manner of gradient descent. The error is reversed layer by layer towards the intermediate layer and the input layer. The error between the actual output target type information and the expected output target type information is reduced and compared with a preset error threshold. When the error is less than the preset error threshold, the learning process is stopped immediately, and the character feature recognition model is obtained.

[0051] Step A3: Filter the frame images within a preset time period using the character feature recognition model, and select the anime character scenes with anime characters, and mark them as scene set A. Then, the anime environment scenes that are not selected within the preset time period are marked as scene set B.

[0052] S2, Repair the scene recognition result received by the building unit:

[0053] S2-1, by establishing a transformation supplementation network model to fill in the missing areas of the scene, and then by overlaying the frame images according to the output of the transformation supplementation network model, a two-dimensional sketch is generated.

[0054] Constructing a transform-complementary network model: The input information consists of scene set A and scene set B, and the output is set as the complete target background image. The specific steps are as follows:

[0055] Step B1: First, scene separation is performed on scene set A to obtain the target person image and target background image; then, the target background image is divided into equal regions with N region points; then, pixel features and contour features are extracted from all region points through convolutional and pooling layers, and feature search and matching are performed in scene set B. Then, the pixel matching difference and contour matching difference between the two in each region point i are calculated and labeled as DXi and DLi, respectively.

[0056] Step B2: When both pixel matching difference DXi and contour matching difference DLi are 0, it indicates a complete match, and the missing areas of the target background image are filled in. When both pixel matching difference DXi and contour matching difference DLi are not 0, it indicates an incomplete match. A Cartesian coordinate system is established with the ratio of pixel matching difference DXi / contour matching difference DLi as the X-axis and region point N as the Y-axis. A characteristic curve of the change in pixel matching difference DXi / contour matching difference DLi - region point N is plotted. Simultaneously, a standard threshold curve is preset in the coordinate system. The area enclosed by the line segment of the characteristic curve above the standard threshold curve and the standard threshold curve is obtained and marked as the transformation area. A transformation area threshold is preset, and the transformation area is compared with the preset transformation area threshold. When the transformation area is greater than the preset transformation area threshold, a transformation signal is generated to fill in the missing areas of the target background image.

[0057] Step B2-1, the specific process of generating the transformed signal is as follows:

[0058] The transformation area is assigned a corresponding correction coefficient, which is then combined with the transformation area to generate a weighted transformation coefficient, denoted as α. Simultaneously, the matching animation environment scene corresponding to the target background image is obtained, and features of the animation environment scene at different scales are extracted. The w1-th and w2-th outputs of the network perception layer are used as the perception feature information, where w1 and w2 are both positive integers and are denoted as feature S1. The image data corresponding to feature S1 is divided into three parts: training set, validation set, and test set in a 5:1:1 ratio. The weighted transformation coefficient α is assigned to feature S1. The transformation supplementary network model is trained until the transformation area is less than the preset transformation area threshold, at which point the network training is immediately stopped.

[0059] S2-2, then import the two-dimensional sketch into the preset 3D depth network model to reconstruct the two-dimensional sketch and generate the corresponding three-dimensional sketch.

[0060] The pre-set 3D depth network model operation process includes:

[0061] Step C1: The input information is a two-dimensional sketch, specifically a W*H binary sketch image I, where I∈{0,1}, where W*H are the width and height of the two-dimensional sketch, respectively; then, features are extracted at different scales, and the feature information is labeled as feature S2;

[0062] Step C2: A three-dimensional space is predefined and equally divided into multiple cubes. Points in the three-dimensional space are projected onto a two-dimensional sketch to obtain their corresponding two-dimensional positions. A 1×1 convolutional kernel is then used to extract the feature information corresponding to the two-dimensional positions. This feature information is then connected within a predefined region to form local feature information, which is labeled as feature S3. Finally, features S2 and S3 are fused to generate a one-dimensional tensor, labeled as S... 合 ;

[0063] Step C3, transfer feature S 合 The corresponding image data is divided into three parts in a 3:1:1 ratio: training set, validation set, and test set. For feature S... 合 Assign appropriate weights, train and learn, and obtain the final output information, which is a 3D sketch with a high-resolution mesh;

[0064] S3, the input end of the editing calculation unit is connected to the output end of the repair construction unit. It generates a new animation image sequence by directly editing the three-dimensional sketch, analyzes the interaction process between the sketch and the animation image sequence, calculates the loss degree in the interaction process, and generates an optimization signal to perform optimization operation when the loss degree exceeds a preset threshold.

[0065] Step D1, Construct the impact analysis model: Label the input information as set U, which contains N0 element values. Label any element value as Xj. Assign corresponding weight coefficients to the N0 element values. Let the weight coefficient of the labeled element value Xj be ej, and let ej be greater than 0, ∑ej=1. Output the impact coefficient Ue of set U by establishing a formula, which is:

[0066] Step D2: Obtain the discrimination mode of the linkage interaction. There are three discrimination modes: the first discrimination mode, the second discrimination mode, and the third discrimination mode. Then, substitute the discrimination parameters corresponding to the three discrimination modes into the influence analysis model to generate the first discrimination coefficient, the second discrimination coefficient, and the third discrimination coefficient.

[0067] Step D2-1, the specific process of generating the first discriminant coefficient is as follows:

[0068] The first discrimination mode is the process of generating a two-dimensional sketch from a frame image:

[0069] Step D2-11: First, obtain the first discrimination parameter within the region within the preset time. The first discrimination parameter includes the transformation area and the line filling length, and are marked as a1 and a2 respectively. Among them, the line filling length BC is the line segment length value for filling the missing region of the target background image.

[0070] Step D2-12: Using the first discriminant parameter as input information, establish the set Aa = {transformed area a1, supplementary line length a2}. Then, substitute the set Aa into the influence analysis model to generate the first discriminant coefficient U1, with the formula: Where β1 and β2 are the weighting coefficients of the transformed area a1 and the supplementary line length a2, respectively; the larger the transformed area and the larger the supplementary line length, the larger the first discrimination parameter.

[0071] Step D2-2, the specific process of generating the second discriminant coefficient is as follows:

[0072] The second discrimination mode is the process of generating a 3D sketch from a 2D sketch:

[0073] Step D2-21, set the function analytical formula: P = SDF(p) to obtain the distance energy value P, represented as feature S. 合 The nearest distance from the sample point p on the corresponding image data to the isosurface of the 3D shape boundary; where SDF(·) is the distance sign function, and SDF(p) is positive inside the isosurface of the shape boundary and negative outside the isosurface of the shape boundary; then, the isosurface coefficient DZ corresponding to the distance energy value P is obtained, and the distance energy value P and the isosurface coefficient are combined, and a corresponding transformation coefficient is assigned to generate the distance exponent b1, the formula is: Where ε is the conversion coefficient, and ε is greater than 0; it should be noted that the isosurface is a plane inside each cube, and by selecting different cube regions, the size of the isosurface can be adjusted to obtain different 3D surface models. The isosurface coefficient can be set by different sizes. The smaller the cube is cut, the more accurate the model is, and the smaller the corresponding isosurface coefficient is.

[0074] Define a standardized formula: Obtain the shape mean squared error value b2 between the 3D sketch and the preset 3D sketch, where Y is the shape of the generated 3D sketch. The shape of the preset 3D sketch;

[0075] Then, the distance exponent b1 and the shape mean square error value b2 are labeled as the second discriminant parameters;

[0076] Step D2-2: Obtain the second discriminant parameter within the preset time period. Using the second loss parameter as input information, establish a set Bb = {distance exponent b1, shape mean square error b2}. Then, substitute the set Bb into the influence analysis model to generate the second discriminant coefficient U2, as shown in the formula: Where β3 and β4 are the weighting coefficients of the distance index b1 and the shape mean square error b2, respectively; the larger the distance index b1 and the shape mean square error b2 are, the larger the second discriminant coefficient U2 will be.

[0077] Step D2-3, the specific process of generating the third discriminant coefficient is as follows:

[0078] The third discrimination mode is the process of generating a 3D sketch into a real image:

[0079] Step D2-31: First, perform preliminary processing on the missing parameters:

[0080] The missing parameters include color missing values, contour missing values, and texture missing values, which are labeled as c1, c2, and c3 respectively.

[0081] Step Sa: Collect the duration between the start and end times of the generation process and mark it as the time threshold T. Then divide the time threshold T into m sub-time nodes, where m is greater than 0. Obtain the missing parameters of each time node. Then establish a rectangular coordinate system with time as the X-axis and missing parameters as the Y-axis. Plot the curve of missing parameter change in the rectangular coordinate system by plotting points.

[0082] Step Sb: In the Cartesian coordinate system, a standard threshold curve is preset. The missing parameter change curve is compared with the preset change threshold curve to obtain the area enclosed by the line segment of the missing parameter change curve above the standard threshold curve and the standard threshold curve. A circle is drawn with the point on the preset change threshold curve as the center and r as the radius. When the circle completely includes the area, the correction coefficient WX1 for the missing parameter is assigned; otherwise, the correction coefficient WX2 for the missing parameter is assigned. The missing parameter is then multiplied by the corresponding correction coefficient to generate the missing degree value c4.

[0083] Step D2-31: Add the missing value c4 to the missing parameters to generate the third discriminant parameter. Obtain the third discriminant parameter within the preset time period. Then, use the third discriminant parameter as input information to establish a set Cc = {color missing value c1, contour missing value c2, texture missing value c3, missing value c4}. Substitute the set Cc into the influence analysis model to generate the third discriminant coefficient. The formula is: Where β5, β6, β7 and β8 are the weighting coefficients for color missing value c1, contour missing value c2, texture missing value c3 and missing degree value c4, respectively.

[0084] Step D3: Construct a 3D dynamic graph using the first discriminant coefficient as the X-axis, the second discriminant coefficient as the Y-axis, and the third discriminant coefficient as the Z-axis. Overlay the 3D dynamic graph with the standard threshold dynamic graph, then obtain its horizontal and vertical screenshots and compare them to generate the overlapping total area cmj and the non-overlapping total area fmj. Set the formula: Calculate the loss value g, where χ is a correction coefficient and χ is greater than 0; then compare the loss value g with the preset loss interval [G1, G2]:

[0085] When g≥G2, a first optimization signal is generated, indicating that the degree of loss is severe;

[0086] When G1<g<G2, a second optimization signal is generated, indicating that the degree of loss is moderate;

[0087] When g≤G1, a third optimization signal is generated, indicating that the degree of loss is minor;

[0088] The text is edited and displayed through the optimization signal, the text content is "The degree of loss reaches level XX, optimization is required", and corresponding optimization operations are performed.

[0089] Combining the above technical solutions: the present invention first identifies animation scenes through a convolutional neural network, on this basis, establishes a transformation supplementary network model based on the scene recognition results, calculates the transformation area through scene separation, region segmentation and feature extraction, and then generates a transformation signal to fill and complete the missing region; further overlays the images to generate a two-dimensional sketch to ensure the integrity of the image, and then deeply generates a corresponding three-dimensional sketch through a preset 3D depth network model; further obtains the discrimination coefficients under different discrimination modes in the interaction process by analyzing the interaction process between the sketch and the animation image sequence, and then generates a loss degree, and performs corresponding optimization operations on the interaction process.

[0090] The setting of the ranges and threshold values is for the convenience of comparison, and the value of the thresholds depends on the amount of sample data and the base number set by those skilled in the art for each group of sample data; as long as the proportional relationship between the parameters and the quantified values is not affected, it is acceptable.

[0091] All the above formulas are subjected to dimension removal for numerical calculation, the formulas are formulas obtained by collecting a large amount of data and performing software simulation to approach the real situation, and the preset parameters in the formulas are set by those skilled in the art according to actual conditions;

[0092] The above description is only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art, within the technical scope disclosed by the present invention, shall be covered by the scope of protection of the present invention through equivalent replacement or modification based on the technical solution of the present invention and the inventive concept thereof.

Claims

1. An improved system for anime sketch generation based on deep learning, characterized in that: It includes a server, a scene recognition unit, a repair and construction unit, and an editing and calculation unit; Once the server detects the running command, the scene recognition unit extracts frame images based on the animation video stream image sequence, and classifies and filters the animation scenes into animation character scenes and animation environment scenes based on the convolutional neural network, and then sends the scene recognition results to the repair and construction unit. The repair building unit receives the scene recognition result, fills in the missing areas of the scene by establishing a transformation supplement network model, and then overlays the frame image according to the output result of the transformation supplement network model to generate a two-dimensional sketch. The two-dimensional sketch is then brought into a preset 3D depth network model to reconstruct the two-dimensional sketch and generate the corresponding three-dimensional sketch. The input of the editing calculation unit is connected to the output of the repair construction unit. New animation image sequences are generated by directly editing 3D sketches. The interaction process between the sketches and the animation image sequences is analyzed, and the loss during the interaction process is calculated. When the loss exceeds a preset threshold, an optimization signal is generated for optimization. The specific process of filtering based on convolutional neural networks is as follows: Step A1: Acquire anime frame images in real time and extract several candidate region frame images. The candidate region images are several images containing anime characters. Step A2: Each neuron in the input layer of the neural network receives the candidate region image and passes it to each neuron in the intermediate layer. The sample candidate region image is processed and transformed to obtain the character features in the candidate region image as target type information. The target type information is then output by each neuron in the output layer. The weights of each layer are then corrected according to the gradient descent method to reduce errors and obtain the character feature recognition model. Step A3: Filter the frame images within a preset time period using the character feature recognition model, and select the anime character scenes with anime characters and mark them as scene set A. Then, the scenes that are not selected within the preset time period are anime environment scenes and are marked as scene set B. The specific working process of the transformed supplementary network model is as follows: Constructing a transform-complementary network model: The input information consists of scene set A and scene set B, and the output is set as the complete target background image. The specific steps are as follows: Step B1: First, scene separation is performed on scene set A to obtain the target person image and target background image; then, the target background image is divided into equal regions with N region points; then, pixel features and contour features are extracted from all region points through convolutional and pooling layers, and feature search and matching are performed in scene set B. Then, the pixel matching difference and contour matching difference between the two in each region point i are calculated and labeled as DXi and DLi, respectively. Step B2: When both pixel matching difference DXi and contour matching difference DLi are 0, it indicates a perfect match, and the missing areas of the target background image are filled in. When the pixel matching difference DXi and the contour matching difference DLi are not zero, it indicates an incomplete match. A Cartesian coordinate system is then established with the ratio of pixel matching difference DXi to contour matching difference DLi as the X-axis and region point N as the Y-axis. A characteristic curve representing the change in pixel matching difference DXi / contour matching difference DLi - region point N is plotted. Simultaneously, a standard threshold curve is preset in the coordinate system. The area enclosed by the line segment of the characteristic curve above the standard threshold curve and the standard threshold curve is obtained and marked as the transformed area. A preset transform area threshold is then set, and the transformed area is compared with this threshold. When the transformed area is greater than the preset transform area threshold, a transformation signal is generated to fill in the missing areas of the target background image. The specific process for generating the transformed signal is as follows: A corresponding correction coefficient is assigned to the transformation area, and this coefficient is combined with the transformation area to generate a weighted transformation coefficient, which is then labeled as α. Simultaneously, the matching animation environment scene corresponding to the target background image is obtained, and features of the animation environment scene at different scales are extracted. The w1-th and w2-th outputs of the network perception layer are used as the perception feature information, where w1 and w2 are both positive integers and are labeled as feature S1. The image data corresponding to feature S1 is divided into three parts in a 5:1:1 ratio: training set, validation set, and test set. A weighted transformation coefficient α is assigned to feature S1. The transformation supplementary network model is trained until the transformation area is less than the preset transformation area threshold, at which point the network training is immediately stopped.

2. The improved system for animation sketch generation based on deep learning according to claim 1, characterized in that: The operation of the preset 3D depth network model includes: Step C1: The input information is a two-dimensional sketch, specifically a W*H binary sketch image I, where I∈{0,1}, where W*H are the width and height of the two-dimensional sketch, respectively; then, features are extracted at different scales, and the feature information is labeled as feature S2; Step C2: A three-dimensional space is predefined and equally divided into multiple cubes. Points in the three-dimensional space are projected onto a two-dimensional sketch to obtain their corresponding two-dimensional positions. A 1×1 convolutional kernel is then used to extract the feature information corresponding to the two-dimensional positions. This feature information is then connected within a predefined region to form local feature information, which is labeled as feature S3. Finally, features S2 and S3 are fused to generate a one-dimensional tensor, labeled as S... 合 ; Step C3, transfer feature S 合 The corresponding image data is divided into three parts in a 3:1:1 ratio: training set, validation set, and test set. For feature S... 合 The appropriate weights are assigned, and the training process is performed to obtain the final output information, which is a 3D sketch with a high-resolution mesh.

3. The improved system for animation sketch generation based on deep learning according to claim 2, characterized in that: The specific working process of the editing and calculation unit is as follows: Step D1, construct the impact analysis model: label the input information as set U, which contains N0 element values. Label any element value as Xj. Assign corresponding weight coefficients to the N0 element values. The weight coefficient of the labeled element value Xj is ej, and ej is greater than 0. Σej=1. Output the impact coefficient Ue of set U by establishing a formula. Step D2: Obtain the discrimination mode of the linkage interaction. There are three discrimination modes: the first discrimination mode, the second discrimination mode, and the third discrimination mode. Then, the corresponding discrimination parameters under the three discrimination modes are substituted into the influence analysis model to generate the first discrimination coefficient, the second discrimination coefficient, and the third discrimination coefficient; Step D3: Construct a three-dimensional dynamic graph with the first discriminant coefficient as the X-axis, the second discriminant coefficient as the Y-axis, and the third discriminant coefficient as the Z-axis. Overlay the three-dimensional dynamic graph with the standard threshold dynamic graph, then obtain its horizontal and vertical screenshots and compare them to generate the total overlapping area cmj and the total non-overlapping area fmj. Set a formula to calculate the loss degree g, and compare and analyze the loss degree with the preset loss interval [G1,G2] to generate the corresponding optimization signal.

4. The improved system for animation sketch generation based on deep learning according to claim 3, characterized in that: The specific process for generating the first discriminant coefficient is as follows: The first discrimination mode is the process of generating a frame image into a two-dimensional sketch: obtaining the first discrimination parameters within a preset time period, the first discrimination parameters include the transformed area and the supplementary line length, and are marked as a1 and a2 respectively; wherein, the supplementary line length BC is the line segment length value for filling and completing the missing area of ​​the target background image; Then, substitute the first discriminant parameter into the influence analysis model to generate the first discriminant coefficient.

5. The improved system for animation sketch generation based on deep learning according to claim 4, characterized in that: The specific process for generating the second discriminant coefficient is as follows: The second discrimination mode is the process of generating a two-dimensional sketch into a three-dimensional sketch: first, a formula is set to obtain the distance energy value P, then the isosurface coefficient DZ corresponding to the distance energy value P is obtained, and the distance energy value P and the isosurface coefficient are combined and the corresponding transformation coefficient is assigned to generate the distance index b1; The shape mean square error value b2 is obtained by setting a formula, and then the distance index b1 and the shape mean square error value b2 are both marked as the second discriminant parameters; Next, obtain the second discrimination parameter within the preset time period, and then substitute the second loss parameter into the impact analysis model to generate the second discrimination coefficient.

6. The improved system for animation sketch generation based on deep learning according to claim 5, characterized in that: The specific process for generating the third discriminant coefficient is as follows: The third discrimination mode is the process of generating a 3D sketch into a real image: The third discrimination mode is the process of generating a 3D sketch into a real image: Obtain the missing parameters in the region within a preset time. The missing parameters include color missing values, contour missing values ​​and texture missing values, and are marked as c1, c2 and c3 respectively; then perform preliminary processing on the missing parameters to generate the missing degree value c4, and collect the missing degree value c4 into the missing parameters to generate the third discrimination parameter; Next, obtain the third discriminant parameter within the preset time period, and then substitute the third loss parameter into the impact analysis model to generate the third discriminant coefficient.

7. The improved system for animation sketch generation based on deep learning according to claim 6, characterized in that: The specific process for preliminary processing of missing parameters is as follows: Step Sa: Collect the duration between the start and end times of the generation process and mark it as the time threshold T. Then divide the time threshold T into m sub-time nodes, where m is greater than 0. Obtain the missing parameters of each time node. Then establish a rectangular coordinate system with time as the X-axis and missing parameters as the Y-axis. Plot the curve of missing parameter change in the rectangular coordinate system by plotting points. In step Sb, a standard threshold curve is preset in the rectangular coordinate system. The missing parameter change curve is compared with the preset change threshold curve to obtain the area enclosed by the line segment of the missing parameter change curve above the standard threshold curve and the standard threshold curve. A circle is drawn with the point on the preset change threshold curve as the center and r as the radius. When the circle completely includes the area, the correction coefficient WX1 for the missing parameter is assigned; otherwise, the correction coefficient WX2 for the missing parameter is assigned. The missing parameter is then multiplied by the corresponding correction coefficient to generate the missing degree value c4.

Images