Animation original drawing layer automatic splitting method based on artificial intelligence
Through the automatic layer splitting method based on deep learning and edge detection, the problem of low efficiency of manual splitting of animation original paintings is solved, efficient and accurate layer splitting is achieved, the cost of VR comics production is reduced and the quality of virtual space construction is improved.
Patent Information
- Application Number
- CN202510862187.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-25
- Publication Date
- 2025-10-03
AI Technical Summary
In the existing technology, the splitting of animation original painting layers mainly relies on manual operation, which leads to high cost and low efficiency in VR comic production, and is prone to errors, affecting the subsequent virtual space construction and interactive effects.
Use deep learning-based image segmentation and edge detection algorithms to automatically identify and split image objects, combine interactive tools for fine-tuning, generate high-quality layer information, and perform post-processing to improve layer quality.
The efficiency of automatic splitting of animation original painting layers has been increased by more than 10 times, which has reduced production costs, reduced errors, and improved the accuracy of layer information and the rationality of subsequent virtual space construction.
Smart Images
Figure CN120747962A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of VR comic technology, and in particular to an artificial intelligence-based method for automatically splitting layers of cartoon original paintings. Background Art
[0002] Virtual reality (VR) technology has experienced rapid global development in recent years, with its applications beginning to permeate every aspect of our lives and work, becoming an essential infrastructure and a form of existence in the "metaverse" era. The United States and China are currently at the forefront of this field. While the popularity of VR headsets in China has also rapidly increased in recent years, the lack of integration between hardware technology and content applications remains a key constraint to the industry's development. With the recent rise of VR devices, research has begun to explore how to integrate VR technology with digital comics, allowing readers to experience comics with three-dimensional virtual reality effects. This has led to the emergence of VR comics.
[0003] In the existing technology, the process of converting traditional two-dimensional comics into VR comics is as follows:
[0004] 1) Split the original comic into comic layers;
[0005] 2) Build a 3D comic book scene in the graphics development engine Unity3D and add interactive programs;
[0006] 3) Add animation, dynamic sales, sound effects, music, dubbing, etc. to the three-dimensional VR comics;
[0007] 4) Synthesize comic stories that can be continuously switched and controlled by clicking.
[0008] However, in the actual VR comic production process, since layer splitting currently relies mainly on manual splitting, the production cost of VR comic works is high and the efficiency is low, which seriously restricts the production speed of VR comics. In addition, manual splitting is prone to errors, resulting in missing layer information or unreasonable model construction, affecting the subsequent virtual space construction and interaction effects. Summary of the Invention
[0009] The purpose of the present invention is to provide an artificial intelligence-based method for automatically splitting layers of animation original pictures to solve the problems raised in the above-mentioned background technology.
[0010] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for automatically splitting layers of an animation original painting based on artificial intelligence, comprising the following steps:
[0011] S1. Image segmentation based on deep learning: Use deep learning models to identify and segment different objects and regions in images;
[0012] S2. Contour analysis based on edge detection algorithm: Use edge detection algorithm to determine the contour of the object;
[0013] S3, layer generation and post-processing: Generate corresponding layer information based on the segmentation and edge detection results, and perform post-processing on the generated layers, such as smoothing and removing redundant information, to improve the layer quality;
[0014] S4. Design and development of interactive segmentation tools: Develop interactive tools that allow users to make fine-tuning corrections, which helps correct possible errors or deficiencies in the model to improve the quality of automatic stratification.
[0015] Furthermore, the step S1 specifically includes the following sub-steps:
[0016] S11. Use pre-trained models or custom trained models to identify specific objects in images, such as people, backgrounds, accessories, etc.
[0017] S12, using a semantic segmentation network to distinguish which object or background each pixel in the image belongs to;
[0018] S13. Analyze color and texture features in images to distinguish different layers and objects.
[0019] Furthermore, in step S13, text data (such as image descriptions, labels, etc.) are combined with multimodal learning methods (such as multimodal embedding and joint learning in deep learning) to better understand and stratify image content.
[0020] Furthermore, in step S2, based on the gradient of the image, an optimized edge detection model operator is used to perform a convolution operation on the image to obtain the gradient, thereby improving the performance of the model, reducing computing resource consumption, and accelerating processing speed.
[0021] Furthermore, the step S2 specifically includes the following sub-steps:
[0022] S21. Read image: use image processing library (such as OpenCV) to read the image to be processed;
[0023] S22. Image preprocessing: including grayscale conversion and noise reduction, converting the input color image into a grayscale image, and using smoothing techniques such as Gaussian filtering to reduce noise in the image;
[0024] S23. Select and optimize edge detection operators: Based on the specific application scenario and image features, select appropriate edge detection operators from Canny, Sobel, Prewitt, Laplacian, and other edge detection operators, and perform operator optimization, including adjusting parameters (such as the high and low thresholds of the Canny operator, which need to be adjusted according to image characteristics), operator simplification (simplifying the operator structure or reducing its size to reduce computational complexity while ensuring detection results), and convolution optimization (using fast convolution algorithms, GPU acceleration, and other technologies to improve the efficiency of convolution operations).
[0025] S24, performing edge detection: performing a convolution operation on the preprocessed image using a selected and optimized edge detection operator to obtain a gradient image, and extracting the image gradient information from the convolution result, including the direction and magnitude of the gradient;
[0026] S25, edge refinement and post-processing: performing non-maximum suppression, double threshold detection, and edge tracking operations on the gradient image to generate the final edge image and extract the contour of the object from the edge image;
[0027] S26. Contour output: output the detected contour in the form of image or data.
[0028] Furthermore, in step S23, the edge detection operators include but are not limited to the Canny operator, the Sobel operator, the Prewitt operator, and the Laplacian operator. For detecting simple horizontal and vertical edges, the Sobel or Prewitt operator is preferably selected, and for detecting edges in complex images, the Canny operator is preferably selected.
[0029] Furthermore, in step S25, edge generation specifically includes the following operations:
[0030] Non-maximum suppression: On the gradient image, only the local maximum points in the gradient direction are retained to refine the edges;
[0031] Dual threshold detection: set two thresholds, high and low. The high threshold is used to preliminarily detect strong edges, and the low threshold is used to connect weak edges to form a complete outline.
[0032] Edge tracing: Use Hough transform or other contour tracing algorithms to connect and organize detected edge points to form continuous contour lines.
[0033] Furthermore, in step S3, the specific operation of layer generation is as follows: receiving the output results of the segmentation algorithm (such as threshold segmentation, region growing, cluster segmentation, etc.) and the edge detection algorithm (such as Canny, Sobel, etc.), and creating a layer for each independent region or object based on the segmentation results. These layers can be binary (containing only the foreground and background), grayscale, or color, depending on the characteristics of the original image and the segmentation algorithm. For the edge detection results, one or more layers can be created to represent different edge types or intensities;
[0034] After the layer is created, label it, that is, assign a unique identifier to each layer for easy reference in subsequent processing, and choose whether to add metadata to the layer as needed, such as the generation time, algorithm parameters used, etc.
[0035] Furthermore, in step S3, the layer post-processing specifically includes:
[0036] Smoothing: Apply smoothing filters (such as mean filters, Gaussian filters, etc.) to reduce noise and unevenness in the layer;
[0037] Remove redundant information: Use morphological operations (such as erosion, dilation, opening, closing, etc.) to remove redundant information such as small isolated areas and burrs on edges. Adjust the parameters of the morphological operations as needed, such as the size and shape of the structural element.
[0038] Connected Domain Analysis: Perform connected domain analysis on a layer to identify and merge adjacent similar regions, reducing fragmentation and redundancy in the layer and improving overall quality.
[0039] Furthermore, in step S4, a graphical user interface (GUI) design tool (such as Qt, Tkinter, etc.) is specifically used to develop an interactive interface, integrating the automatic segmentation algorithm and fine-tuning correction function on the same platform, so as to facilitate one-stop operation for users and provide rich user interaction options, such as undo, redo, zoom in, zoom out, move, etc., to meet the different needs of users.
[0040] The present invention provides an artificial intelligence-based method for automatically splitting layers of animation original paintings, which has the following beneficial effects:
[0041] The present invention uses a deep learning model to identify and segment different objects and areas in the image, and uses an optimized edge detection model operator to perform convolution operations on the image to determine the contours of the objects; the superimposed application of these artificial intelligence technologies realizes the automatic splitting of animation layers, which is an industry first. The splitting efficiency is improved by more than 10 times, which greatly shortens the production cycle. It not only reduces the production cost of VR comic works, but also avoids the problem of easy errors in manual splitting, making the missing layer information or model construction more reasonable, which is conducive to the subsequent virtual space construction and interactive effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a schematic flow chart of the steps of an artificial intelligence-based method for automatically splitting layers of an animation original painting of the present invention;
[0043] Figure 2 This is an example diagram of image segmentation based on deep learning for an artificial intelligence-based method for automatic splitting of animation original painting layers in the present invention;
[0044] Figure 3 This is an example of contour analysis based on edge detection algorithm for an artificial intelligence-based method for automatically splitting animation original painting layers. Figure 1 ;
[0045] Figure 4 This is an example of contour analysis based on edge detection algorithm for an artificial intelligence-based method for automatically splitting animation original painting layers. Figure 2 . DETAILED DESCRIPTION
[0046] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.
[0047] like Figure 1-Figure 4 As shown, a method for automatically splitting layers of anime original paintings based on artificial intelligence includes the following steps:
[0048] S1. Image segmentation based on deep learning: Use deep learning models to identify and segment different objects and regions in the image. This step specifically includes the following sub-steps:
[0049] S11. Use pre-trained models or custom trained models to identify specific objects in images, such as people, backgrounds, accessories, etc.
[0050] S12, using a semantic segmentation network to distinguish which object or background each pixel in the image belongs to;
[0051] S13. Analyze the color and texture features in the image to distinguish different layers and objects, and combine text data (such as image descriptions, tags, etc.) and use multimodal learning methods (such as multimodal embedding in deep learning, joint learning, etc.) to better understand and layer image content, such as Figure 2 shown.
[0052] S2. Contour analysis based on edge detection algorithms: Use edge detection algorithms to determine the contours of the object. This step is based on the image's gradient and uses an optimized edge detection model operator to perform convolution operations on the image to obtain the gradient. This improves the model's performance, reduces computing resource consumption, and speeds up processing. This step specifically includes the following sub-steps:
[0053] S21. Read image: use image processing library (such as OpenCV) to read the image to be processed;
[0054] S22. Image preprocessing: including grayscale conversion and noise reduction, converting the input color image into a grayscale image, and using smoothing techniques such as Gaussian filtering to reduce noise in the image;
[0055] S23. Select and optimize edge detection operators: Based on the specific application scenario and image features, select a suitable operator from edge detection operators such as Canny, Sobel, Prewitt, and Laplacian, and perform operator optimization, including adjusting parameters (such as the high and low thresholds of the Canny operator, which need to be adjusted according to image characteristics), operator simplification (simplifying the operator structure or reducing its size to reduce computational complexity while ensuring detection results), and convolution optimization (using fast convolution algorithms, GPU acceleration, and other technologies to improve the efficiency of convolution operations). Edge detection operators include but are not limited to the Canny operator, Sobel operator, Prewitt operator, and Laplacian operator. For detecting simple horizontal and vertical edges, the Sobel or Prewitt operator is preferred. For detecting edges in complex images, the Canny operator is preferred.
[0056] S24, performing edge detection: performing a convolution operation on the preprocessed image using a selected and optimized edge detection operator to obtain a gradient image, and extracting the image gradient information from the convolution result, including the direction and magnitude of the gradient;
[0057] S25, edge refinement and post-processing: Perform non-maximum suppression, double threshold detection, and edge tracking on the gradient image to generate the final edge image and extract the object contour from the edge image. Specifically, the following operations are performed:
[0058] Non-maximum suppression: On the gradient image, only the local maximum points in the gradient direction are retained to refine the edges;
[0059] Dual threshold detection: set two thresholds, high and low. The high threshold is used to preliminarily detect strong edges, and the low threshold is used to connect weak edges to form a complete outline.
[0060] Edge tracking: Use Hough transform or other contour tracking algorithms to connect and organize detected edge points to form continuous contour lines;
[0061] S26. Contour output: output the detected contour in the form of image or data.
[0062] S3, layer generation and post-processing: Generate corresponding layer information based on the segmentation and edge detection results, and perform post-processing on the generated layers, such as smoothing and removing redundant information, to improve the layer quality.
[0063] In this embodiment, the specific operation of layer generation is as follows: receiving the output results of the segmentation algorithm (such as threshold segmentation, region growing, cluster segmentation, etc.) and the edge detection algorithm (such as Canny, Sobel, etc.), and creating a layer for each independent region or object based on the segmentation results. These layers can be binary (containing only the foreground and background), grayscale, or color, depending on the characteristics of the original image and the segmentation algorithm. For the edge detection results, one or more layers can be created to represent different edge types or intensities; after the layer is created, the layer is marked, that is, a unique identifier is assigned to each layer for easy reference in subsequent processing, and whether to add metadata to the layer, such as the generation time, the algorithm parameters used, etc., can be selected as needed.
[0064] In this embodiment, the layer post-processing specifically includes:
[0065] Smoothing: Apply smoothing filters (such as mean filters, Gaussian filters, etc.) to reduce noise and unevenness in the layer;
[0066] Remove redundant information: Use morphological operations (such as erosion, dilation, opening, closing, etc.) to remove redundant information such as small isolated areas and burrs on edges. Adjust the parameters of the morphological operations as needed, such as the size and shape of the structural element.
[0067] Connected Domain Analysis: Perform connected domain analysis on a layer to identify and merge adjacent similar regions, reducing fragmentation and redundancy in the layer and improving overall quality.
[0068] S4. Design and Develop Interactive Segmentation Tools: Develop interactive tools that allow users to fine-tune and correct potential model errors or deficiencies, thereby improving the quality of automatic stratification. This step involves developing an interactive interface using graphical user interface (GUI) design tools (such as Qt and Tkinter). This interface integrates the automatic segmentation algorithm and fine-tuning functions into a single platform, allowing for one-stop user operation. The interface also provides a variety of user interaction options, such as undo, redo, zoom in, zoom out, and move, to meet diverse user needs.
[0069] The embodiments of the present invention are presented for purposes of illustration and description and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments are chosen and described in order to better illustrate the principles of the invention and its practical application and to enable those skilled in the art to understand the invention and design various embodiments with various modifications as suited for specific applications.
Claims
1. A method for automatically splitting layers of animation original paintings based on artificial intelligence, characterized in that: The following steps are involved: S1. Image segmentation based on deep learning: Use deep learning models to identify and segment different objects and regions in images; S2. Contour analysis based on edge detection algorithm: Use edge detection algorithm to determine the contour of the object; S3, layer generation and post-processing: Generate corresponding layer information based on the segmentation and edge detection results, and post-process the generated layers; S4. Design and development of interactive segmentation tools: Develop interactive tools that allow users to make fine-tuning corrections to improve the quality of automatic stratification.
2. The method for automatically splitting layers of an animation original picture based on artificial intelligence according to claim 1, characterized in that: The step S1 specifically includes the following sub-steps: S11. Use a pre-trained model or a custom trained model to identify specific objects in the image; S12, using a semantic segmentation network to distinguish which object or background each pixel in the image belongs to; S13. Analyze color and texture features in images to distinguish different layers and objects.
3. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 2, characterized in that: In step S13, a multimodal learning method is used in combination with text data to better understand and stratify image content.
4. The method for automatically splitting layers of an animation original picture based on artificial intelligence according to claim 1, characterized in that: In step S2, based on the gradient of the image, an optimized edge detection model operator is used to perform a convolution operation on the image to obtain the gradient, thereby improving the performance of the model, reducing computing resource consumption, and accelerating processing speed.
5. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 1, characterized in that: The step S2 specifically includes the following sub-steps: S21. Read image: use the image processing library to read the image to be processed; S22. Image preprocessing: including grayscale conversion and noise reduction, converting the input color image into a grayscale image, and using Gaussian filtering to reduce noise in the image; S23. Select and optimize edge detection operators: Select appropriate operators based on specific application scenarios and image features, and perform operator optimization, including parameter adjustment, operator simplification, and convolution optimization. S24, performing edge detection: performing a convolution operation on the preprocessed image using an edge detection operator to obtain a gradient image, and extracting the image's gradient information from the convolution result, including the direction and magnitude of the gradient; S25, edge refinement and post-processing: performing non-maximum suppression, double threshold detection, and edge tracking operations on the gradient image to generate the final edge image and extract the contour of the object from it; S26. Contour output: output the detected contour in the form of image or data.
6. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 5, characterized in that: In step S23, edge detection operators include but are not limited to the Canny operator, the Sobel operator, the Prewitt operator, and the Laplacian operator. For detecting simple horizontal and vertical edges, the Sobel or Prewitt operator is preferably selected, and for detecting edges in complex images, the Canny operator is preferably selected.
7. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 5, characterized in that: In step S25, edge generation specifically includes the following operations: Non-maximum suppression: On the gradient image, only the local maximum points in the gradient direction are retained to refine the edges; Dual threshold detection: set two thresholds, high and low. The high threshold is used to preliminarily detect strong edges, and the low threshold is used to connect weak edges to form a complete outline. Edge tracking: Use the Hough transform algorithm to connect and organize the detected edge points to form a continuous contour line.
8. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 1, characterized in that: In step S3, the specific operation of layer generation is as follows: receiving the output results of the segmentation algorithm and the edge detection algorithm, and creating a layer for each independent area or object based on the segmentation results. For the edge detection results, one or more layers can be created to represent different edge types or intensities; After the layers are created, you can tag them by assigning a unique identifier to each layer and optionally add metadata to the layers.
9. The method for automatically splitting layers of anime original paintings based on artificial intelligence according to claim 1, characterized in that: In step S3, the layer post-processing specifically includes: Smoothing: Apply a smoothing filter to reduce noise and unevenness in the layer. Remove redundant information: Use morphological operations to remove redundant information such as small isolated areas and burrs on edges; Connected Domain Analysis: Perform connected domain analysis on a layer to identify and merge adjacent similar areas.
10. The method for automatically splitting layers of animation original pictures based on artificial intelligence according to claim 1, characterized in that: In step S4, a graphical user interface design tool is specifically used to develop an interactive interface, integrating the automatic segmentation algorithm and the fine-tuning correction function on the same platform, facilitating one-stop operation for users and providing rich user interaction options to meet the different needs of users.