A Deep Learning-Based Vectorization Method and System for Black and White Structure Lines in Thangka Painting

Abstract

This invention proposes a deep learning-based method and system for vectorizing the black and white structural lines of Thangka paintings, relating to the field of image processing technology. The method involves: acquiring a black and white structural line raster image; extracting the skeleton and boundaries of the corresponding lines from the raster image, and segmenting the skeleton and boundaries to obtain multiple skeleton segments and multiple boundary segments; performing topological reconstruction based on the selected skeleton and boundary segment information to obtain the topological structure and topological vectors; aligning, cropping, and extracting features from the black and white structural line raster image to output drawing blocks and embedding vectors; inputting the embedding vectors, topological vectors, and state parameters into an RNN decoder to generate stroke parameters for each time step; converting the stroke parameters into a local 2D raster image through differential rendering; and pasting the local 2D raster image back onto the global canvas to obtain a vectorized raster image. This application helps improve the automation and accuracy of converting black and white structural line raster images into vector representations.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for vectorizing the black and white structural lines of Thangka paintings based on deep learning. Background Technology

[0002] Currently, most Thangka images are still hand-painted by Thangka artists. Experienced artists typically begin by selecting and combining a series of elements, and depending on the size and complexity of the combination, it can take months or even years to complete a single Thangka image. Existing digital Thangka images are generally stored as raster images. Image information stored as pixel matrices is not conducive to image editing and design, while vector graphics, due to their geometric representation advantages, are not only unaffected by resolution during rendering but also facilitate artist creation.

[0003] Chinese patent CN118710552A discloses a method, system, and storage medium for Thangka image restoration. The method includes: segmenting the input image into non-overlapping sub-regions and performing non-linear transformations; simultaneously introducing a vector quantization codebook to better capture and preserve image structural information and details; using a parallel CSWin resolution Transformer module to enhance contextual modeling capabilities through a cross-shaped window and local enhancement positional encoding; and a novel multi-scale feature guidance module adaptively learning feature information of non-missing regions through local knowledge at different scales. However, the above scheme employs non-overlapping sub-region segmentation and non-linear transformations, resulting in inconsistent-sized drawing blocks after alignment and cropping, and the output is a reconstructed grayscale / phase image, which can easily affect the accuracy of subsequent feature extraction and decoding. Therefore, it is essential to provide a deep learning-based method and system for vectorizing Thangka black and white structural lines to improve the automation and accuracy of converting black and white structural line raster images into vector representations. Summary of the Invention

[0004] In view of this, this invention proposes a deep learning-based method and system for vectorizing the black and white structural lines of Thangka paintings. The thinning algorithm extracts the skeleton and boundaries simultaneously, and the topological vectors generated by topological reconstruction fully encode the connectivity and bifurcation structure of the lines. The dynamic window mechanism, designed for structural lines of different densities and thicknesses, significantly improves the automation and accuracy of converting black and white structural line raster images into vector representations.

[0005] This invention provides a deep learning-based method for vectorizing the black and white structural lines of Thangka paintings, the method comprising:

[0006] Acquire a black-and-white structured line raster image to be processed;

[0007] Extract the skeleton and boundary of the corresponding lines in the black and white structured line raster image, and segment the skeleton and boundary to obtain multiple skeleton segments and multiple boundary segments;

[0008] Reliability screening is performed on all adjacent skeleton segments, and topology reconstruction is performed based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by encoding the topology structure.

[0009] The black and white structure line raster image processed by the dynamic window mechanism is input into the CNN encoder, and the black and white structure line raster image is aligned, cropped and feature extracted to output drawing blocks of the same size and embedding vectors corresponding to the drawing blocks.

[0010] The embedding vector, the topology vector, and the state parameters are input into the RNN decoder to generate stroke parameters corresponding to each time step. The stroke parameters are converted into local 2D raster images through differential rendering, and the local 2D raster images are pasted back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed, and the vectorized raster image is obtained.

[0011] Based on the above technical solutions, preferably, the reliability screening of all adjacent skeleton segments specifically includes:

[0012] Calculate the directional similarity between the first skeleton segment and the second skeleton segment, wherein the first skeleton segment and the second skeleton segment are adjacent skeleton segments;

[0013] If the directional similarity between the first skeleton segment and the second skeleton segment is greater than the similarity threshold, then the first skeleton segment and the second skeleton segment are determined to be continuous skeleton segments.

[0014] Calculate the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment;

[0015] If the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is less than the consistency threshold, then the first skeleton segment is determined to be an unreliable skeleton segment, and the unreliable skeleton segment is removed.

[0016] Based on the above technical solutions, preferably, the alignment, cropping, and feature extraction of the black and white structured line raster image specifically includes:

[0017] The dynamic window to be cropped in the black and white structured line raster image is divided into multiple spatial grid blocks of the same size;

[0018] Multiple sampling points are set within each spatial grid block. The value of each sampling point within the spatial grid block is calculated using bilinear interpolation. The average value of each sampling point within the current spatial grid block is then used to obtain the pixel value corresponding to the spatial grid block.

[0019] The spatial grid block is input into the CNN encoder for feature extraction to obtain the embedding vector corresponding to the spatial grid block.

[0020] More preferably, the step of inputting the embedding vector, the topology vector, and the state parameters into the RNN decoder to generate stroke parameters corresponding to each time step specifically includes:

[0021] Based on the image feature information of the current drawing block in the embedding vector and the image structure topology information corresponding to the current drawing block in the topology vector, the RNN decoder outputs the initial stroke parameters and the hidden state update function.

[0022] The stroke parameters for each time step are generated based on the width factor of the previous time step, the cursor position of the previous time step, the initial stroke parameters, and the hidden state update function.

[0023] More preferably, the step of converting the stroke parameters into a local 2D raster image through differential rendering specifically includes:

[0024] The stroke parameters are sequentially input into a multi-layer fully connected layer, a multi-layer convolutional layer, and an activation function in the differentiable rendering module to convert the stroke parameters into a two-dimensional rasterized image. The multi-layer fully connected layer is used to convert the features corresponding to the stroke parameters into three-dimensional tensors to form a feature map; the multi-layer convolutional layer is used to rearrange the pixels of the feature map to obtain a raster image with a target resolution; and the activation function is used to map the raster image with the target resolution to obtain a local 2D raster image.

[0025] More preferably, after completing the vectorization of the black and white structure line raster image and obtaining the vectorized raster image, the method further includes:

[0026] Based on the vectorized raster image and the target image, all loss functions are summed with preset weights to obtain the total loss function. The CNN encoder and the RNN decoder are then backpropagated based on the total loss function. The loss function includes a perceptual loss function, an out-of-bounds penalty loss function, a stroke regularization loss function, a smoothness loss function, and a topological consistency loss function.

[0027] More preferably, the stroke parameters include quadratic Bézier curve control points, stroke width factor, softened pen lifting state, and softened pen ending state.

[0028] A second aspect of this application provides a deep learning-based vectorization processing system for the black and white structural lines of Thangka paintings. This system includes an acquisition and segmentation module, a data processing module, and an image vectorization module.

[0029] The acquisition and segmentation module is used to acquire the black and white structure line raster image to be processed, extract the skeleton and boundary of the corresponding line in the black and white structure line raster image through a thinning algorithm, and segment the skeleton and the boundary to obtain multiple skeleton segments and multiple boundary segments.

[0030] The data processing module is used to perform reliability screening on all adjacent skeleton segments, and to perform topology reconstruction based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by the topology structure encoding. The black and white structure line raster image processed by the dynamic window mechanism is input into the CNN encoder, and the black and white structure line raster image is aligned, cropped and feature extracted to output drawing blocks of the same size and the embedding vector corresponding to the drawing blocks.

[0031] The image vectorization module is used to input the embedding vector, the topology vector, and the state parameters into the RNN decoder to generate stroke parameters corresponding to each time step. The stroke parameters are converted into local 2D raster images through differential rendering, and the local 2D raster images are pasted back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed, and the vectorized raster image is obtained.

[0032] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0033] In a fourth aspect, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a deep learning-based vectorization processing method for the black and white structural lines of Thangka paintings.

[0034] The deep learning-based vectorization method for the black and white structural lines of Thangka paintings provided in this invention has the following advantages over existing technologies:

[0035] (1) By extracting the skeleton and boundary together through the thinning algorithm and performing reliability screening on adjacent skeleton segments, noise, breakpoints and pseudo-thin lines can be effectively removed, ensuring the accuracy of subsequent contour segmentation and topological relationships. Moreover, the topological vector generated by topological reconstruction fully encodes the connectivity and bifurcation structure of the lines. The dynamic window mechanism adaptively selects the appropriate field of view for structural lines of different densities and thicknesses, avoiding information redundancy or loss. Alignment clipping ensures that each drawing block is consistent in space, scale and direction, so that the embedding vector extracted by the CNN encoder has high comparability and splicability. At the same time, in the RNN The decoder performs differentiable rendering on the stroke parameters output at each step and writes back to the global canvas in a differentiable tiling manner, constructing an end-to-end vectorization process that can be reverse-optimized. The model can continuously adjust the stroke width, curvature, and position itself. The final output vectorized graphics retain the original details and have SVG / CAD level continuity and scalability, significantly improving the automation and accuracy of converting black and white structural line raster images into vector representations. It takes into account both structural topology and visual details, and is well adapted to black and white line drawings of different styles and noise levels. It is especially good at traditional paintings with complex patterns and rich details.

[0036] (2) By using bilinear interpolation to generate multi-point sampling within each grid block, sub-pixel-level grayscale estimation of dynamic windows under arbitrary translation and micro-rotation is realized, which greatly reduces pixel jitter and geometric error during cropping. Furthermore, the average value of multiple sampling points within the grid block is equivalent to performing local low-pass filtering on the original raster image, which effectively smooths noise and avoids aliasing artifacts caused by direct downsampling, thus improving the robustness of subsequent feature extraction. At the same time, dynamic windows of arbitrary size and shape are uniformly divided into grid blocks of equal size, so that the drawing blocks input to the CNN encoder always maintain the same spatial resolution and receptive field, eliminating feature scale drift caused by different positions and different local proportions, significantly reducing the sensitivity of the CNN encoder to simple geometric deviations and scanning noise during the training stage, accelerating the network convergence speed, and improving the generalization ability of line drawings of different styles. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A flowchart illustrating the deep learning-based vectorization method for Thangka black and white structural lines provided by this invention;

[0039] Figure 2This is a schematic diagram of the stroke generation module provided by the present invention;

[0040] Figure 3 A schematic diagram of the structure of the differentiable rendering module provided by the present invention;

[0041] Figure 4 This is a schematic diagram of the vectorization processing system for the black and white structural lines of Thangka provided by the present invention;

[0042] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0043] Explanation of reference numerals in the attached diagram: 1. Vectorization system for Thangka black and white structural lines; 11. Acquisition and segmentation module; 12. Data processing module; 13. Image vectorization module; 2. Electronic equipment; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0045] Because the Thangka dataset is relatively small and insufficient for effective training, this paper chooses the Quick,Draw! dataset as supplementary data. This publicly available dataset contains a large number of user-drawn sketches and was used by Google to train the Sketch-RNN model. It includes tens of millions of drawings by users, encompassing 345 categories of objects and concepts, such as animals, daily necessities, and vehicles. The images in the dataset were initially saved in vector format, recording the start and end points of each stroke and the drawing order. Furthermore, although the lines in the images are relatively simple, the shapes and structures of the lines in the sketches exhibit considerable diversity due to differences in user drawing habits. This diversity is also significant for improving the model's generalization ability.

[0046] The above dataset is formatted with reference to the stroke-3 format in the SketchRNN model, which is a combination of relative coordinates and stroke states. In this format, each stroke consists of a relative position offset and a stroke state. The relative position offset (Δ...) x ,Δ y This refers to the coordinate offset relative to the previous point, and this offset ranges from [-1, 1]. 2 Stroke status ( pThe value range of ) is [0,1]. The stroke state is used to record the state of the stroke. Specifically, a value of 1 indicates that a line is currently being drawn, while a value of 0 indicates that the pen is lifted, preparing to draw the next independent line or move to the next drawing position. This state indication can effectively help the model distinguish different strokes and ensure the correctness of the drawing order.

[0047] This invention discloses a deep learning-based method for vectorizing the black and white structural lines of Thangka paintings, with reference to... Figure 1 The steps of this method include S1 to S4.

[0048] Step S1: Acquire the black and white structured line raster image to be processed;

[0049] Step S2: Extract the skeleton and boundary of the corresponding lines in the black and white structure line raster image, and segment the skeleton and boundary to obtain multiple skeleton segments and multiple boundary segments.

[0050] In this step, the stroke generation module progressively generates vector lines in the image, enabling the model to gradually transform the input image into a continuous sequence of lines. The main task of the stroke generation module is to generate corresponding vectorized stroke sequences based on the input image information.

[0051] Each stroke is offset by a relative coordinate offset (Δ) x ,Δ y ) and pen status p This describes its position and drawing state. However, to generate smoother and more flexible curves, and to achieve a high degree of expressiveness in vectorized images, the strokes are represented as follows:

[0052] α t =( x c , y c ,Δ x ,Δ y , w ,Δ s , p ) t

[0053] in,( x c , y c )∈[-1,1] 2 These are the intermediate control points of the Bézier curve, used to construct a quadratic Bézier curve; w ∈[0,1] is used to control different widths, Δ s ∈[0, k Used to control the size of the drawing window.

[0054] From control point P 0( x 0, y 0) P 1( x 1, y 1) and P 2( x 2, y 2) The specified quadratic Bézier curve is represented as:

[0055] B ( τ )=(1- τ ) 2 P 0+2(1- τ ) τP 1+ τ 2 P 2 τ ∈[0,1]

[0056] Define the stroke parameters of a quadratic Bézier curve as follows:

[0057] q t =( x 0, y 0, x 1, y 1, x 2, y 2, r 0, r 2) t

[0058] in,( x 0, y 0) t =(0,0), ( x 1, y 1) t =( x c , y c ), ( x 2, y 2) t =(Δ x ,Δ y ), ( r 0) t = w t-1 , ( r 2) t = w t .

[0059] In addition, the stroke state is a continuous value. P Since the integers ∈[0,1] cannot be directly used as binary states, softagmax is used to process them into approximate binary states.

[0060]

[0061] Among them, the proportionality coefficient β =10, i Indicates the first in the image i coordinates, j Indicates the first in the image j Coordinates.

[0062] Furthermore, during the vectorization of line art, the topology is used to record the connections between adjacent lines. Each line may intersect or connect with other lines at multiple points, and the topology describes these relationships to ensure that the generated vector lines conform to the connection patterns of the original image. For example, at "T" and "X" shaped connection points, topology information helps the model correctly handle intersections.

[0063] The input image is thinned using classic thinning algorithms (such as the Zhang-Suen algorithm) to extract the skeleton and boundaries of the lines. For the skeleton, two important node types are defined: endpoints and intersections. Endpoints (valence=1) represent the start or end point of a line; intersections (valence≥3) represent nodes where multiple line segments intersect. After the skeleton and boundaries are extracted, the skeleton is further segmented into a series of skeleton segments composed of connected pixels. Simultaneously, the boundaries are divided into multiple boundary segments to ensure a correspondence between the boundaries and the skeleton.

[0064] Step S3: Reliability screening is performed on all adjacent skeleton segments, and topology reconstruction is performed based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by the topology structure encoding.

[0065] This step also includes steps S31 to S34.

[0066] Step S31: Calculate the directional similarity between the first skeleton segment and the second skeleton segment, wherein the first skeleton segment and the second skeleton segment are adjacent skeleton segments.

[0067] Step S32: If the directional similarity between the first skeleton segment and the second skeleton segment is greater than the similarity threshold, then the first skeleton segment and the second skeleton segment are determined to be continuous skeleton segments.

[0068] In this step, after skeleton extraction, the reliability of the skeleton segments needs to be screened to ensure accurate topology generation in connected or intersecting regions. For the connection relationship between two adjacent skeleton segments, the following formula is used to determine whether they should be connected:

[0069]

[0070] in, t ( c i ) represents skeleton point c i tangent direction, b l ( c i ) represents skeleton point c i The left boundary point, b r ( c i ) represents skeleton point c i If the directional similarity between the right boundary points meets a certain threshold (such as 0.99), then the two skeleton segments are considered to be continuous.

[0071] Step S33: Calculate the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment.

[0072] Step S34: If the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is less than the consistency threshold, then the first skeleton segment is determined to be an unreliable skeleton segment and the unreliable skeleton segment is removed.

[0073] In this step, when there is a correspondence between the skeleton segment and the boundary segment, the topological correctness can be further verified by calculating the consistency of the tangents between the boundary and the skeleton. For Projected onto skeleton points c i :

[0074]

[0075] in, c i Indicates the first on the skeleton i One point, b i Indicates the first on the skeleton i The boundary points corresponding to each point. t ( b i The tangent direction at the boundary point is indicated by ). When the consistency is below a threshold (e.g., 0.95), the skeleton segment is considered unreliable and is removed.

[0076] After removing unreliable skeleton segments, the skeleton structure is topologically reconstructed using boundary segment information. At intersections or connection points, the boundary lines near the point are observed first. Since boundary lines are usually relatively complete in the image, they can provide the overall contour shape of the region. When breaks or unreliable parts are detected in the skeleton, reasonable paths to connect these breaks are found by comparing them with the boundaries.

[0077] In some special cases, boundary segment information may not be sufficient to provide effective reconstruction clues (such as at very complex intersections or certain boundary breaks). By utilizing the directional information (tangent direction) of adjacent points on the skeleton, the connection pattern of the lines can be estimated, predicting the extension path of the next skeleton segment. This "blind" tangent matching method can roughly recover a reasonable connection path.

[0078] Furthermore, after completing the topology construction and reconstruction of the skeleton and boundaries, the topological information can be encoded into topological vectors. Each topological vector describes the geometric position, orientation, and connectivity of a key point, and includes the following:

[0079] topo i =[ x i , y i , θ i , d i , c i1 ,..., c in ]

[0080] in,( x i , y i () represents the coordinates of the key point. θ i This indicates the angle or direction between the keypoint and its adjacent keypoints. d i Indicates the type of point (intersection, endpoint). c i1 ,..., c in This indicates the connection relationship between this key point and other key points.

[0081] Step S4: Input the black and white structure line raster image processed by the dynamic window mechanism into the CNN encoder, and perform alignment cropping and feature extraction on the black and white structure line raster image to output drawing blocks of the same size and the embedding vectors corresponding to the drawing blocks.

[0082] In this step, the CNN encoder is responsible for extracting high-level features from local regions of the input image (i.e., the currently drawn region) and generating embedding vectors suitable for processing by the RNN decoder. The encoder's input includes cropped image patches and the global image and canvas. The cropped image patches are small 128×128 pixel blocks cropped from the image and canvas according to the current drawing window. The global image and canvas are scaled down to the same size as the small patches and serve as additional global guidance information to help the model identify regions that have not yet been drawn.

[0083] like Figure 2 As shown, in the design of the CNN encoder, the first layer uses CoordConv, a convolutional operation that incorporates coordinate information, which enhances the model's perception of spatial location in the image. Subsequently, the encoder employs multiple convolutional layers, sequentially adding instance normalization and ReLU activation functions to extract high-level features from the image. The specific structure is shown in Table 1.

[0084] Table 1

[0085]

[0086] After the convolutional layer processing, the encoder flattens the output features into a one-dimensional vector and inputs it into a fully connected layer (FC), which further compresses the flattened features into a 128-dimensional embedding vector. Z t It is used to characterize the detailed features and overall contextual information of the current local region.

[0087] This step also includes steps S41 to S43.

[0088] Step S41: Divide the dynamic window to be cropped in the black and white structure line raster image into multiple spatial grid blocks of the same size.

[0089] In this step, handling images of arbitrary resolution is a challenge for most traditional learning-based vector graphics generation algorithms. The introduction of a dynamic window mechanism effectively solves this problem, as it updates its position and size at every moment—for example, expanding the window to draw long strokes and shrinking it to draw details. The dynamic window mechanism manages the model's drawing window, allowing the model to focus on drawing the currently needed area. The model uses a virtual pen and dynamic windows... Q The combination of these factors, achieved by setting the window scaling factor Δ at each time step, allows for adjustments to the window scaling factor. s To change its size W Given a window scaling factor Δ s And the movement of the window Δ Qt =(Δ x ,Δ y ) t ∈[-1,1] 2 The update rules for dynamic windows are as follows:

[0090]

[0091]

[0092]

[0093]

[0094] in, W I Indicates the size of the input image. W min This represents the minimum value of a predefined dynamic window. Initial value. W 0 is set to 128×128. Q 0 represents a random position.

[0095] Step S42: Set multiple sampling points within each spatial grid block, calculate the value of each sampling point within the spatial grid block using bilinear interpolation, and average the value of each sampling point within the current spatial grid block to obtain the pixel value corresponding to the spatial grid block.

[0096] In image processing tasks, traditional cropping methods (such as RollPooling) typically employ nearest-neighbor interpolation. After scaling the image's position and coordinates, they directly use a crude rounding operation for floating-point numbers. This pixel deviation introduces quantization errors during alignment (which typically amplify with increasing quantization iterations). This not only affects the accuracy of the cropped content but also hinders gradient propagation across upstream and downstream modules, preventing effective end-to-end model training.

[0097] In the Mask R-CNN work, the RoIAlign method was used to address the problem caused by quantization errors. As an improvement on RoIPooling, RoIAlign accurately captures image features by dividing each cropping window into multiple spatial grids and performing bilinear interpolation sampling within each grid. Specifically, RoIAlign sets multiple sampling points (typically one or four) within each grid, calculates the pixel value of the sampling point using bilinear interpolation with the four pixels closest to it, and averages these values ​​to obtain the final value for the grid, thus avoiding quantization errors. This method not only maintains the consistency between the cropping region and the window but also allows gradients to flow throughout the cropping operation, thereby supporting end-to-end model training.

[0098] During the dynamic window cropping process, the stroke generation module uses RoIAlign to ensure that the cropping window remains consistent with the floating dynamic window. The specific steps are as follows: based on the window size... W t-1 and resampling size W r =128×128, dividing the window into spatial grid blocks. Several sampling points are set within each grid, and the values ​​of these sampling points are calculated using bilinear interpolation. The final value of each grid block is obtained by averaging these values. The resulting image block size after cropping is fixed at 128×128, thus eliminating the need for quantization operations, avoiding quantization errors, and ensuring that the cropped area always maintains precise alignment with the dynamic window.

[0099] Furthermore, the coordinate system of the original image space is defined as follows: coord The coordinate system of the cropped image patch space is defined as follows: coord' . win 1 represents the initial cropping window, which is in the original image space. coord The area to be clipped is usually represented by a floating window; win2 This represents the adjusted cropping window, its position and size are... coord The coordinates are determined in the system (by integer alignment) and then transformed to the coordinate system of the cropped image. coord' middle.

[0100] Initial cropping window win The top left position of 1 is represented as:

[0101] ;

[0102] Initial cropping window win The bottom right position of 1 is represented as:

[0103] ;

[0104] Adjusted cropping window win2 The position is represented as:

[0105]

[0106] ;

[0107] Adjusted cropping window win2 Center position and dimensions:

[0108]

[0109] ;

[0110] In the coordinate system of the cropped image coord' Adjusted cropping window win2 Central location:

[0111] ;

[0112] In the coordinate system of the cropped image coord' Adjusted cropping window win2 Size:

[0113] .

[0114] Step S43: Input the spatial grid block into the CNN encoder for feature extraction to obtain the embedding vector corresponding to the spatial grid block.

[0115] In this embodiment, bilinear interpolation is used to generate multi-point sampling within each grid block, enabling sub-pixel-level grayscale estimation of the dynamic window under arbitrary translation and micro-rotation conditions, greatly reducing pixel jitter and geometric errors during cropping. Averaging the values ​​of multiple sampling points within the grid block is equivalent to performing a local low-pass filter on the original raster image, effectively smoothing noise and avoiding aliasing artifacts caused by direct downsampling, thus improving the robustness of subsequent feature extraction. Unifying the dynamic window of arbitrary size and shape into uniformly sized grid blocks ensures that the drawing blocks input to the CNN encoder always maintain the same spatial resolution and receptive field, eliminating feature scale drift caused by different positions and local proportions. Since all grid blocks adopt the same sampling-means strategy, their output embedding vectors have consistent statistical distribution and semantic alignment characteristics, ensuring natural and seamless contextual connections between blocks when the RNN decoder generates strokes sequentially. The combined effect of subpixel alignment and noise suppression significantly reduces the sensitivity of the CNN encoder to simple geometric deviations and scanning noise during the training phase, accelerates network convergence, and improves its generalization ability to line art of different styles (thickness, texture, noise). The gridded sampling-means mechanism preserves line direction and subtle differences in shade while avoiding "granularity" or "blurring" phenomena, achieving high-fidelity restoration of the original black and white structural lines and laying a solid foundation for accurate fitting of subsequent vector stroke parameters.

[0116] Step S5: Input the embedding vector, topological vector, and state parameters into the RNN decoder to generate stroke parameters corresponding to each time step. Convert the stroke parameters into a local 2D raster image through differential rendering, and paste the local 2D raster image back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed, and obtain the vectorized raster image.

[0117] This step also includes steps S51 to S53.

[0118] Step S51: Based on the image feature information of the current drawing block in the embedding vector and the image structure topology information corresponding to the current drawing block in the topology vector, the RNN decoder outputs the initial stroke parameters and the hidden state update function.

[0119] In this step, the RNN decoder uses Long Short-Term Memory (LSTM) network units to progressively generate stroke parameters for each time step. When generating the stroke at each step, the RNN decoder receives the embedding vector generated by the encoder. Z t Topological information vector topo t and the hidden state of the previous time step. h t-1 The embedding vectors generated by the encoder provide image feature information for the current drawing window. The topological information vector contains topological information about the image structure, such as line connections, intersections, and contour shapes. This information is provided in the form of topological vectors or graph embedding vectors, allowing the decoder to reference the overall structural relationships at each generation step. The hidden state from the previous time step is used to synthesize these inputs when generating stroke parameters at each step; the RNN decoder then outputs the stroke parameters. α t And updated hidden status h t .

[0120] In addition, the model incorporates the width factor from the previous time step. w t-1 The cursor position at the previous time step Q t-1 The upper and lower bounds of the scaling factor are used to optimize the decoder output. The width factor of the previous time step is used to predict the width of the current time step. w t This helps the decoder maintain consistent stroke widths. The cursor position of the previous time step provides the drawing position of the previous time step, ensuring that the model is not drawn outside the canvas in the current time step. The upper and lower bounds of the scaling factor provide the upper and lower bounds of the window scaling factor, helping the decoder to keep the window size within a specified range when adjusting it, and are defined as follows:

[0121]

[0122]

[0123] Step S52: Based on the width factor of the previous time step, the cursor position of the previous time step, the initial stroke parameters, and the hidden state update function, generate the stroke parameters corresponding to each time step. The stroke parameters include the quadratic Bézier curve control points, the stroke width factor, the softened pen lifting state, and the softened pen ending state.

[0124] Step S53: The stroke parameters are sequentially input into the multi-layer fully connected layer, multi-layer convolutional layer, and activation function in the differentiable rendering module to convert the stroke parameters into a two-dimensional rasterized image. The multi-layer fully connected layer is used to convert the features corresponding to the stroke parameters into three-dimensional tensors to form a feature map; the multi-layer convolutional layer is used to rearrange the pixels of the feature map to obtain a raster image with the target resolution; and the activation function is used to map the raster image with the target resolution to obtain a local 2D raster image.

[0125] like Figure 3 As shown, the goal of the differentiable rendering module is to convert the predicted 1D vector parameters into a 2D raster image. This allows gradients to be backpropagated from the rendered output to the stroke parameters during training using raster-level loss, ensuring end-to-end training of the system. The specific structure of the differentiable rendering module is shown in Table 2.

[0126] Table 2

[0127]

[0128] Furthermore, for a given quadratic Bézier curve parameter q t Through multiple fully connected layers, the feature representation is gradually transformed from 10 dimensions to a higher dimension. This step uses the ReLU activation function to ensure the non-linear expressiveness of the features. After the fully connected layers, the features are reshaped into a three-dimensional tensor, making its spatial dimension conform to a shape of 16×16×16. In the next stage, the feature map undergoes multiple convolutions, where each convolution layer uses the ReLU activation function to increase the non-linearity of the features. Pixel Shuffle is an upsampling strategy whose core idea is to rearrange the channel information in the feature map into spatial dimensions, gradually transforming the image from low resolution to high resolution. Finally, after several pixel shuffles, the image is upsampled from 16×16 to 128×128, thus obtaining a raster image of the target resolution. After convolution and upsampling operations, the final output tensor is processed by the Sigmoid activation function, making the pixel values ​​map to the range [0,1]. The final output is a raster image with a resolution of ( N Two-dimensional rasterized image (128, 128) S t The image is tiled onto a full-resolution canvas based on the window coordinates and size. The differentiable tile module in the model is used to paste the generated local stroke images back onto the global canvas in a similar manner.

[0129] Furthermore, based on the vectorized raster image and the target image, all loss functions are summed with preset weights to obtain the total loss function, and the CNN encoder and RNN decoder are backpropagated based on the total loss function. The loss functions include the perceptual loss function, the out-of-bounds penalty loss function, the stroke regularization loss function, the smoothness loss function, and the topological consistency loss function.

[0130] In this embodiment, perceptual loss is a loss function that measures the high-level semantic similarity between the generated vectorized image and the target image. Unlike traditional pixel-level losses (such as L1 or L2 loss), perceptual loss focuses on the semantic consistency and global structure of the image. Specifically, it extracts features through a pre-trained convolutional neural network (such as a VGG network), calculates the difference between the generated image and the target image in the feature space, and thus obtains a result that is more consistent with human visual perception.

[0131] For a given generated image and target image ( y ), through sensor networks Φ Extract their feature maps. In the perceptron network... j The layer, the extracted feature map is defined as Φ j (), its dimension is D j × H j × W j , representing the number of channels, height, and width of the feature, respectively. j The perceptual loss of a layer is defined as:

[0132]

[0133] The overall perception loss is a weighted sum of the losses of all selected layers:

[0134]

[0135] The relu1_2, relu2_2, relu3_3 and relu5_1 layers in the VGG-16 model were selected to calculate the perceptual loss.

[0136] During training, the model gradually generates images by dynamically moving and scaling a window relative to the image. However, this relative movement and scaling can cause the window to extend beyond the image boundaries, a phenomenon known as "out-of-bounds" errors. To avoid this, the model introduces an out-of-bounds penalty loss to constrain the dynamic window's movement and scaling within the image's boundaries, thereby guiding the model to generate valid, in-bounds image content.

[0137] Penalty for Stroke Offsets: This section primarily addresses the issue of stroke parameters going out of bounds due to relative movement (offset). If the window's position exceeds the image boundary after relative movement, a penalty is applied. Specifically, given the cursor position at each step... and window size The values ​​after relative shifting and scaling, and their corresponding boundary clipping values. Q t and W t Then the boundary penalty for the shift offset can be expressed as:

[0138]

[0139] in, T This represents the total number of moves. This represents the distance beyond the boundary. The formula penalizes instances where the offset exceeds the image boundary by summing all distances beyond the boundary.

[0140] Out-of-boundary penalty for scaling factor: The window scaling factor Δ also needs to be limited. s Within an appropriate range. The upper bound of a given window. W max and the lower realm W min The out-of-bounds penalty loss for scaling factors is defined as follows:

[0141]

[0142] The out-of-bounds penalty loss is defined as:

[0143] L out = L mo + L so

[0144] In the use of vector graphics, excessive redundant or loose strokes increase the complexity of editing. Therefore, stroke regularization loss is introduced to reduce redundant strokes in the generated image. To minimize the number of strokes while maintaining the visual similarity of the image, the formula for stroke regularization loss is defined as follows:

[0145]

[0146] in, p t ∈[0,1], p t =1 indicates that drawing is currently in progress. p t=0 indicates that the pen has been lifted. In the actual calculation, softagmax was used for approximate binarization.

[0147] When generating curves, it is generally desirable for the changes between adjacent control points to be smooth and continuous, rather than with large abrupt changes, which can lead to unnatural sharp angles in the curve. Smoothness loss achieves smoother curve generation by constraining the distance changes between adjacent control points, minimizing the difference in distance between them.

[0148] Let the first t The first control point and the first t+ The distance between 1 control point is:

[0149]

[0150] To ensure the smoothness of the curve, the smoothness loss is defined as the average of the sum of the squares of the differences between adjacent distances:

[0151] .

[0152] Topology consistency loss aims to enhance the accuracy of skeleton structure reconstruction at complex intersections, such as T-shaped and X-shaped connection points. This loss function guides the model to ensure that strokes at these intersections are connected in a reasonable topological order, avoiding incorrect connections or gaps.

[0153] Topology consistency loss primarily focuses on two aspects: tangent consistency and segment continuity. Tangent consistency ensures that connection points maintain alignment with the boundary in both tangent direction and location to conform to the topology. Segment continuity ensures that adjacent segments connect smoothly while remaining aligned with the boundary, avoiding abrupt changes or deviations from the expected path. For a given set of adjacent skeleton points... p i and p i+1 and the corresponding boundary points b i and b i+1 The loss formula is designed as follows:

[0154]

[0155]

[0156] L topo = αL tc +βL ba .

[0157] The overall loss function is defined as:

[0158] L total = L perc +λ out L out +λ reg L reg +λ smooth L smooth +λ topo L topo .

[0159] Furthermore, in vectorization tasks, measuring the similarity between rendered input and output remains an open question, and it is currently difficult to find a similarity metric that perfectly aligns with human perception. Chamfer distance can effectively measure the similarity between graphs, and its calculation formula is as follows:

[0160]

[0161] in, and These are two point sets, representing all sampled points in the target image and the rendered output image, respectively.

[0162] On the other hand, the Chamfer distance primarily focuses on the overall shape and structure of a graphic, lacking sensitivity to details within the image. This makes it potentially inaccurate in reflecting the true quality of rendered images with complex details or blurred edges. To address this deficiency, this paper introduces the PerceptualScore, a metric based on intermediate layer features of deep neural networks, which pays greater attention to the similarity of details, textures, and complex structures within an image. Its calculation formula is as follows:

[0163]

[0164] in, and It is an image I 1 and I 2. Feature representation of the intermediate layer in the VGG-16 network This represents the square of the Euclidean distance, used to calculate the difference between feature maps.

[0165] In this embodiment, the skeleton and boundaries are extracted together by a thinning algorithm, and the reliability of adjacent skeleton segments is screened. This effectively removes noise, breakpoints, and pseudo-thin lines, ensuring the accuracy of subsequent contour segmentation and topological relationships. Furthermore, the topological vectors generated by topological reconstruction fully encode the connectivity and bifurcation structure of the lines. The dynamic window mechanism adaptively selects appropriate views for structural lines of different densities and thicknesses, avoiding information redundancy or loss. Alignment clipping ensures that each drawing block is consistent in space, scale, and direction, thereby making the embedding vectors extracted by the CNN encoder highly comparable and splicable. At the same time, in R... The NN decoder performs differentiable rendering on the stroke parameters output at each step and writes back to the global canvas in a differentiable tiling manner, constructing an end-to-end vectorization process that can be reverse-optimized. The model can continuously adjust the stroke width, curvature, and position itself. The final output vectorized graphics not only retain the original details but also have SVG / CAD level continuity and scalability, significantly improving the automation and accuracy of converting black and white structural line raster images into vector representations. It takes into account both structural topology and visual details, and is well adapted to black and white line drawings of different styles and noise levels. It is especially good at traditional paintings with complex patterns and rich details.

[0166] Based on the above method, this application discloses a deep learning-based vectorization processing system for the black and white structural lines of Thangka paintings, with reference to... Figure 4 The Thangka black and white structural line vectorization processing system 1 includes an acquisition and segmentation module 11, a data processing module 12, and an image vectorization module 13, wherein...

[0167] The acquisition and segmentation module 11 is used to acquire the black and white structure line raster image to be processed, extract the skeleton and boundary of the corresponding line in the black and white structure line raster image through a thinning algorithm, and segment the skeleton and boundary to obtain multiple skeleton segments and multiple boundary segments.

[0168] The data processing module 12 is used to perform reliability screening on all adjacent skeleton segments and perform topology reconstruction based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by the topology structure encoding. The black and white structure line raster image processed by the dynamic window mechanism is input into the CNN encoder, and the black and white structure line raster image is aligned, cropped and feature extracted to output drawing blocks of the same size and the embedding vector corresponding to the drawing blocks.

[0169] The image vectorization module 13 is used to input the embedding vector, topological vector and state parameters into the RNN decoder to generate the stroke parameters corresponding to each time step. The stroke parameters are converted into local 2D raster images through differential rendering, and the local 2D raster images are pasted back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed and the vectorized raster image is obtained.

[0170] In one example, the data processing module 12 is used to calculate the directional similarity between a first skeleton segment and a second skeleton segment based on a skeleton connection function, wherein the first skeleton segment and the second skeleton segment are adjacent skeleton segments; if the directional similarity between the first skeleton segment and the second skeleton segment is greater than a similarity threshold, then the first skeleton segment and the second skeleton segment are determined to be continuous skeleton segments; based on a projection consistency function, the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is calculated; if the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is less than a consistency threshold, then the first skeleton segment is determined to be an unreliable skeleton segment, and the unreliable skeleton segment is removed.

[0171] In one example, the data processing module 12 is used to divide the dynamic window to be cropped in the black and white structured line raster image into multiple spatial grid blocks of the same size; set multiple sampling points in each spatial grid block, calculate the value of each sampling point in the spatial grid block by bilinear interpolation, average the value of each sampling point in the current spatial grid block to obtain the pixel value corresponding to the spatial grid block; input the spatial grid block into the CNN encoder for feature extraction to obtain the embedding vector corresponding to the spatial grid block.

[0172] In one example, the image vectorization module 13 is used to enable the RNN decoder to output initial stroke parameters and hidden state update function based on the image feature information of the current drawing block in the embedding vector and the image structure topology information corresponding to the current drawing block in the topology vector; and to generate stroke parameters corresponding to each time step based on the width factor of the previous time step, the cursor position of the previous time step, the initial stroke parameters, and the hidden state update function.

[0173] In one example, the image vectorization module 13 is used to sequentially input stroke parameters into the multi-layer fully connected layer, multi-layer convolution and activation function in the differentiable rendering module, so that the stroke parameters are converted into a two-dimensional rasterized image. The multi-layer fully connected layer is used to convert the features corresponding to the stroke parameters into three-dimensional tensors to form a feature map; the multi-layer convolution is used to rearrange the pixels of the feature map to obtain a raster image with a target resolution; and the activation function is used to map the raster image with the target resolution to obtain a local 2D raster image.

[0174] In one example, the image vectorization module 13 is used to sum all loss functions with preset weights based on the vectorized raster image and the target image to obtain the total loss function, and to perform backpropagation on the CNN encoder and RNN decoder based on the total loss function. The loss functions include the perceptual loss function, the out-of-bounds penalty loss function, the stroke regularization loss function, the smoothness loss function, and the topology consistency loss function.

[0175] In one example, the stroke parameters include the control points of the quadratic Bézier curve, the stroke width factor, the softened lifting state, and the softened ending state.

[0176] Please see Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 2 may include: at least one processor 21, at least one network interface 24, user interface 23, memory 25, and at least one communication bus 22.

[0177] The communication bus 22 is used to enable communication between these components.

[0178] The user interface 23 may include a display screen and a camera. Optionally, the user interface 23 may also include a standard wired interface and a wireless interface.

[0179] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0180] The processor 21 may include one or more processing cores. The processor 21 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 25, and by calling data stored in the memory 25. Optionally, the processor 21 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 21 and may be implemented as a separate chip.

[0181] The memory 25 may include random access memory (RAM) or read-only memory. Optionally, the memory 25 may include non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 25 may also be at least one storage device located remotely from the aforementioned processor 21. Figure 5 As shown, the memory 25, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for vectorizing the black and white structural lines of Thangka paintings based on deep learning.

[0182] exist Figure 5 In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and obtain the user input data; while the processor 21 can be used to call an application stored in the memory 25 that is a deep learning-based vectorization processing method for Thangka black and white structural lines. When executed by one or more processors, the electronic device executes one or more methods as described in the above embodiments.

[0183] A computer-readable storage medium storing instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described in the embodiments above.

[0184] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0185] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0190] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

[0191] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for vectorizing the black and white structural lines of Thangka paintings based on deep learning, characterized in that, The method includes: Acquire a black-and-white structured line raster image to be processed; Extract the skeleton and boundary of the corresponding lines in the black and white structured line raster image, and segment the skeleton and boundary to obtain multiple skeleton segments and multiple boundary segments; Reliability screening is performed on all adjacent skeleton segments, and topology reconstruction is performed based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by encoding the topology structure. The reliability screening of all adjacent skeleton segments specifically includes: Calculate the directional similarity between the first skeleton segment and the second skeleton segment, wherein the first skeleton segment and the second skeleton segment are adjacent skeleton segments; If the directional similarity between the first skeleton segment and the second skeleton segment is greater than the similarity threshold, then the first skeleton segment and the second skeleton segment are determined to be continuous skeleton segments. Calculate the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment; If the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is less than the consistency threshold, then the first skeleton segment is determined to be an unreliable skeleton segment and the unreliable skeleton segment is removed. The black and white structure line raster image processed by the dynamic window mechanism is input into the CNN encoder, and the black and white structure line raster image is aligned, cropped and feature extracted to output drawing blocks of the same size and embedding vectors corresponding to the drawing blocks. The embedding vector, the topology vector, and the state parameters are input into the RNN decoder to generate stroke parameters corresponding to each time step. The stroke parameters are converted into local 2D raster images through differential rendering, and the local 2D raster images are pasted back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed, and the vectorized raster image is obtained. The step of inputting the embedding vector, the topology vector, and the state parameters into the RNN decoder to generate stroke parameters for each time step specifically includes: Based on the image feature information of the current drawing block in the embedding vector and the image structure topology information corresponding to the current drawing block in the topology vector, the RNN decoder outputs the initial stroke parameters and the hidden state update function. Based on the width factor of the previous time step, the cursor position of the previous time step, the initial stroke parameters, and the hidden state update function, the stroke parameters corresponding to each time step are generated. The stroke parameters include the quadratic Bézier curve control points, the stroke width factor, the softened pen lifting state, and the softened pen ending state.

2. The method for vectorizing the black and white structural lines of Thangka paintings based on deep learning as described in claim 1, characterized in that, The alignment, cropping, and feature extraction of the black-and-white structured line raster image specifically includes: The dynamic window to be cropped in the black and white structured line raster image is divided into multiple spatial grid blocks of the same size; Multiple sampling points are set within each spatial grid block. The value of each sampling point within the spatial grid block is calculated using bilinear interpolation. The average value of each sampling point within the current spatial grid block is then used to obtain the pixel value corresponding to the spatial grid block. The spatial grid block is input into the CNN encoder for feature extraction to obtain the embedding vector corresponding to the spatial grid block.

3. The method for vectorizing the black and white structural lines of Thangka paintings based on deep learning as described in claim 1, characterized in that, The step of converting the stroke parameters into a local 2D raster image through differential rendering specifically includes: The stroke parameters are sequentially input into a multi-layer fully connected layer, a multi-layer convolutional layer, and an activation function in the differentiable rendering module to convert the stroke parameters into a two-dimensional rasterized image. The multi-layer fully connected layer is used to convert the features corresponding to the stroke parameters into three-dimensional tensors to form a feature map; the multi-layer convolutional layer is used to rearrange the pixels of the feature map to obtain a raster image with a target resolution; and the activation function is used to map the raster image with the target resolution to obtain a local 2D raster image.

4. The method for vectorizing the black and white structural lines of Thangka paintings based on deep learning as described in claim 1, characterized in that, After the vectorization of the black and white structure line raster image is completed and the vectorized raster image is obtained, the process further includes: Based on the vectorized raster image and the target image, all loss functions are summed with preset weights to obtain the total loss function. The CNN encoder and the RNN decoder are then backpropagated based on the total loss function. The loss function includes a perceptual loss function, an out-of-bounds penalty loss function, a stroke regularization loss function, a smoothness loss function, and a topological consistency loss function.

5. A deep learning-based vectorization system for the black and white structural lines of Thangka paintings, characterized in that, The Thangka black and white structural line vectorization processing system (1) includes an acquisition and segmentation module (11), a data processing module (12), and an image vectorization module (13), wherein, The acquisition and segmentation module (11) is used to acquire the black and white structure line raster image to be processed, extract the skeleton and boundary of the corresponding line of the black and white structure line raster image through the thinning algorithm, and segment the skeleton and the boundary to obtain multiple skeleton segments and multiple boundary segments; The data processing module (12) is used to perform reliability screening on all adjacent skeleton segments, and to perform topology reconstruction based on the screened skeleton segments and boundary segment information to obtain the topology structure corresponding to the skeleton segment and the topology vector formed by the topology structure encoding. The black and white structure line raster image processed by the dynamic window mechanism is input into the CNN encoder, and the black and white structure line raster image is aligned, cropped and feature extracted to output drawing blocks of the same size and the embedding vector corresponding to the drawing blocks. The reliability screening of all adjacent skeleton segments specifically includes: Calculate the directional similarity between the first skeleton segment and the second skeleton segment, wherein the first skeleton segment and the second skeleton segment are adjacent skeleton segments; If the directional similarity between the first skeleton segment and the second skeleton segment is greater than the similarity threshold, then the first skeleton segment and the second skeleton segment are determined to be continuous skeleton segments. Calculate the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment; If the consistency projection value between the first skeleton segment and the boundary segment corresponding to the first skeleton segment is less than the consistency threshold, then the first skeleton segment is determined to be an unreliable skeleton segment and the unreliable skeleton segment is removed. The image vectorization module (13) is used to input the embedding vector, the topology vector and the state parameters into the RNN decoder to generate stroke parameters corresponding to each time step, convert the stroke parameters into a local 2D raster image through differential rendering, and paste the local 2D raster image back to the global canvas in a differentiable tiled manner until the vectorization of the black and white structure line raster image is completed and the vectorized raster image is obtained. The step of inputting the embedding vector, the topology vector, and the state parameters into the RNN decoder to generate stroke parameters for each time step specifically includes: Based on the image feature information of the current drawing block in the embedding vector and the image structure topology information corresponding to the current drawing block in the topology vector, the RNN decoder outputs the initial stroke parameters and the hidden state update function. Based on the width factor of the previous time step, the cursor position of the previous time step, the initial stroke parameters, and the hidden state update function, the stroke parameters corresponding to each time step are generated. The stroke parameters include the quadratic Bézier curve control points, the stroke width factor, the softened pen lifting state, and the softened pen ending state.

6. An electronic device, characterized in that, The device includes a processor (21), a memory (25), a user interface (23), and a network interface (24). The memory (25) is used to store instructions. The user interface (23) and the network interface (24) are used to communicate with other devices. The processor (21) is used to execute the instructions stored in the memory (25) to cause the electronic device (2) to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images