Image line drawing generation system based on conversion from bit image to non-bit image

Abstract

The invention relates to the technical field of image line drawing generation, in particular to an image line drawing generation system based on conversion from a bit image to a non-bit image. The system comprises a bit image acquisition module, a feature extraction module, a connecting mark marking module and a line drawing generation module. According to the invention, depth image information and color image information of an image are acquired through a bit image acquisition module, a feature extraction module establishes a convolutional neural network model for feature extraction, structural features and edge features of the image are extracted, a joint mark marking module analyzes feature data, and a joint mark marking result is obtained. And the line drawing generation module carries out quantitative processing on extracted structural features and edge features and dynamically adjusts parameters of a generator under different line drawing intensities according to loss information fed back by a discriminator, so that the printing precision and the model quality are improved, and the production efficiency is improved. And it is ensured that the 3D printed image is distinct in gradation, obvious in boundary and more authentic.

Description

Technical Field

[0001] The present invention relates to the technical field of image line drawing generation, and in particular to an image line drawing generation system based on bitmap image conversion into non-bitmap image. Background Art

[0002] In the 3D printing process, high-quality model data is the basis for ensuring the printing effect. Although bitmap images can present image details delicately (such as Chinese patent application number: CN202210079343.6), due to pixel storage, aliasing and blurring are prone to occur during the scaling and editing of 3D printed models, which seriously affects the printing accuracy and model quality. Converting bitmap images into non-bitmap image line drawings and describing the object contour and structure with vector lines can enable the model to be infinitely scaled without distortion in 3D printing. In order to extract features from images and identify the density of line drawings in images, the generator parameters are dynamically adjusted according to the feedback loss information, and high-precision line drawings are performed on dense areas of the image. At the same time, special marks are made for the joints of multiple textures to ensure the authenticity of 3D printing. Therefore, we propose an image line drawing generation system based on bitmap image conversion to non-bitmap image. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of aliasing and blurring that are prone to occur during model scaling and editing in 3D printing, which seriously affect the printing accuracy and model quality. In order to be able to extract features from the image and identify the density of line drawings in the image, the generator parameters are dynamically adjusted according to the feedback loss information, and high-precision line drawings are performed on the dense areas of the image. At the same time, special markings are performed on the joints of multiple textures to ensure the authenticity of 3D printing.

[0004] To achieve the above-mentioned purpose, the present invention provides an image line drawing generation system based on bitmap image conversion into non-bitmap image, comprising a bitmap image acquisition module, a feature extraction module, a joint mark marking module and a line drawing generation module;

[0005] The image acquisition module acquires the depth image information and color image information of the image to generate fused data, and inputs the fused data into the feature extraction module to establish a convolutional neural network model for extracting features, and outputs the structural features and edge features of the image;

[0006] The joint mark module uses geometric marks and color marks to specially mark the joints of different textures, and feeds the joint mark information back to the feature extraction module, and uses the spatial filtering method to perform secondary fusion of the joint mark information, image structure features and edge features to generate line drawing data;

[0007] The line drawing generation module quantizes the line drawing data, classifies the line drawing density of the image according to the quantized features, uses a generative adversarial network, determines the difference between the generated image and the real image through a discriminator, and feeds back loss information, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information.

[0008] Compared with the prior art, the present invention has the following beneficial effects:

[0009] 1. In the image line drawing generation system based on bitmap image conversion to non-bitmap image, the bitmap image acquisition module collects the depth image information and color image information of the image, and the depth image and the color image are registered and fused to generate fusion data. The feature extraction module takes the fusion data of the depth image and the color image as input to establish a convolutional neural network model for extracting features, and extracts the structural features and edge features of the image. The line drawing generation module quantizes the extracted structural features and edge features, and classifies the line drawing density of the image according to the quantized features. According to the loss information fed back by the discriminator, the parameters of the generator under different line drawing densities are dynamically adjusted, so as to realize the dynamic adjustment of the accuracy parameters of the generator according to different line drawing densities of the image, thereby improving the printing accuracy and model quality;

[0010] 2. The joint marking module analyzes the feature data, identifies the joint information between different textures, uses geometric markers and color markers to specially mark the joints of different textures, and feeds the joint marking information back to the feature extraction module. The spatial filtering method is used to fuse the joint marking information, image structure features and edge features, highlighting the joint information in the feature data, ensuring that the 3D printed image has clear layers, obvious boundaries, and is more realistic.

[0011] On the basis of the above technical solution, the present invention can also be improved as follows:

[0012] As a further improvement of the technical solution, the image acquisition module uses a feature matching algorithm to register and fuse the depth image and the color image to generate fused data, and the steps are as follows:

[0013] S1. Image registration: Use the calibration plate to obtain the feature points of the calibration plate in the depth image and color image, and calculate the transformation matrix between the two, so as to align the depth image and the color image to the same coordinate system;

[0014] S2, feature point detection: using feature point detection algorithm, detect feature points in depth image and color image respectively;

[0015] S3, feature matching: calculate the distance between the descriptor of each feature point in the depth image and the descriptors of all feature points in the color image, and select the feature point with the smallest distance as the matching point;

[0016] S4, image fusion: According to the matched feature points, the weighted average method is used to assign different weights to the depth information and color information, and then the pixel values ​​at the corresponding positions are weighted combined.

[0017] The beneficial effect of adopting the above further scheme is that when collecting image information, the color image contains rich visual information such as texture and color, the depth image can more accurately determine the position and boundaries of the object, and the feature matching algorithm can find the corresponding feature points in the depth image and the color image. The two images are accurately aligned through these feature points, avoiding information misalignment or deviation, thereby achieving more accurate information fusion. The fused image can provide richer features for the target recognition algorithm, thereby improving the recognition accuracy and positioning accuracy.

[0018] As a further improvement of the technical solution, the feature extraction module includes a model building unit and a feature extraction unit;

[0019] The model building unit performs data enhancement operation on the fused data, divides the enhanced fused data into a training set, a validation set and a test set, defines the input layer dimension according to the dimension of the fused data, uses multiple convolutional layers to stack and build a basic convolutional block, extracts local features of different scales, inserts a pooling layer between the convolutional layers, uses a fully connected layer to map the extracted features to an output space, and outputs the structural features and edge features of the image respectively by the output layer, and establishes a convolutional neural network model for extracting features;

[0020] The feature extraction unit compares the structural features and edge features of the convolutional neural network output image with the input fusion data, uses a mean square error loss function to measure the difference between the structural features predicted by the convolutional neural network model and the actual structural features, and optimizes the parameters of the convolutional neural network model with the goal of minimizing the difference value, thereby accurately extracting the structural features and edge features of the image.

[0021] The beneficial effect of adopting the above further scheme is that when establishing a convolutional neural network model for extracting features, each convolutional layer can learn local features of different levels and types. The shallow convolutional layer can capture relatively simple features in the image, such as low-level features such as edges and corners. As the convolutional layers are stacked, the network can gradually learn more complex and abstract features, such as high-level features such as parts and overall shapes of objects. In this way, the network can extract rich and diverse local features from the image, which helps to more comprehensively describe the content of the image;

[0022] When extracting the structural and edge features of an image, the mean square error loss function intuitively measures the average square difference between the predicted structural features and the true structural features, amplifying larger differences and causing the convolutional neural network model to pay more attention to samples with poor prediction results, thereby prompting the model to work hard to reduce these larger errors and improve the overall prediction accuracy.

[0023] As a further improvement of the present technical solution, the model building unit concatenates the feature map of the shallow convolution layer with the feature map of the deep convolution layer, and introduces jump connections so that the model can simultaneously utilize feature information at different levels and retain the details and edge information of the image.

[0024] The beneficial effect of adopting the above further scheme is that when the convolutional neural network performs convolution operation, the feature map extracted by the shallow convolution layer usually contains low-level, detailed information of the image, while the feature map of the deep convolution layer captures more high-level, semantic information of the image. By splicing the two, the model can fuse multi-scale feature information, which can not only use shallow detailed features for precise positioning and description, but also use deep semantic features for more abstract understanding and classification, thereby improving the comprehensive understanding and expression ability of image content.

[0025] As a further improvement of the present technical solution, when the feature extraction unit extracts edge features, a single-channel convolution layer is used to output a single-channel feature map with the same size as the input image. When extracting key structural features, multiple channel outputs are designed, and each channel corresponds to a different type of structural feature.

[0026] The beneficial effect of adopting the above further scheme is that, for edge features, which are essentially a relatively single and clear feature, the single-channel convolution layer focuses on capturing this single type of feature, and can concentrate all computing resources on detecting the existence and position of edges. For structural features, which are diverse, such as the shape and texture of objects, different types of structural features require different convolution kernels for extraction. By designing multiple channel outputs, each channel can use a different convolution kernel, thereby extracting different types of structural features in parallel.

[0027] As a further improvement of the present technical solution, the joint mark marking module determines the marking intensity according to the color saturation and brightness, mark size and mark density of the joint mark to reflect the importance and obviousness of the joint mark.

[0028] The beneficial effect of adopting the above further solution is that when the joint mark is specially marked, the characteristics of the joint mark can be quantified by measuring and analyzing the specific attributes such as color saturation, brightness, size and density. Instead of vaguely describing the obviousness of the joint mark, it is represented by a specific value or level, making the assessment of the joint mark more accurate and objective.

[0029] As a further improvement of the technical solution, the line drawing generation module includes a line drawing distinction unit and a parameter adjustment unit;

[0030] The line drawing distinguishing unit quantizes the line drawing data using a threshold-based quantization method, and distinguishes the density of the line drawings according to the quantized features;

[0031] The parameter adjustment unit inputs the feature-fused line drawing data and the random noise vector into the generator, and the discriminator judges the image generated by the generator, constructs a generative adversarial network, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information fed back by the discriminator to generate accurate line drawing images.

[0032] The beneficial effect of adopting the above further scheme is that when quantizing the line drawing data, the characteristic values ​​are classified according to the threshold value, which can highlight the key information in the data. When distinguishing the density of the line drawing, the line drawing part and the background part can be clearly distinguished through threshold quantization, only focusing on the characteristics of the line drawing area, ignoring irrelevant information such as the background, making the analysis of the density of the line drawing more focused and effective;

[0033] When dynamically adjusting the parameters of the generator under different line drawing densities, the generator can capture the image features under different line drawing densities by learning the line drawing data after feature fusion. The addition of random noise vectors increases the diversity of generated images. The discriminator continuously judges the images generated by the generator and feeds back loss information, prompting the generator to adjust parameters according to these feedbacks, thereby more accurately learning the distribution of real data and generating more realistic and higher-quality images.

[0034] As a further improvement of the technical solution, the loss information of the discriminator in the parameter adjustment unit consists of two parts, the image true and false discrimination loss and the line drawing density discrimination loss, wherein the formula of the image true and false discrimination loss is:

[0035] ;

[0036] in, is the image authenticity discrimination loss, is the batch size, For labels, is a real image, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The generated image;

[0037] Among them, the formula for line drawing density discrimination loss is:

[0038] ;

[0039] in, is the line drawing density discrimination loss, is the batch size, For real images, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The resulting image.

[0040] The beneficial effect of adopting the above further scheme is that, through the above formula, the image true and false discrimination loss and the line drawing density discrimination loss can be accurately calculated. The image true and false discrimination loss and the line drawing density discrimination loss each guide the model to learn and optimize in different aspects, avoiding the imbalance caused by the model being too biased towards one aspect during the training process.

[0041] As a further improvement of the present technical solution, the parameter adjustment unit introduces an attention mechanism, calculates the attention weight by scaling the dot product attention, pays attention to important areas in the generator image, and highlights important features.

[0042] The beneficial effect of adopting the above further scheme is that when the generator generates an image, the attention mechanism can automatically identify important areas in the generator image, such as the outline and details of the object, and give these areas higher attention. By strengthening the key features, the generated image is visually clearer and more accurate, and can better present the shape and structure of the object, thereby improving the overall quality and realism of the image.

[0043] As a further improvement of the present technical solution, the parameter adjustment unit trains the discriminator and the generator using forward propagation and updates the parameters through back propagation. The generator continuously adjusts the generator parameters according to the loss information fed back by the discriminator.

[0044] The beneficial effect of adopting the above further scheme is that when the discriminator and the generator are trained, the forward propagation and back propagation processes enable the generator and the discriminator to interact and constrain each other. The generator attempts to minimize the loss of the discriminator feedback, while the discriminator attempts to maximize this loss. This adversarial training method can find a balance between the two. By continuously adjusting the generator parameters, it can avoid either the generator or the discriminator being too strong or too weak, thereby ensuring the stable training of the entire generative adversarial network.

[0045] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0047] Figure 2 It is a schematic diagram of the overall details of the present invention;

[0048] Figure 3 It is a schematic diagram of the feature fusion process of the present invention;

[0049] Figure 4 It is a schematic diagram of the loss feedback process of the present invention.

[0050] The meaning of each number in the figure is:

[0051] 100, image acquisition module; 200, feature extraction module; 210, model building unit; 220, feature extraction unit; 300, joint mark marking module; 400, line drawing generation module; 410, line drawing distinction unit; 420, parameter adjustment unit. DETAILED DESCRIPTION

[0052] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] At present, 3D printing is prone to problems such as aliasing and blurring during model scaling and editing, which seriously affect the printing accuracy and model quality. In order to be able to extract features from the image and identify the density of line drawings in the image, the generator parameters are dynamically adjusted according to the feedback loss information, and high-precision line drawings are performed on the dense areas of the image. At the same time, special markings are made at the joints of multiple textures to ensure the authenticity of 3D printing.

[0054] Therefore, the present invention proposes that the depth image information and color image information of the image are collected by the image acquisition module, the feature extraction module takes the fusion data of the depth image and the color image as input, establishes a convolutional neural network model for extracting features, extracts the structural features and edge features of the image, the joint marking module analyzes the feature data, uses the edge detection algorithm to detect the edge information in the data, and uses geometric marks and color marks to specially mark the joints of different textures, the line drawing generation module quantizes the extracted structural features and edge features, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information fed back by the discriminator.

[0055] The details are as follows:

[0056] See also Figure 1 As shown, the present invention provides an image line drawing generation system based on bitmap image conversion into non-bitmap image, including a bitmap image acquisition module 100, a feature extraction module 200, a joint mark marking module 300 and a line drawing generation module 400;

[0057] The image acquisition module 100 acquires the depth image information and color image information of the image to generate fused data, and inputs the fused data into the feature extraction module 200 to establish a convolutional neural network model for extracting features, and outputs the structural features and edge features of the image;

[0058] An RGB-D camera is used to collect the color information and depth information of an image at the same viewing angle, obtain a depth image and a color image, and ensure that the depth image and the color image have the same size.

[0059] For example, the collected color information and depth information are as follows:

[0060] Pixel location Red channel (R) Green channel (G) Blue channel (B) Depth value (cm) （1,1） 150 80 200 30 （1,2） 200 120 60 35 （2,1） 100 220 50 25 （2,2） 180 90 130 32

[0061] For color images, the scale-invariant feature transform (SIFT) algorithm can be used to extract features. The algorithm considers information such as image brightness and contrast, and generates feature descriptors by detecting key points in the image (such as corner points, edge points, etc.) and calculating the gradient direction and amplitude of the surrounding area. For example, for the pixel point (1,1), its color value is (150,80,200). The SIFT algorithm considers the color changes of the point and its surrounding pixels and calculates a feature descriptor containing multiple dimensional information, assuming it is [a1,a2,...,an];

[0062] For depth images, since their main information is the depth distance of objects, a feature extraction method based on depth change can be used. For example, the depth difference between each pixel and its adjacent pixel is calculated to obtain the depth change rate. For pixel (1,1), its depth value is 30 cm, and the depth value of the adjacent pixel (1,2) is 35 cm. The depth change rate is (35−30)÷1=5. Based on the depth change rate, some local statistical information (such as mean and variance) is combined to construct a feature descriptor for the depth image.

[0063] The image acquisition module 100 uses a feature matching algorithm to register and fuse the depth image and the color image to generate fused data. The steps are as follows:

[0064] S1. Image registration: Use the calibration plate to obtain the feature points of the calibration plate in the depth image and color image, and calculate the transformation matrix between the two, so as to align the depth image and the color image to the same coordinate system;

[0065] S2, feature point detection: using feature point detection algorithm, detect feature points in depth image and color image respectively;

[0066] S3, feature matching: calculate the distance between the descriptor of each feature point in the depth image and the descriptors of all feature points in the color image, and select the feature point with the smallest distance as the matching point;

[0067] S4, image fusion: according to the matched feature points, the weighted average method is used to assign different weights to the depth information and color information, and then the pixel values ​​at the corresponding positions are weighted combined;

[0068] Depth images provide depth information of objects in the scene, allowing us to understand the distance relationship between objects and the camera, while color images contain rich visual information such as texture and color. Through registration and fusion, the advantages of the two are combined so that the fused data contains both the appearance characteristics of the object and its spatial position information, greatly enriching the amount of information contained in the image and providing a more comprehensive data basis for subsequent analysis and processing.

[0069] like Figure 2 As shown, the feature extraction module 200 includes a model building unit 210 and a feature extraction unit 220;

[0070] The model building unit 210 performs data enhancement operation on the fused data, divides the enhanced fused data into a training set, a validation set and a test set, defines the input layer dimension according to the dimension of the fused data, uses multiple convolutional layers to stack and build a basic convolution block, extracts local features of different scales, inserts a pooling layer between the convolutional layers, uses a fully connected layer to map the extracted features to an output space, and outputs the structural features and edge features of the image respectively by the output layer, and builds a convolutional neural network model for extracting features;

[0071] The feature extraction unit 220 compares the structural features and edge features of the image output by the convolutional neural network with the input fusion data, uses a mean square error loss function to measure the difference between the structural features predicted by the convolutional neural network model and the actual structural features, and optimizes the parameters of the convolutional neural network model with the goal of minimizing the difference value, thereby accurately extracting the structural features and edge features of the image;

[0072] The steps to build a convolutional neural network model for feature extraction are as follows:

[0073] 1. Constructing basic convolution blocks: The basic convolution blocks are composed of multiple stacked convolution layers. Each convolution layer can learn different local features in the image. By stacking multiple convolution layers, more complex and abstract features can be extracted. The main function of the convolution layer is to slide the convolution kernel on the image to perform convolution operations, thereby extracting the features of the image;

[0074] 2. Introducing nonlinearity: After each convolutional layer, the ReLU (Rectified Linear Unit) activation function is used. The expression of the ReLU function is f(x)=max(0,x). It can introduce nonlinear factors, allowing the neural network to learn more complex functional relationships and enhance the expressiveness of the model;

[0075] 3. Insert pooling layer: insert the maximum pooling layer between the convolutional layers. The function of the pooling layer is to downsample the feature map, reduce the size of the feature map, reduce the amount of calculation, and increase the robustness of the feature;

[0076] 4. Build a convolutional neural network to extract features: Combine multiple basic convolutional blocks and pooling layers to build a complete convolutional neural network model. At the end of the model, you can add a fully connected layer to map the extracted features to the target category.

[0077] The preprocessed fusion dataset is divided into training set, validation set and test set, with a ratio of 7:2:1. The training set is used for model parameter learning, the validation set is used to evaluate the performance of the model and adjust the hyperparameters during the training process, and the test set is used to finally evaluate the generalization ability of the model.

[0078] In order to ensure that the model can utilize both detail information and high-level semantic information, the model building unit 210 concatenates the feature map of the shallow convolution layer with the feature map of the deep convolution layer, and introduces a skip connection so that the model can utilize feature information at different levels at the same time and retain the details and edge information of the image;

[0079] The feature map of the shallow convolution layer retains the detailed information of the image, such as edges and textures, while the feature map of the deep convolution layer contains the high-level semantic information of the image, such as the category and overall structure of the object. The introduction of skip connections can fuse these two different levels of feature information, so that the model can use both detailed information and high-level semantic information, thereby improving the performance of the model;

[0080] Among them, skip connection refers to the fact that in a neural network with skip connection, in addition to the normal forward propagation path, there are some additional connections. These connections allow information to bypass some intermediate layers and pass directly from the shallower layers of the network to the deeper layers. In this way, when the network performs back propagation to calculate the gradient, the gradient can be more directly propagated to the shallower layers through these skip connections, thereby solving the problem of gradient vanishing or gradient exploding that may occur in the training process of traditional neural networks, making the training of deep networks more stable and easier;

[0081] The shallow feature map and the deep feature map with matched sizes are spliced ​​in the channel dimension. The spliced ​​feature map can be input into the subsequent convolutional layer for further feature extraction and processing to obtain richer feature representation.

[0082] In order to better perform feature extraction, when the feature extraction unit 220 extracts edge features, a single-channel convolution layer is used to output a single-channel feature map with the same size as the input image. When extracting key structural features, multiple channel outputs are designed, and each channel corresponds to a different type of structural feature.

[0083] Each pixel value in a single-channel feature map directly corresponds to the edge response intensity of that position in the input image. This intuitive representation makes it easy for subsequent processing to determine whether the position is an edge and the strength of the edge based on the pixel value. For example, in an image segmentation task, an appropriate threshold is set based on the pixel value to distinguish edge pixels from non-edge pixels. This is very friendly to visualization operations. The single-channel feature map can be directly displayed in the form of a grayscale image, which makes it easy for researchers to intuitively observe the effect of edge extraction and quickly evaluate the performance of the algorithm.

[0084] Single-channel output allows the model to focus on extracting edge features, avoiding information interference caused by multiple channels. In edge extraction tasks, there is no need to consider the relationship and interaction between different channels, which can capture edge information in the image more purely and improve the accuracy of edge extraction;

[0085] Multi-channel output can separate different types of structural features, making it easier to analyze and process each feature separately. At the same time, in subsequent network layers, the features of these different channels can be fused to further enhance the expressiveness of the features. For example, in a convolutional neural network, the features of different channels are weighted and combined through convolution operations to obtain a more advanced feature representation. This feature separation and fusion method makes the model more flexible and adaptable, and can adjust the combination of features according to different task requirements to improve the performance of the model.

[0086] Multi-channel feature representation can capture more details and changes in the image, making the model more adaptable and generalizable to different image data. For example, when processing images taken under different lighting conditions and at different angles, multi-channel structural features can help the model better identify the target object and reduce the impact of environmental factors on model performance.

[0087] Furthermore, the joint mark module 300 uses geometric marks and color marks to specially mark the joints of different textures, and feeds the joint mark information back to the feature extraction module 200, and uses the spatial filtering method to perform secondary fusion of the joint mark information, image structure features and edge features to generate line drawing data;

[0088] like Figure 3 As shown, the depth image determines the boundary and outline of the object, distinguishes objects at different depth levels, and uses the color difference and texture features in the color image to assist in determining the edge and details of the object. The edge, corner point and depth change feature points of the object in the depth image are registered and fused with the feature points with drastic color changes and rich textures in the color image to generate point cloud data.

[0089] Taking the fused point cloud data of the depth image and the color image as input, a convolutional neural network model for feature extraction is established to extract the structural features and edge features of the image. The spatial filtering method is used to fuse the joint mark information, image structural features and edge features to highlight the joint information in the feature data.

[0090] Among them, spatial filtering is a method of processing images or feature data in the spatial domain. By designing appropriate filters, different features can be fused and specific information can be highlighted.

[0091] The spatial filtering method is as follows:

[0092] The median filter is used to filter the seam mark information, image structure features and edge features to remove noise points. Weights are assigned to the seam mark information, image structure features and edge features according to their importance. Since the seam information should be highlighted, a higher weight is assigned to the seam mark information. According to the position of the seam mark information, the image is divided into seam area and non-seam area. In the seam area, the proportion of the seam mark information in the fusion is increased, and in the non-seam area, the proportion of the seam mark information is appropriately reduced, and more consideration is given to the image structure features and edge features.

[0093] In order to better distinguish the importance and obviousness of the joint mark, the joint mark marking module 300 determines the marking intensity according to the color saturation and brightness, mark size and mark density of the joint mark to reflect the importance and obviousness of the joint mark;

[0094] According to the evaluation results of the importance and conspicuity of the joint, weights are set for the color saturation and brightness, mark size, and mark density. For example, if the importance of the joint is mainly reflected in the visual conspicuity, the weights of color saturation and brightness can be relatively high. If the importance of the joint is related to the coverage, the weight of the mark size can be larger.

[0095] According to the calculated marking intensity value, different marking intensity levels are set, such as weak marking, medium marking and strong marking. For example, the marking intensity value can be divided into several intervals, each of which corresponds to a marking intensity level:

[0096] Weak marking: The marking strength value is low, suitable for joints with low importance and visibility;

[0097] Medium marking: The marking intensity value is moderate, and it is used for joints with medium importance and obviousness;

[0098] Strong marking: The marking intensity value is higher, and it is used for joints with high importance and obviousness;

[0099] In practical applications, the calculation method and classification of marking intensity need to be dynamically adjusted according to the actual effect. The effect of marking can be evaluated through user feedback, actual observation, etc. to ensure that the marking intensity can accurately reflect the importance and obviousness of the joint. At the same time, multiple verifications and optimizations are carried out to improve the accuracy and reliability of the marking intensity determination method.

[0100] In addition, the line drawing generation module 400 quantizes the line drawing data, classifies the line drawing density of the image according to the quantized features, uses the generative adversarial network, determines the difference between the generated image and the real image through the discriminator, and feeds back the loss information, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information;

[0101] like Figure 4 As shown, the feature-fused line drawing data and the random noise vector are input into the generator, and the generated line drawing image and the corresponding line drawing density are used as the input of the discriminator to construct a generative adversarial network. According to the loss information fed back by the discriminator, the parameters of the generator under different line drawing densities are dynamically adjusted.

[0102] The line drawing generation module 400 includes a line drawing distinction unit 410 and a parameter adjustment unit 420;

[0103] The line drawing distinguishing unit 410 quantizes the line drawing data using a threshold-based quantization method, and distinguishes the density of the line drawing according to the quantized features;

[0104] The parameter adjustment unit 420 inputs the line drawing data after feature fusion and the random noise vector into the generator, and the discriminator judges the image generated by the generator, constructs a generative adversarial network, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information fed back by the discriminator to generate accurate line drawing images;

[0105] The threshold-based quantization method is as follows:

[0106] Set thresholds: Based on prior knowledge of energy and entropy statistics or actual application scenarios, determine a series of thresholds to divide the range of statistics into n intervals. For example, for energy statistics, you may know that when the energy value is less than T1, it represents a characteristic state, and between T1 and T2, it represents another state, and so on;

[0107] Interval mapping: For a given statistic S, determine which threshold interval it falls within and then map it to the corresponding quantized value.

[0108] In order to better calculate the loss information of the discriminator, the loss information of the discriminator in the parameter adjustment unit 420 is composed of two parts, the image true and false discrimination loss and the line drawing density discrimination loss, wherein the formula of the image true and false discrimination loss is:

[0109] ;

[0110] in, is the image authenticity discrimination loss, is the batch size, For labels, is a real image, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The generated image;

[0111] Among them, the formula for line drawing density discrimination loss is:

[0112] ;

[0113] in, is the line drawing density discrimination loss, is the batch size, is a real image, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The generated image.

[0114] The main task of the discriminator is to distinguish the real data from the fake data generated by the generator as accurately as possible. A batch of real samples are randomly selected from the real data set, and a batch of fake samples are generated by the generator based on the random noise vector. The real samples and fake samples are input into the discriminator respectively, and the output results of the discriminator for the real samples and fake samples are obtained. The output result is usually the probability that the sample is a real sample;

[0115] The above formulas are used to calculate the image authenticity discrimination loss and the line drawing density discrimination loss respectively. The total loss of the discriminator is obtained by summing them up. The gradient of the discriminator loss function to the discriminator parameters is calculated, and then the optimization algorithm is used to update the discriminator parameters to reduce the loss.

[0116] During the training process, the parameters of the discriminator and generator are usually updated alternately. First, the generator is fixed and the parameters of the discriminator are updated so that the discriminator can better distinguish between real and fake samples. Then the discriminator is fixed and the parameters of the generator are updated so that the generator can generate more realistic samples. This cycle is repeated until the model converges. Through adversarial training of the discriminator and generator, the generator will continuously learn how to generate samples that are closer to the real data distribution. The discriminator is like a "critic" that provides feedback to the generator, prompting the generator to improve its generation ability. The samples finally generated can be significantly improved in quality, detail, and diversity.

[0117] In order to better focus on important areas in the image, the parameter adjustment unit 420 introduces an attention mechanism, and calculates attention weights by scaling dot product attention, so as to focus on important areas in the generator image and highlight important features;

[0118] For each feature map, it is mapped to the query, key, and value spaces respectively through linear transformation. Assume that the input feature map is F, and its dimension is [H, W, C], where H and W are the height and width of the feature map, and C is the number of channels. Define three learnable weight matrices Wq, Wk, and Wv, and multiply F with these three matrices respectively to obtain the query matrix Q, key matrix K, and value matrix V;

[0119] Calculate the dot product of the query matrix Q and the key matrix K to get the attention score matrix S. In order to avoid the dot product result being too large, which will cause the gradient to disappear or become unstable, divide the attention score matrix S by the square root of the key vector dimension dk. Apply the softmax function to the scaled attention score matrix S to convert it into the attention weight matrix A. Use the attention weight matrix A to perform weighted summation on the value matrix V to obtain the feature map processed by the attention mechanism, focus on the important areas in the generator image, and highlight the important features.

[0120] In order to better adjust the generator parameters, the parameter adjustment unit 420 uses forward propagation to train the discriminator and the generator. Through back propagation and parameter update, the generator continuously adjusts the generation parameters according to the loss information fed back by the discriminator.

[0121] The generator receives a random noise vector as input. In image generation tasks, the random noise vector is usually sampled from a certain distribution (such as a Gaussian distribution). The generator transforms the input noise vector through its own network structure (such as a multi-layer perceptron, convolutional neural network, etc.) to generate false samples;

[0122] The fake samples generated by the generator are input into the discriminator. The task of the discriminator is to determine whether the input sample is a real sample or a fake sample. Its output is a probability value, indicating the possibility that the sample is a real sample;

[0123] After calculating the generator loss, the back-propagation algorithm is used to calculate the gradient of the loss function with respect to all trainable parameters of the generator. The back-propagation algorithm is based on the chain rule, starting from the loss function, and calculating the contribution of each parameter to the loss layer by layer.

[0124] Repeat the above process of forward propagation, loss calculation, back propagation and parameter update for multiple iterations of training. In each iteration, the discriminator evaluates the new samples generated by the generator and feeds back new loss information. The generator continuously adjusts its own parameters based on these feedbacks, making the generated samples closer and closer to the distribution of real samples, thereby improving the quality of generated samples.

[0125] In summary, the working principle of this solution is as follows:

[0126] In the image line drawing generation system based on bitmap image conversion to non-bitmap image, the bitmap image acquisition module 100 acquires the depth image information and color image information of the image, and the depth image and the color image are registered and fused to generate fused data. The feature extraction module 200 takes the fused data of the depth image and the color image as input to establish a convolutional neural network model for extracting features, and extracts the structural features and edge features of the image. The line drawing generation module 400 quantizes the extracted structural features and edge features, and classifies the density of the line drawings of the image according to the quantized features. According to the loss information fed back by the discriminator, the different line drawing density is dynamically adjusted. The parameters of the generator under the density of line drawing can dynamically adjust the accuracy parameters of the generator according to the different line drawing densities of the image, thereby improving the printing accuracy and model quality. The joint marking module 300 analyzes the feature data, identifies the joint information between different textures, and uses geometric marks and color marks to specially mark the joints of different textures, and feeds the joint marking information back to the feature extraction module 200. The spatial filtering method is used to fuse the joint marking information, image structure features and edge features, and highlight the joint information in the feature data, thereby ensuring that the 3D printed image has clear layers, obvious boundaries, and is more realistic.

[0127] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An image line drawing generation system based on the conversion of bitmap images into non-bitmap images, characterized in that: It comprises a bitmap image acquisition module (100), a feature extraction module (200), a joint mark marking module (300) and a line drawing generation module (400); The image acquisition module (100) acquires depth image information and color image information of an image to generate fused data, and inputs the fused data into a feature extraction module (200) to establish a convolutional neural network model for extracting features, and outputs structural features and edge features of the image; The joint mark module (300) uses geometric marks and color marks to specially mark the joints of different textures, and feeds the joint mark information back to the feature extraction module (200), and uses a spatial filtering method to perform secondary fusion on the joint mark information, image structure features and edge features to generate line drawing data; The line drawing generation module (400) quantizes the line drawing data, classifies the line drawing density of the image according to the quantized features, uses a generative adversarial network to determine the difference between the generated image and the real image through a discriminator, and feeds back loss information, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information.

2. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 1, characterized in that: The image acquisition module (100) uses a feature matching algorithm to register and fuse the depth image and the color image to generate fused data, and the steps are as follows: S1. Image registration: Use the calibration plate to obtain the feature points of the calibration plate in the depth image and color image, and calculate the transformation matrix between the two, so as to align the depth image and the color image to the same coordinate system; S2, feature point detection: using feature point detection algorithm, detect feature points in depth image and color image respectively; S3, feature matching: calculate the distance between the descriptor of each feature point in the depth image and the descriptors of all feature points in the color image, and select the feature point with the smallest distance as the matching point; S4, image fusion: According to the matched feature points, the weighted average method is used to assign different weights to the depth information and color information, and then the pixel values ​​at the corresponding positions are weighted combined.

3. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 1, characterized in that: The feature extraction module (200) comprises a model building unit (210) and a feature extraction unit (220); The model building unit (210) performs a data enhancement operation on the fused data, divides the enhanced fused data into a training set, a validation set and a test set, defines the input layer dimension according to the dimension of the fused data, uses a plurality of convolutional layers to stack and build a basic convolutional block, extracts local features of different scales, inserts a pooling layer between the convolutional layers, maps the extracted features to an output space using a fully connected layer, and outputs the structural features and edge features of the image respectively by the output layer, thereby building a convolutional neural network model for extracting features; The feature extraction unit (220) compares the structural features and edge features of the image output by the convolutional neural network with the input fusion data, uses a mean square error loss function to measure the difference between the structural features predicted by the convolutional neural network model and the actual structural features, and optimizes the parameters of the convolutional neural network model with the goal of minimizing the difference value, thereby accurately extracting the structural features and edge features of the image.

4. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 3, characterized in that: The model building unit (210) concatenates the feature map of the shallow convolution layer with the feature map of the deep convolution layer, and introduces a jump connection so that the model can simultaneously utilize feature information at different levels and retain the details and edge information of the image.

5. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 3, characterized in that: When the feature extraction unit (220) extracts edge features, a single-channel convolution layer is used to output a single-channel feature map with the same size as the input image. When extracting key structural features, multiple channel outputs are designed, and each channel corresponds to a different type of structural feature.

6. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 1, characterized in that: The joint mark marking module (300) determines the marking intensity according to the color saturation and brightness, marking size and marking density of the joint mark, reflecting the importance and obviousness of the joint mark.

7. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 1, characterized in that: The line drawing generation module (400) comprises a line drawing distinction unit (410) and a parameter adjustment unit (420); The line drawing distinguishing unit (410) quantizes the line drawing data using a threshold-based quantization method, and distinguishes the density of the line drawings according to the quantified features; The parameter adjustment unit (420) inputs the line drawing data after feature fusion and the random noise vector into the generator, and the discriminator judges the image generated by the generator, constructs a generative adversarial network, and dynamically adjusts the parameters of the generator under different line drawing densities according to the loss information fed back by the discriminator to generate accurate line drawing images.

8. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 7, characterized in that: The loss information of the discriminator in the parameter adjustment unit (420) consists of two parts, the image true and false discrimination loss and the line drawing density discrimination loss, wherein the formula of the image true and false discrimination loss is: ； in, is the image authenticity discrimination loss, is the batch size, For labels, For real images, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The generated image; Among them, the formula for line drawing density discrimination loss is: ； in, is the line drawing density discrimination loss, is the batch size, For real images, is a random noise vector, is the line drawing density label, For the discriminator to the real image and tags The output, The generator is based on the noise vector and line drawing density label The generated image.

9. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 7, characterized in that: The parameter adjustment unit (420) introduces an attention mechanism, calculates attention weights by scaling dot product attention, pays attention to important areas in the generator image, and highlights important features.

10. The image line drawing generation system based on bitmap image conversion into non-bitmap image according to claim 7, characterized in that: The parameter adjustment unit (420) trains the discriminator and the generator using forward propagation and updates the parameters through back propagation. The generator continuously adjusts the generator parameters according to the loss information fed back by the discriminator.

Images