Method, system, device and medium for simultaneously identifying and segmenting hand-drawn sketches

Abstract

A method, system, device and medium for simultaneous recognition and segmentation of hand-drawn sketches, the method comprising: obtaining a hand-drawn sketch in a scalable vector graphics format from hand-drawn sketch acquisition software, and converting the result into an RGB format image and graph format data; constructing a dual-stream network for simultaneous recognition and segmentation based on a convolutional neural network and a graph convolutional network; inputting the RGB format hand-drawn sketch image into the convolutional neural network stream to obtain global features and category recognition results that characterize the hand-drawn sketch category; inputting the graph format hand-drawn sketch data into the graph convolutional network stream to obtain stroke-level features and point-level features, and splicing the global features, stroke-level features and point-level features to achieve point-level segmentation; utilizing the KL divergence method to achieve supervision of the segmentation results; the system, device and medium are used for simultaneous recognition and segmentation of hand-drawn sketches; the present invention can simultaneously perform hand-drawn sketch recognition and segmentation tasks, is more efficient than performing only segmentation or only recognition tasks, and improves the accuracy of hand-drawn sketch recognition and segmentation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method, system, device and medium for simultaneously recognizing and segmenting hand-drawn sketches. Background Art

[0002] With the development of electronic devices and pressure-sensing equipment, people are gradually shifting away from traditional pen-and-paper sketching methods. More and more people are choosing to sketch using touchscreen devices (iPads, sketchpads, etc.). Sketching is one of the most important activities in product design, widely used in design cognition and design thinking research. It plays an irreplaceable role in the identification, development, expression, deduction, and formation of design concepts.

[0003] With the development of the internet and the continuous advancement of technology, computing power has greatly increased. Convolutional neural networks, one of the most mature branches of artificial intelligence, have achieved remarkable success in areas such as image recognition and segmentation. Graph convolutional networks, proposed to solve graph problems, have proven effective in solving problems such as graph node clustering and segmentation. The biggest difference between hand-drawn sketches and traditional images is their ordered nature. When drawn, people create them one stroke at a time. After the sketch is completed, its image and graph characteristics can be leveraged simultaneously to achieve better recognition and segmentation results. However, the current mainstream approach directly uses convolutional neural networks for recognition and recurrent neural networks or graph neural networks for segmentation, failing to leverage both the image and graph characteristics of hand-drawn sketches.

[0004] Patent application CN112580614A discloses a hand-drawn sketch recognition method based on an attention mechanism. The method includes inputting the original hand-drawn sketch into a deep convolutional neural network to obtain a feature map output by the last convolutional layer; inputting the feature map into a channel attention module to obtain a feature map optimized based on the channel attention; training a classification network for predicting the vertical flip of the hand-drawn sketch, inputting the original hand-drawn sketch into the trained classification network to obtain a vertical flip spatial attention map; combining the feature map optimized based on the channel attention and the vertical flip spatial attention map to calculate a feature map optimized based on the vertical flip spatial attention; and finally outputting the recognition result through a fully connected layer. However, this invention can only realize the recognition of hand-drawn sketches. Summary of the Invention

[0005] In order to solve the above problems, the purpose of the present invention is to provide a method, system, device and medium for simultaneously identifying and segmenting hand-drawn sketches, making full use of the advantages of convolutional neural networks in extracting global features and graph convolutional networks in extracting point-level features, using the global features of the convolutional neural network to augment the features of the graph convolutional network, and using the category recognition results of the convolutional neural network to supervise the segmentation results of the graph convolutional network. Hand-drawn sketches can be recognized and segmented simultaneously, and the recognition results can be used to optimize the segmentation results, thereby improving the accuracy of hand-drawn sketch recognition and segmentation.

[0006] The technical solution adopted in the present invention is:

[0007] A method for simultaneously recognizing and segmenting hand-drawn sketches, comprising the following steps:

[0008] S1, obtaining a hand-drawn sketch in a scalable vector graphics format from a hand-drawn sketch acquisition software, and converting it into an RGB format image and graph format data;

[0009] S2, builds a two-stream network for simultaneous recognition and segmentation based on convolutional neural networks and graph convolutional networks;

[0010] S3, inputs the RGB format hand-drawn sketch image obtained by S1 into the convolutional neural network stream obtained by S2 to obtain global features and category recognition results that characterize the hand-drawn sketch category;

[0011] S4: Input the hand-drawn sketch data in graph format obtained in S1 into the graph convolutional network flow to obtain stroke-level features and point-level features, and then splice the global features, stroke-level features, and point-level features to achieve point-level segmentation;

[0012] S5, based on the prior information of the component categories that each hand-drawn sketch category should contain and the category recognition results of the convolutional neural network flow obtained according to S3, uses the KL divergence method to supervise the segmentation results of S4.

[0013] Furthermore, in S1, hand-drawn sketch data in scalable vector graphics format is obtained from hand-drawn drawing software, the scalable vector graphics is converted into an RGB image, and a point sequence is extracted from the scalable vector graphics to generate a graph as input to the simultaneous recognition and segmentation network.

[0014] Furthermore, in S2, the dual-stream network model based on convolutional neural network and graph convolutional network simultaneously accepts the same hand-drawn RGB image format data and graph structure data as input, the convolutional neural network flow uses ResNet18 as the backbone network, and the graph convolutional network flow uses four-layer dynamic graph convolution as the backbone network. The convolutional neural network flow provides the graph convolutional network flow with global features and component prediction probabilities derived from classification results as supervision information; the loss function of the dual-stream network model based on convolutional neural network and graph convolutional network during training is L=L c +λ1L s +λ2L kl , where L c is the cross entropy loss for hand-drawn category classification of convolutional neural network flow, L s is the cross entropy loss for the point-level component prediction of the graph convolutional network flow, and λ1 and λ2 are weight coefficients.

[0015] Furthermore, the specific method of S3 is as follows: the hand-drawn sketch image in RGB format is input to the convolutional neural network flow, the features of the last fully connected layer of the convolutional neural network are used as the global features representing the hand-drawn category, and the output of the classifier is used as the recognition result of the hand-drawn category; based on the prior information of the components that each hand-drawn category should contain, a conversion matrix from the hand-drawn category to the component category is constructed When the i-th hand-drawn image contains the j-th component, M(i,j) = 1, otherwise M(i,j) = 0; get N in S3 C Recognition probability of hand-drawn categories After that, the calculation can get the component N S dimensional predicted probability

[0016] Furthermore, the specific method of S4 is as follows: inputting the hand-drawn sketch data in graph format into the graph convolution network flow, first sampling the data in scalable vector graphics format, and ensuring that the number of nodes input to the graph convolution network is constant N P ; Then, for the point-level features learned by the graph convolutional network, according to the information of the points contained in each hand-drawn stroke contained in the scalable vector graphics format, the features of all points belonging to the same stroke are pooled to obtain the stroke-level features; the point-level features of each point, the stroke-level features to which it belongs, and the global features output by the convolutional neural network flow are spliced ​​to obtain the multi-scale features of each point, and then the component category is predicted by the classifier to obtain the component prediction probability of the i-th point.

[0017] Furthermore, the specific sampling strategy for the data in the scalable vector graphics format is as follows: if the number of points in the current hand-drawn sketch drawing sequence is greater than N P, reduce the number of points of each stroke in proportion to the percentage of the number of points of each stroke in the total number of points of the hand-drawn sketch; if there is a stroke whose proportion is too small before reducing the number of points, the number of points in the stroke will not be reduced.

[0018] Furthermore, in said S5, the The predicted probability at the component level is calculated as follows: Where V j represents the set of points contained in the j-th stroke, |V j | represents the number of points in the set, and J represents the number of strokes in the hand-drawn drawing. KL divergence is then used to supervise the component prediction probability based on the category recognition result. The supervision method is to add a loss function to the neural network. The loss function L is used to train and optimize the entire two-stream network model, so that the model can simultaneously recognize and segment hand-drawn sketches, and use the recognition results to optimize the segmentation results, thereby improving the accuracy of hand-drawn sketch recognition and segmentation.

[0019] A system for simultaneously recognizing and segmenting hand-drawn sketches, comprising:

[0020] The data conversion module is used to convert the hand-drawn data in scalable vector graphics format into RGB image data and hand-drawn data in graph format as input of the network model.

[0021] The dual-stream network module is used to simultaneously perform hand-drawn category recognition and component segmentation on the data processed by the data conversion module.

[0022] The training optimization module is used to construct a loss function and use a deep learning training method to optimize the training of the dual-stream network model of the present invention.

[0023] A device for simultaneously recognizing and segmenting hand-drawn sketches, comprising:

[0024] memory for storing computer programs;

[0025] A processor is configured to implement the method of simultaneously identifying and segmenting a hand-drawn sketch in steps S1 to S5 when executing the computer program.

[0026] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, a user can simultaneously recognize and segment a hand-drawn sketch.

[0027] The beneficial effects of the present invention are:

[0028] (1) This invention takes advantage of the advantages of convolutional neural networks in extracting global features and graph convolutional networks in extracting point-level features. The global features of the convolutional neural network are used to augment the features of the graph convolutional network, so that the features ultimately used by the graph convolutional network for segmentation include global features, stroke-level features, and point-level features, which is more conducive to improving segmentation accuracy. Experimental verification shows that the global features of the convolutional neural network can better characterize the global characteristics of hand-drawings than the global pooling results of the graph convolutional network's point-level features.

[0029] (2) The present invention uses the category recognition results of the convolutional neural network to supervise the segmentation results of the graph convolutional network, taking into full consideration the fact that hand-drawn recognition is easier to implement and has higher accuracy than hand-drawn segmentation. It uses the prior information of the hand-drawn components contained in each hand-drawn category and the recognition results to optimize the segmentation results, thereby improving the accuracy of hand-drawn sketch recognition and segmentation.

[0030] (3) The present invention simultaneously performs hand-drawn sketch recognition and segmentation tasks, and the two tasks promote each other. During the recognition and segmentation process, the global features used for recognition and the local features used for segmentation are fused. During the training process, better results are achieved compared to only segmentation or only recognition tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 Flowchart of an embodiment of the present invention.

[0033] Figure 2 Schematic diagram of the network structure for simultaneous identification and segmentation in the present invention.

[0034] Figure 3 Schematic diagram of the graph convolutional network structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0036] The embodiment of the present invention provides a method for simultaneously identifying and segmenting hand-drawn sketches. Figure 1As shown, the following processing methods are included:

[0037] S1, obtain the scalable vector graphics format data of the hand-drawn sketch from the hand-drawn sketch acquisition software. The scalable vector graphics format data of the hand-drawn sketch can be converted into RGB image and image format data. The scalable vector graphics format data can be regarded as a tree structure. The tree structure is divided into files, layers, and strokes according to the hierarchical relationship. The hand-drawn sketch point sequence can be obtained from the third-level stroke level data, S = {Pos i |i=1,2,…,N}, N represents the number of points in the hand-drawn sketch, Pos i Represents the i-th point in the entire hand-drawn sketch. Each point in the hand-drawn sketch sequence is represented as a triple, Pos i =<x,y,st>, where x represents the horizontal coordinate of the current point, y represents the vertical coordinate of the current point, and st represents the stroke number of the current point. By smoothly connecting adjacent points within a stroke, the hand-drawn sketch point sequence can be converted into a binary grayscale image, which can then be converted into an RGB three-channel image as input for the convolutional neural network. When converting the hand-drawn sketch point sequence into graph-format data, points are used as graph nodes, and edges exist between adjacent nodes within a stroke. This creates a node set and adjacency matrix.

[0038] S2, builds a two-stream network based on convolutional neural networks and graph convolutional networks, and performs hand-drawn sketch recognition and segmentation at the same time. Based on the advantages of convolutional neural networks in extracting global features, convolutional neural networks are used to extract global features and category recognition from RGB format images. In view of the stroke characteristics of hand-drawn sketches, graph convolutional networks are used to extract local features of hand-drawn sketch format data, and local features and global features are integrated to perform point-level segmentation. Using the prior knowledge of the component categories contained in each category, the category recognition results obtained by the convolutional neural network are used to supervise the segmentation results. The two-stream model is as follows Figure 2 As shown, the convolutional neural network uses the ResNet18 network and the graph convolutional network uses Figure 3 The graph convolution structure shown.

[0039] S3 uses a convolutional neural network to extract features from the hand-drawn sketch image to obtain hand-drawn sketch recognition features, and then uses a category classifier to perform category recognition on the hand-drawn sketch.

[0040] First, the convolutional neural network ResNet18 is used to extract hand-drawn features. The residual structure allows for deeper networks, faster convergence, and easier optimization, while also requiring fewer parameters and reducing complexity compared to previous models. ResNet18 contains 17 convolutional layers; the first layer has 64 convolution kernels, each with a kernel size of 7x7 and a stride of 2. The following 16 kernels are grouped into four groups, each with a kernel size of 3 and a stride of 2. The number of kernels used is 64, 128, 256, and 512, respectively. After passing through the convolutional layers of ResNet18, the hand-drawn sketches produce 512x7x7 features. To account for the sparsity of hand-drawn sketches, the final pooling layer of ResNet18 is modified to convolve the 7x7 features, resulting in 512x1x1 recognition features.

[0041] Secondly, the recognition classifier is used to classify the hand-drawn sketch features and obtain the recognition probability P C .

[0042] S4, the graph convolution network uses a network structure containing 4 dynamic graph convolution units. The feature dimension of each graph convolution unit is 32. Finally, the features of the 4 winter graph convolution units are concatenated into the final 128-dimensional features, as shown in Figure 3 As shown in Figure 2, each dynamic graph convolution unit consists of three parts: updating the graph structure, updating edge features, and updating node features. A residual structure is used for the last three dynamic convolution units to improve stability. The features obtained by each dynamic convolution unit are concatenated to obtain the final point-level features.

[0043] When dynamically updating the network structure, the K-nearest neighbor algorithm is used. The neighborhood of each node in the hand-drawn sketch is calculated. If there is a disconnect, a pairing between the two nodes is added to the edge to improve connectivity. As the number of layers in the graph convolutional network increases, the K-nearest neighbor algorithm is used to obtain and connect neighboring nodes within the neighborhood of each hand-drawn sketch node. The higher the number of layers in the graph convolutional network, the larger the neighborhood when using the K-nearest neighbor algorithm.

[0044] It is known that the node features of layer l nodes i, j are The updated edge features are:

[0045]

[0046] Among them, MLP is a linear layer used to update features; is the feature concatenation operation; ReLU is the activation function; θ l are the linear layer parameters.

[0047] Based on the updated graph structure and edge features, new node features can be evolved as follows:

[0048]

[0049] Among them, j:(i,j)∈El indicates that node j is located in the neighborhood of node i in the updated graph structure.

[0050] After obtaining the updated node features in the above manner, a maximum pooling operation is performed on the nodes belonging to the same stroke to obtain the stroke-level features belonging to that stroke. Furthermore, for each node, the global features in S3 and the stroke-level and point-level features in S4 are concatenated.

[0051] S5, obtained in S4 The predicted probability at the component level is calculated as follows: Where V j represents the set of points contained in the j-th stroke, |V j | represents the number of points in the set, and J represents the number of strokes in the hand-drawn drawing. KL divergence is then used to supervise the component prediction probability based on the category recognition result. The supervision method is to add a loss function to the neural network. The loss function L is used to train and optimize the entire two-stream network model, so that the model can simultaneously recognize and segment hand-drawn sketches, and use the recognition results to supervise the segmentation results to improve the accuracy of hand-drawn sketch segmentation.

[0052] The method of the present invention was compared with SketchGNN, the most advanced segmentation network in the field of hand-drawn image segmentation, on a dataset. The segmentation accuracy of the present invention reached 96.9% and the recognition accuracy reached 92.2%, while the segmentation accuracy of SketchGNN was 88.4%. Compared with existing segmentation methods based on local features, the present invention can simultaneously learn local and global features of hand-drawn sketches. The learned features can better represent the hand-drawn sketch information. Moreover, by using graph convolutional networks and convolutional neural networks, it can overcome the sequential nature of drawing and obtain better segmentation and recognition results by fusing global and local features.

[0053] The present invention is used for hand-drawn sketch recognition and segmentation, including hand-drawn sketch recognition, hand-drawn sketch segmentation, hand-drawn sketch online recognition, hand-drawn sketch recommendation, and hand-drawn sketch target detection, all of which can construct actual application scenarios based on hand-drawn sketch recognition and segmentation information. For example, when drawing a shape using Visio, it is necessary to drag the corresponding shape template from the toolbar, and then adjust the size and angle of the template to obtain the final shape. According to hand-drawn sketch recognition, the user directly draws the shape in the drawing area, and the hand-drawn sketch recognition and segmentation algorithm obtains the shape category, angle, and size information based on the drawn shape, and directly recommends the corresponding category shape to the user, eliminating the user's search process.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for simultaneously recognizing and segmenting hand-drawn sketches, characterized by: The specific steps include: S1, obtaining scalable vector graphics format data of the hand-drawn sketch from the hand-drawn sketch acquisition software, and converting the scalable vector graphics format data of the hand-drawn sketch into RGB image and graph format data; The scalable vector graphics format data is regarded as a tree structure, which is divided into files, layers, and strokes according to the hierarchical relationship; the hand-drawn sketch point sequence can be obtained from the third-level stroke-level data, S = {Pos i |i=1,2,…,N}, N represents the number of points in the hand-drawn sketch, Pos i Represents the i-th point in the entire hand-drawn sketch; each point in the hand-drawn sketch drawing sequence is represented as a triple, Pos i =<x,y,st>, where x represents the horizontal coordinate of the current point, y represents the vertical coordinate of the current point, and st represents the stroke number of the current point. When converting the hand-drawn sketch point sequence into graph format data, the points are used as graph nodes, and edges exist between adjacent nodes within a stroke, thereby creating a node set and an adjacency matrix. S2, builds a two-stream network for simultaneous recognition and segmentation based on convolutional neural networks and graph convolutional networks; Among them, the dual-stream network model based on convolutional neural network and graph convolutional network simultaneously accepts the same hand-drawn RGB image format data and graph structure data as input. The convolutional neural network flow uses ResNet18 as the backbone network, and the graph convolutional network flow uses four-layer dynamic graph convolution as the backbone network. The convolutional neural network flow provides the graph convolutional network flow with global features and component prediction probabilities derived from classification results as supervision information; the loss function of the dual-stream network model based on convolutional neural network and graph convolutional network during training is L=L c +λ1L s +λ2L kl , where L c Cross entropy loss for hand-drawn category classification of convolutional neural network flow, L s is the cross entropy loss for predicting the point-level components of the graph convolutional network flow, and λ1 and λ2 are weight coefficients; S3, inputs the hand-drawn sketch image in RGB format obtained by S1 into the convolutional neural network stream obtained by S2 to obtain the global features and category recognition results that characterize the hand-drawn sketch category; specifically: inputs the hand-drawn sketch image in RGB format into the convolutional neural network stream, and the features of the last fully connected layer of the convolutional neural network are used as the global features that characterize the hand-drawn category, and the output of the classifier is used as the recognition result of the hand-drawn category; based on the prior information of the components contained in each hand-drawn category, a conversion matrix from the hand-drawn category to the component category is constructed When the i-th hand-drawn image contains the j-th component, M(i,j) = 1, otherwise M(i,j) = 0; get N in S3 C Recognition probability of hand-drawn categories After that, the calculation can get the component N S dimensional predicted probability S4: Input the hand-drawn sketch data in graph format obtained in S1 into the graph convolutional network flow to obtain stroke-level features and point-level features, and then splice the global features, stroke-level features, and point-level features to achieve point-level segmentation; Specifically, the hand-drawn sketch data in graph format is input to the graph convolution network flow, and the data in scalable vector graphics format is first sampled to ensure that the number of nodes input to the graph convolution network is constant N P ; Then, based on the point-level features learned by the graph convolutional network and the information of each hand-drawn stroke contained in the scalable vector graphics format, the features of all points belonging to the same stroke are pooled to obtain the stroke-level features; The point-level features of each point, the stroke-level features to which it belongs, and the global features output by the convolutional neural network flow are spliced ​​together to obtain the multi-scale features of each point, and then the component category is predicted by the classifier to obtain the component prediction probability of the i-th point S5, based on the prior information of the component categories contained in each hand-drawn sketch category and the category recognition results of the convolutional neural network flow obtained from S3, uses the KL divergence method to supervise the segmentation results of S4; The predicted probability at the component level is calculated as follows: Where V j represents the set of points contained in the j-th stroke, |V j | represents the number of points in the set, and J represents the number of strokes in the hand-drawn drawing. KL divergence is then used to supervise the component prediction probability based on the category recognition result. The supervision method is to add a loss function to the neural network. The loss function L is used to train and optimize the entire two-stream network model, so that the model can simultaneously recognize and segment hand-drawn sketches, and use the recognition results to optimize the segmentation results, thereby improving the accuracy of hand-drawn sketch recognition and segmentation.

2. The method for simultaneously identifying and segmenting hand-drawn sketches according to claim 1, wherein: In S1, hand-drawn sketch data in a scalable vector graphics format is obtained from a hand-drawn drawing software, the scalable vector graphics is converted into an RGB image, and a point sequence is extracted from the scalable vector graphics to generate a graph as an input to the simultaneous recognition and segmentation network.

3. The method for simultaneously identifying and segmenting hand-drawn sketches according to claim 1, wherein: In S4, the graph convolution network uses a network structure containing 4 dynamic graph convolution units. The feature dimension of each graph convolution unit is 32. Finally, the features of the 4 dynamic graph convolution units are spliced ​​into the final 128-dimensional features. Each dynamic graph convolution unit consists of three parts: updating the graph structure, updating the edge features, and updating the node features. The residual structure is used for the last three dynamic convolution units to increase stability. The features obtained by each dynamic convolution unit are spliced ​​to obtain the final point-level features. When dynamically updating the network structure, the K-nearest neighbor algorithm is used to calculate the other nodes in the neighborhood of each node in the hand-drawn sketch. If they are not connected, a pairing of the current two nodes is added to the edge to improve the connectivity between the nodes. As the number of layers in the graph convolutional network increases, the K-nearest neighbor algorithm is used to obtain the neighboring nodes in the neighborhood of each node in the hand-drawn sketch and connect them. The higher the number of layers in the graph convolutional network, the larger the neighborhood when using the K-nearest neighbor algorithm. It is known that the node feature of layer l node i, j is f i l , The updated edge features are: Among them, MLP is a linear layer used to update features; is the feature concatenation operation; ReLU is the activation function; θ l is the linear layer parameter; Based on the updated graph structure and edge features, the new node features evolve into: Where j:(i,j)∈E l Indicates that node j is located in the neighborhood of node i in the updated graph structure; After obtaining the updated node features in the above manner, a maximum pooling operation is performed on the nodes belonging to the same stroke to obtain the stroke-level features belonging to the stroke; then, for each node, the global features in S3 and the stroke-level features and point-level features in S4 are spliced ​​together.

4. The method for simultaneously identifying and segmenting hand-drawn sketches according to claim 1, wherein: The specific sampling strategy for sampling data in scalable vector graphics format is: if the number of points in the current hand-drawn sketch drawing sequence is greater than N P , reduce the number of points of each stroke in proportion to the percentage of the number of points of each stroke in the total number of points of the hand-drawn sketch; if there is a stroke whose proportion is too small before reducing the number of points, the number of points in the stroke will not be reduced.

5. A system for the method according to any one of claims 1 to 4, characterized in that: include: A data conversion module is used to convert hand-drawn data in scalable vector graphics format into RGB image data and hand-drawn data in graph format as input to the network model; A dual-stream network module, used to simultaneously perform hand-drawn category recognition and component segmentation on the data processed by the data conversion module; The training optimization module is used to construct a loss function and use a deep learning training method to optimize the training of the dual-stream network model of the present invention.

6. A device for simultaneously recognizing and segmenting hand-drawn sketches, characterized by: include: memory for storing computer programs; A processor, configured to implement the method for simultaneously identifying and segmenting hand-drawn sketches as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the user can simultaneously identify and segment the hand-drawn sketch based on the method for simultaneously identifying and segmenting the hand-drawn sketch according to any one of claims 1 to 4.

Images