Cartoon image classification method and device, electronic equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU FRAUDMETRIX TECH CO LTD
- Filing Date
- 2022-09-15
- Publication Date
- 2026-05-22
AI Technical Summary
Existing cartoon character classification technologies cannot categorize cartoon characters according to the animation works they belong to, and cannot meet the needs of different regional cultural differences.
By constructing a relationship network of cartoon characters and using feature extraction network models and feature relationship network models, the similarity between image features and central node features is calculated to determine the animation works to which the cartoon characters belong.
It achieves accurate classification of cartoon characters, enables quick searching of the animation works to which a cartoon character belongs, adapts to cultural differences in different regions, and improves the accuracy and efficiency of classification.
Smart Images
Figure CN115457326B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and specifically relates to a method, apparatus, electronic device and storage medium for classifying cartoon characters. Background Technology
[0002] With the significant increase in the number of domestic and international animation works, the cartoon characters included in these works are also showing a rapid growth trend. Due to cultural differences between different regions, some comics may contain cartoon characters that do not conform to regional cultures. Therefore, it is necessary to classify each cartoon character according to the animation work to which it belongs.
[0003] However, existing technologies for classifying cartoon characters are limited to identifying the identity of the cartoon character, and cannot classify the cartoon character according to the animation work to which it belongs.
[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0005] This application provides a method, apparatus, electronic device, and storage medium for classifying cartoon characters.
[0006] Some embodiments of this application provide a method for classifying cartoon characters, the method comprising:
[0007] The image containing the cartoon character to be classified is input into the feature extraction network model to obtain the image features of the cartoon character to be classified.
[0008] By inputting multiple cartoon character relationship networks into the feature relationship network model, the central node features corresponding to the multiple cartoon character relationship networks are obtained;
[0009] Based on the image features and the central node features corresponding to the multiple cartoon character relationship networks, the classification result of the cartoon character to be classified is output.
[0010] Optionally, the cartoon character relationship network is constructed through the following steps:
[0011] Using each cartoon character as a node, construct multiple relationship networks between the nodes of each cartoon character;
[0012] The relationship information contained in each cartoon character is obtained, and the image features corresponding to each cartoon character are extracted using the feature extraction network model. The relationship information includes: character relationships and plot relationships.
[0013] In the nodes corresponding to each cartoon character, the image features corresponding to each cartoon character and the relationship information contained in each cartoon character are input to obtain the relationship network of the multiple cartoon characters.
[0014] Optionally, after obtaining the plurality of cartoon character relationship networks, the method further includes:
[0015] By establishing known connections between the central node containing the central node's features and other nodes in the target cartoon character relationship network, a cartoon community network is obtained.
[0016] By inputting the cartoon community network into the feature relationship network model, the unknown connection relationships between the central node and other nodes in the target cartoon character relationship network are obtained;
[0017] The unknown connection relationship is used to update the cartoon club network.
[0018] Optionally, the step of outputting the classification result of the cartoon character to be classified based on the image features and the central node features corresponding to the multiple cartoon character relationship networks includes:
[0019] Calculate the similarity between the image features and the features of each of the central nodes;
[0020] The center node feature with the highest similarity to the image features is identified as the target node feature;
[0021] The works corresponding to the cartoon character relationship network to which the target node features belong are identified as the works to which the cartoon character to be classified belongs;
[0022] Output the works to which the cartoon characters to be categorized belong.
[0023] Optionally, calculating the similarity between the image features and the features of each of the central nodes includes:
[0024] The image features and each of the central node features are mapped to a vector space to obtain the vectors of the image features and the vectors of each of the central node features;
[0025] Calculate the similarity parameter between the vector of the image features and the vector of each of the center node features, wherein the similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, dot product;
[0026] Based on the similarity parameter, the similarity between the image features and the features of each of the central nodes is calculated.
[0027] Optionally, the feature extraction network model is trained through the following steps:
[0028] Obtain a dataset of cartoon images;
[0029] By removing non-cartoon images from the cartoon image dataset, a cartoon image dataset is obtained.
[0030] The feature extraction network model is obtained by training the cartoon character dataset.
[0031] Optionally, the feature relationship network model is trained through the following steps:
[0032] The network parameters contained in the cartoon character relationship network are input into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network.
[0033] The error between the predicted classification result and the actual classification result of each node is calculated using a loss function.
[0034] Based on the error, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated to obtain the feature relationship network model.
[0035] Some embodiments of this application provide a cartoon character classification device, the device comprising:
[0036] First extraction module: used to input an image containing a cartoon character to be classified into a feature extraction network model to obtain the image features of the cartoon character to be classified;
[0037] The second extraction module is used to input multiple cartoon character relationship networks into the feature relationship network model to obtain the central node features corresponding to the multiple cartoon character relationship networks respectively.
[0038] Classification module: used to output the classification result of the cartoon character to be classified based on the image features and the central node features corresponding to the multiple cartoon character relationship networks.
[0039] Optionally, the cartoon character relationship network is constructed through the following steps:
[0040] Using each cartoon character as a node, construct multiple relationship networks between the nodes of each cartoon character;
[0041] The relationship information contained in each cartoon character is obtained, and the image features corresponding to each cartoon character are extracted using the feature extraction network model. The relationship information includes: character relationships and plot relationships.
[0042] In the nodes corresponding to each cartoon character, the image features corresponding to each cartoon character and the relationship information contained in each cartoon character are input to obtain the relationship network of the multiple cartoon characters.
[0043] Optionally, after obtaining the plurality of cartoon character relationship networks, the method further includes:
[0044] By establishing known connections between the central node containing the central node's features and other nodes in the target cartoon character relationship network, a cartoon community network is obtained.
[0045] By inputting the cartoon community network into the feature relationship network model, the unknown connection relationships between the central node and other nodes in the target cartoon character relationship network are obtained;
[0046] The unknown connection relationship is used to update the cartoon club network.
[0047] Optionally, the classification module is further configured to:
[0048] Calculate the similarity between the image features and the features of each of the central nodes;
[0049] The center node feature with the highest similarity to the image features is identified as the target node feature;
[0050] The works corresponding to the cartoon character relationship network to which the target node features belong are identified as the works to which the cartoon character to be classified belongs;
[0051] Output the works to which the cartoon characters to be categorized belong.
[0052] Optionally, the classification module further includes a calculation unit for:
[0053] The image features and each of the central node features are mapped to a vector space to obtain the vectors of the image features and the vectors of each of the central node features;
[0054] Calculate the similarity parameter between the vector of the image features and the vector of each of the center node features, wherein the similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, dot product;
[0055] Based on the similarity parameter, the similarity between the image features and the features of each of the central nodes is calculated.
[0056] Optionally, the feature extraction network model is trained through the following steps:
[0057] Obtain a dataset of cartoon images;
[0058] By removing non-cartoon images from the cartoon image dataset, a cartoon image dataset is obtained.
[0059] The feature extraction network model is obtained by training the cartoon character dataset.
[0060] Optionally, the feature relationship network model is trained through the following steps:
[0061] The network parameters contained in the cartoon character relationship network are input into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network.
[0062] The error between the predicted classification result and the actual classification result of each node is calculated using a loss function.
[0063] Based on the error, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated to obtain the feature relationship network model.
[0064] Some embodiments of this application provide an electronic device, including:
[0065] Memory containing computer-readable code;
[0066] One or more processors, when the computer-readable code is executed by the one or more processors, the electronic device performs the cartoon character classification method as described above.
[0067] Some embodiments of this application provide a computer program including computer-readable code that, when run on an electronic device, causes the electronic device to perform the cartoon character classification method described above.
[0068] Some embodiments of this application provide a storage medium storing the cartoon character classification method described above.
[0069] The cartoon character classification methods, devices, electronic devices, and storage media provided in some embodiments of this application can make full use of the rich plot connections and character connections between various cartoon characters to classify the cartoon characters to be classified into their respective cartoon works.
[0070] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0071] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0072] Figure 1 A schematic diagram illustrating a cartoon character classification method provided in this application is shown.
[0073] Figure 2 This illustration schematically shows a flowchart of the method for feature extraction from an image containing multiple cartoon characters to be classified, as provided in this application.
[0074] Figure 3 The illustration shows a schematic diagram of a cartoon character relationship network provided in this application;
[0075] Figure 4 This illustration schematically shows a method flowchart for outputting the work to which a cartoon character to be classified belongs, as provided in this application.
[0076] Figure 5 A schematic diagram illustrating a method for calculating the similarity between features provided in this application is shown.
[0077] Figure 6 The schematic diagram illustrates a method flowchart for updating a cartoon community network provided in this application;
[0078] Figure 7 The schematic diagram illustrates a flowchart of a feature extraction network model provided in this application;
[0079] Figure 8 A schematic diagram illustrating a process for obtaining a feature relationship network model provided in this application is shown.
[0080] Figure 9 A schematic diagram of a three-layer graph convolutional neural network provided in this application is shown.
[0081] Figure 10 A schematic diagram of the structure of a cartoon character classification device provided in this application is shown.
[0082] Figure 11 A schematic diagram of an electronic device that can implement the method according to this application is shown;
[0083] Figure 12 A storage unit for holding or carrying program code implementing methods according to some embodiments of this application is illustrated schematically. Detailed Implementation
[0084] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0085] For example, Figure 1 The schematic diagram illustrates a flowchart of a cartoon character classification method provided in this application, the method comprising:
[0086] Step 101: Input the image containing the cartoon character to be classified into the feature extraction network model to obtain the image features of the cartoon character to be classified.
[0087] In this embodiment, the feature extraction network model includes, but is not limited to, CNN (Convolutional Neural Network). An image containing the cartoon character to be classified is input into the feature extraction network model to obtain the image features of the cartoon character to be classified.
[0088] Optionally, refer to Figure 2 This paper illustrates a flowchart of a method for feature extraction from an image containing multiple cartoon characters to be classified, as provided in this application. First, the cartoon characters to be classified in the image are detected and located. Then, image segmentation techniques are used to segment the detected cartoon characters to obtain the multiple cartoon characters to be classified within the image. Next, a CNN network is used to extract features from each cartoon character to obtain the image features of each cartoon character to be classified.
[0089] Step 102: Input the multiple cartoon character relationship networks into the feature relationship network model to obtain the central node features corresponding to the multiple cartoon character relationship networks.
[0090] In this application embodiment, the cartoon character relationship network refers to a connection network that includes the character relationships or plot relationships between multiple cartoon characters. The feature relationship network model includes, but is not limited to, GCN (Graph Convolutional Network).
[0091] refer to Figure 3The diagram illustrates a cartoon character relationship network provided in this application, including a first cartoon character relationship network 10 and a second cartoon character relationship network 20. Solid lines in the diagram represent connections established based on known character or plot relationships between various cartoon characters.
[0092] Step 103: Based on the image features and the central node features corresponding to the multiple cartoon character relationship networks, output the classification result of the cartoon character to be classified.
[0093] In the embodiments of this application, reference continues to be made to Figure 3 The central node refers to the intersection of relationships between various cartoon characters in a cartoon character relationship network. For example, "Big Head Son and Small Head Dad" is the central node of the first cartoon character relationship network 10, and "Big Head Son and Small Head Dad Sequel" is the central node of the second cartoon character relationship network 20. The features of the central node encompass the image features of each cartoon character in the cartoon character relationship network. The dashed lines in the diagram represent cartoon characters in the cartoon character relationship network that have not yet established a connection with the central node. The relationships between each cartoon character and the central node can be established through training a feature relationship network model and are reflected in the final generated visualization. The crosses on the solid lines in the diagram represent incorrect relationships in the established connections, meaning that the cartoon character does not belong to the cartoon character relationship network corresponding to the connected central node.
[0094] This embodiment of the application inputs an image containing the cartoon character to be classified into a feature extraction network model to obtain the image features of the cartoon character to be classified. Then, it inputs multiple cartoon character relationship networks into a feature relationship network model to obtain the central node features corresponding to each of the multiple cartoon character relationship networks. Finally, based on the image features and the central node features corresponding to the multiple cartoon character relationship networks, the classification result of the cartoon character to be classified is output. This fully utilizes the rich plot connections and character connections between various cartoon characters to classify the cartoon character to be classified into its respective cartoon work.
[0095] Optionally, refer to Figure 4 Step 103 includes:
[0096] Step 1031: Calculate the similarity between the image features and the features of each of the central nodes.
[0097] In this embodiment, similarity refers to the similarity between the feature vector of the image feature and the feature vector of the center node feature.
[0098] Step 1032: Determine the center node feature with the highest similarity to the image features as the target node feature.
[0099] In this embodiment of the application, the central node feature with the highest image feature similarity to the cartoon character to be classified is determined as the target node feature.
[0100] For example, a user inputs a cartoon character to be classified, and four cartoon character relationship networks have been established, each containing a central node. Assuming the similarity between the image features of the cartoon character to be classified and the features of the central nodes in the four cartoon character relationship networks are 0.2, 0.5, 0.9, and 0.1 respectively, then the central node feature with a similarity of 0.9 to the image features of the cartoon character to be classified is determined as the target node feature. As another example, a user inputs a cartoon character to be classified, and five cartoon character relationship networks have been established, each containing a central node. Assuming the similarity between the image features of the cartoon character to be classified and the features of the central nodes in the four cartoon character relationship networks are 0.3, 0.7, 0.95, 0.1, and 0.4 respectively, then the central node feature with a similarity of 0.95 to the image features of the cartoon character to be classified is determined as the target node feature.
[0101] Step 1033: Determine the works corresponding to the cartoon character relationship network to which the target node features belong as the works to which the cartoon character to be classified belongs.
[0102] In this embodiment of the application, each cartoon character relationship network is associated with a work. The work corresponding to the cartoon character relationship network to which the target node feature belongs is determined as the work to which the cartoon character to be classified belongs. At this time, the classification of the cartoon character to be classified is completed.
[0103] Optionally, after classifying the cartoon characters to be classified, the image features of the cartoon characters to be classified can be input into their respective cartoon character relationship networks, and connections between the cartoon characters to be classified and other cartoon characters can be established. As the number of classifications increases, the accuracy of the feature relationship network model in extracting the central node features of the cartoon character relationship network can be improved.
[0104] Step 1034: Output the works to which the cartoon characters to be classified belong.
[0105] In this embodiment of the application, the works to which the cartoon characters to be classified belong are output.
[0106] This application embodiment calculates the similarity between image features and the features of each central node, determines the central node feature with the highest similarity to the image features as the target node feature, and identifies the works corresponding to the cartoon character relationship network to which the target node feature belongs as the works to which the cartoon character to be classified belongs. Finally, the works to which the cartoon character to be classified belongs are output, enabling rapid retrieval of cartoon works to which the cartoon character to be classified belongs. This allows users to quickly find cartoon works when they do not know which cartoon character belongs, enabling them to watch the work. It can also quickly find works to which cartoon characters are prohibited by regional culture, thus prohibiting such cartoon works.
[0107] Optionally, refer to Figure 5 Step 1031 includes:
[0108] Step 10311: Map the image features and each of the central node features to a vector space to obtain the vectors of the image features and the vectors of each of the central node features.
[0109] In this embodiment of the application, the image features and the features of each central node are mapped to the vector space respectively to obtain the vector of the image features and the vector of each central node feature.
[0110] Step 10312: Calculate the similarity parameter between the vector of the image features and the vector of each of the center node features. The similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, and dot product.
[0111] In this embodiment of the application, a similarity parameter is calculated between the vector of image features and the vector of features of each center node. The similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, and dot product.
[0112] Step 10313: Based on the similarity parameter, calculate the similarity between the image features and the features of each of the central nodes.
[0113] In this embodiment of the application, the similarity between image features and the features of each central node is calculated based on the similarity parameter.
[0114] For example, when the similarity parameters include Euclidean distance and cosine, assuming the Euclidean distance between the vector of the image feature and the vector of the first central node feature is 4 and the cosine is 0.5, and the Euclidean distance between the vector of the image feature and the vector of the second central node feature is 3 and the cosine is 0.8, since the Euclidean distance between the vector of the image feature and the vector of the second central node feature is less than the Euclidean distance between the vector of the image feature and the vector of the first central node feature, and the cosine between the vector of the image feature and the vector of the second central node feature is greater than the cosine between the vector of the image feature and the vector of the first central node feature, the second central node feature is the central node feature with the greatest similarity to the image feature. Therefore, the second central node feature is determined as the target node feature, and the work corresponding to the cartoon character relationship network to which the second central node feature belongs is determined as the work to which the cartoon character to be classified belongs.
[0115] Some embodiments of this application map image features and each central node feature to a vector space, then calculate the similarity parameter between the vectors of the image features and the vectors of each central node feature based on the obtained vectors of the image features and the vectors of each central node feature. Finally, based on the similarity parameter, the similarity between the image features and each central node feature is calculated. This allows the selection of the central node feature with the highest similarity to the image features of the cartoon character to be classified from multiple central node features, thereby achieving accurate calculation of the similarity between the image features of the cartoon character to be classified and each central node feature, and thus improving the accuracy of subsequent classification of the cartoon character to be classified.
[0116] Figure 6 The schematic diagram illustrates a method flowchart for updating a cartoon community network provided in this application, including:
[0117] Step 201: Using each cartoon character as a node, construct multiple relationship networks between the nodes of each cartoon character.
[0118] In this embodiment of the application, each cartoon character is used as a node to construct a relationship network between the nodes of each cartoon character.
[0119] For example, refer to Figure 3 The first cartoon character relationship network 10 is a cartoon character relationship network containing 11 cartoon characters as nodes, and the second cartoon character relationship network 20 is a cartoon character relationship network containing 3 cartoon characters as nodes.
[0120] Step 202: Obtain the relationship information contained in each cartoon character, and use the feature extraction network model to extract the image features corresponding to each cartoon character respectively. The relationship information includes: character relationship and plot relationship.
[0121] In this embodiment, the character and plot relationships between various cartoon characters are obtained, and a feature extraction network model is used to extract the image features corresponding to each cartoon character. The cartoon characters can be obtained through internet searches or by utilizing existing cartoon character datasets; this application does not limit the methods used.
[0122] Step 203: Input the image features corresponding to each cartoon character and the relationship information contained in each cartoon character into the nodes corresponding to each cartoon character to obtain the relationship network of multiple cartoon characters.
[0123] In this embodiment, image features corresponding to each cartoon character and relational information contained in each cartoon character are input into the nodes corresponding to each cartoon character to obtain a network of relationships between multiple cartoon characters. Since each node corresponding to each cartoon character contains a large amount of real relational information with other nodes, this information reduces the negative effects of false information, thereby reducing the noise impact caused by manual annotation.
[0124] Step 204: Establish known connections between the central node containing the central node features and other nodes in the target cartoon character relationship network to obtain the cartoon community network.
[0125] In this embodiment, the cartoon community network refers to a network of cartoon character relationships where a central node has established known connections with other nodes. These known connections refer to the character relationships and / or plot relationships between nodes obtained beforehand by the user through a search. The cartoon community network is obtained by establishing known connections between the central node, which contains the characteristics of the central node, and other nodes in the target cartoon character relationship network. For example... Figure 3 As shown, the central node of the first cartoon character relationship network 10 is "Big Head Son and Small Head Dad". The solid lines between the various cartoon characters in the figure represent known connections between the various cartoon characters. The solid lines between the various cartoon characters and the central node represent known connections between the various cartoon characters and the central node.
[0126] Step 205: Input the cartoon community network into the feature relationship network model to obtain the unknown connection relationship between the central node and other nodes in the target cartoon character relationship network.
[0127] In the embodiments of this application, such as Figure 3 As shown, after establishing known connections between the central node and other nodes, the first cartoon character relationship network 10 becomes a cartoon community network. Inputting the cartoon community network into the feature relationship network model yields the unknown connections between the central node and other nodes in the target cartoon character relationship network, such as... Figure 3 The dashed lines connecting the central node to other nodes are shown in the diagram. Therefore, this embodiment does not require modeling and recognizing all cartoon characters one by one, nor does it require establishing known connections between the nodes corresponding to all cartoon characters and the central node in advance. This saves the effort spent on establishing known connections between the nodes.
[0128] Step 206: Update the cartoon community network using the unknown connection relationship.
[0129] In this embodiment of the application, the update of the cartoon community network is completed after the unknown connection relationship between the central node and other nodes in the cartoon community network is established.
[0130] This application embodiment constructs multiple relationship networks between nodes of each cartoon character, using each cartoon character as a node, to obtain the relationship information contained in each cartoon character, and uses a feature extraction network model to extract the image features corresponding to each cartoon character. By inputting the image features and relationship information of each cartoon character into the nodes corresponding to each cartoon character, a large number of cartoon character relationship networks with interwoven relationships can be obtained. By establishing known connections between the central node containing the central node's features and other nodes in the target cartoon character relationship network, a cartoon community network is obtained. Then, the cartoon community network is input into the feature relationship network model to obtain unknown connections between the central node and other nodes in the target cartoon character relationship network. Finally, the unknown connections are used to update the cartoon community network, further completing the relationships between the central node and other nodes, resulting in a more complete cartoon community network, thereby improving the accuracy of classifying the cartoon characters to be classified.
[0131] Figure 7 The schematic diagram illustrates a process for obtaining a feature extraction network model according to this application, including:
[0132] Step 301: Obtain the cartoon image dataset.
[0133] In the embodiments of this application, the methods for obtaining cartoon image datasets include, but are not limited to: obtaining cartoon image datasets from posters of various cartoon works, or obtaining cartoon image datasets from anime websites.
[0134] Step 302: Remove non-cartoon images from the cartoon image dataset to obtain the cartoon image dataset.
[0135] In this embodiment, non-cartoon characters refer to cartoon images that do not possess a living form, such as buildings and electronic devices. An image segmentation algorithm is used to remove non-cartoon characters from the cartoon image dataset, resulting in a cartoon character dataset.
[0136] Step 303: Train the cartoon character dataset to obtain the feature extraction network model.
[0137] In this embodiment of the application, a feature extraction network model is obtained by training a cartoon character dataset.
[0138] The embodiments of this application can remove non-cartoon characters from a large number of cartoon images to obtain a large number of cartoon characters, thereby improving the training efficiency of the cartoon character dataset and thus improving the feature extraction network model's ability to extract image features of cartoon characters.
[0139] Optionally, Figure 8 The schematic diagram illustrates a process for obtaining a feature relationship network model provided in this application, including:
[0140] Step 401: Input the network parameters contained in the cartoon character relationship network into a three-layer graph convolutional neural network to obtain the prediction classification results of each node in the cartoon character relationship network.
[0141] refer to Figure 9 The diagram illustrates a three-layer graph convolutional neural network provided in this application, including an input layer A, three hidden layers B, an output layer C, and an activation function D. The activation function D is ReLU (Rectified Linear Unit).
[0142] In this embodiment, the network parameters of the cartoon character relationship network are input into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network. The mathematical expression of the three-layer graph convolutional neural network is shown in formula (1):
[0143]
[0144] Where A is the input, which is composed of the relational adjacency matrix A' + the corresponding node attribute relation matrix V' + the identity matrix I'. Let A' be the degree matrix corresponding to the adjacency matrix, H be the corresponding hidden layer element matrix, and W be its corresponding weight matrix.
[0145] Step 402: Calculate the error between the predicted classification result of each node and the actual classification result of each node using the loss function.
[0146] In this embodiment, a loss function is used to calculate the error between the predicted classification result and the actual classification result of each node. The loss function includes, but is not limited to, the cross-entropy loss function and the squared difference loss function.
[0147] Step 403: Update the weight coefficients of each hidden layer in the three-layer graph convolutional neural network based on the error to obtain the feature relationship network model.
[0148] In this embodiment, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated based on the error between the predicted classification result of each node and the actual classification result of each node, so as to obtain the feature relationship network model.
[0149] This application embodiment involves inputting the network parameters of the cartoon character relationship network into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network. A loss function is used to calculate the error between the predicted classification results and the actual classification results of each node. Based on this error, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated. This continuously reduces the error between the predicted and actual classification results of each node, thereby improving the classification accuracy of the feature relationship network model for each node in the cartoon character relationship network.
[0150] Of course, the above is just an illustrative description, and the specific data can be set according to actual needs. No restrictions are imposed here.
[0151] Figure 10 The schematic diagram illustrates the structure of a cartoon character classification device 50 provided in this application, the device comprising:
[0152] First extraction module 501: used to input an image containing a cartoon character to be classified into a feature extraction network model to obtain the image features of the cartoon character to be classified;
[0153] The second extraction module 502 is used to input multiple cartoon character relationship networks into the feature relationship network model to obtain the central node features corresponding to the multiple cartoon character relationship networks respectively.
[0154] Classification module 503: Used to output the classification result of the cartoon character to be classified based on the image features and the central node features corresponding to the multiple cartoon character relationship networks.
[0155] Optionally, the cartoon character relationship network is constructed through the following steps:
[0156] Using each cartoon character as a node, construct multiple relationship networks between the nodes of each cartoon character;
[0157] The relationship information contained in each cartoon character is obtained, and the image features corresponding to each cartoon character are extracted using the feature extraction network model. The relationship information includes: character relationships and plot relationships.
[0158] In the nodes corresponding to each cartoon character, the image features corresponding to each cartoon character and the relationship information contained in each cartoon character are input to obtain the relationship network of the multiple cartoon characters.
[0159] Optionally, after obtaining the plurality of cartoon character relationship networks, the method further includes:
[0160] By establishing known connections between the central node containing the central node's features and other nodes in the target cartoon character relationship network, a cartoon community network is obtained.
[0161] By inputting the cartoon community network into the feature relationship network model, the unknown connection relationships between the central node and other nodes in the target cartoon character relationship network are obtained;
[0162] The unknown connection relationship is used to update the cartoon club network.
[0163] Optionally, the classification module is further configured to:
[0164] Calculate the similarity between the image features and the features of each of the central nodes;
[0165] The center node feature with the highest similarity to the image features is identified as the target node feature;
[0166] The works corresponding to the cartoon character relationship network to which the target node features belong are identified as the works to which the cartoon character to be classified belongs;
[0167] Output the works to which the cartoon characters to be categorized belong.
[0168] Optionally, the classification module further includes a calculation unit for:
[0169] The image features and each of the central node features are mapped to a vector space to obtain the vectors of the image features and the vectors of each of the central node features;
[0170] Calculate the similarity parameter between the vector of the image features and the vector of each of the center node features, wherein the similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, dot product;
[0171] Based on the similarity parameter, the similarity between the image features and the features of each of the central nodes is calculated.
[0172] Optionally, the feature extraction network model is trained through the following steps:
[0173] Obtain a dataset of cartoon images;
[0174] By removing non-cartoon images from the cartoon image dataset, a cartoon image dataset is obtained.
[0175] The feature extraction network model is obtained by training the cartoon character dataset.
[0176] Optionally, the feature relationship network model is trained through the following steps:
[0177] The network parameters contained in the cartoon character relationship network are input into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network.
[0178] The error between the predicted classification result and the actual classification result of each node is calculated using a loss function.
[0179] Based on the error, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated to obtain the feature relationship network model.
[0180] The embodiments of this application can make full use of the rich plot connections and character connections between various cartoon characters to classify the cartoon characters to be classified into their respective cartoon works.
[0181] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a storage medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0182] For example, Figure 11An electronic device is shown that can implement the methods according to this application. The electronic device conventionally includes a processor 610 and a computer program product or storage medium in the form of memory 620. Memory 620 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 620 has storage space 630 for program code 631 for performing any of the method steps described above. For example, storage space 630 for program code may include various program codes 631 respectively for implementing the various steps in the methods described above. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. Such computer program products are typically as shown in the references. Figure 12 The portable or fixed storage unit is described above. This storage unit may have the same characteristics as... Figure 11 The memory 620 in the electronic device is similarly arranged as storage segments, storage spaces, etc. Program code can be compressed, for example, in an appropriate form. Typically, the storage unit includes computer-readable code 631', that is, code that can be read by a processor such as 610, which, when executed by the electronic device, causes the electronic device to perform the various steps in the methods described above.
[0183] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0184] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.
[0185] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0186] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for classifying cartoon characters, characterized in that, The method includes: The image containing the cartoon character to be classified is input into the feature extraction network model to obtain the image features of the cartoon character to be classified. Multiple cartoon character relationship networks are input into a feature relationship network model to obtain the central node features corresponding to each of the multiple cartoon character relationship networks. This includes: constructing multiple relationship networks between nodes of each cartoon character, using each cartoon character as a node; obtaining the relationship information contained in each cartoon character, and using the feature extraction network model to extract the image features corresponding to each cartoon character, the relationship information including character relationships and plot relationships; inputting the image features corresponding to each cartoon character and the relationship information contained in each cartoon character into the nodes corresponding to each cartoon character to obtain the multiple cartoon character relationship networks; the central node features are used to characterize the image features of each cartoon character in the cartoon character relationship network. Based on the image features and the central node features corresponding to the multiple cartoon character relationship networks, the classification result of the cartoon character to be classified is output, including: calculating the similarity between the image features and each of the central node features; determining the central node feature with the highest similarity to the image features as the target node feature; determining the work corresponding to the cartoon character relationship network to which the target node feature belongs as the work to which the cartoon character to be classified belongs; and outputting the work to which the cartoon character to be classified belongs.
2. The cartoon character classification method according to claim 1, characterized in that, After obtaining the relationship network of the multiple cartoon characters, the method further includes: By establishing known connections between the central node containing the central node's features and other nodes in the target cartoon character relationship network, a cartoon community network is obtained. By inputting the cartoon community network into the feature relationship network model, the unknown connection relationships between the central node and other nodes in the target cartoon character relationship network are obtained; The unknown connection relationship is used to update the cartoon club network.
3. The cartoon character classification method according to claim 1, characterized in that, The step of calculating the similarity between the image features and the features of each of the central nodes includes: The image features and each of the central node features are mapped to a vector space to obtain the vectors of the image features and the vectors of each of the central node features; Calculate the similarity parameter between the vector of the image features and the vector of each of the center node features, wherein the similarity parameter includes one or more of the following parameters: Euclidean distance, cosine, dot product; Based on the similarity parameter, the similarity between the image features and the features of each of the central nodes is calculated.
4. The cartoon character classification method according to claim 1, characterized in that, The feature extraction network model is trained through the following steps: Obtain a dataset of cartoon images; By removing non-cartoon images from the cartoon image dataset, a cartoon image dataset is obtained. The feature extraction network model is obtained by training the cartoon character dataset.
5. The cartoon character classification method according to claim 1, characterized in that, The feature relationship network model is trained through the following steps: The network parameters contained in the cartoon character relationship network are input into a three-layer graph convolutional neural network to obtain the predicted classification results of each node in the cartoon character relationship network. The error between the predicted classification result and the actual classification result of each node is calculated using a loss function. Based on the error, the weight coefficients of each hidden layer in the three-layer graph convolutional neural network are updated to obtain the feature relationship network model.
6. A cartoon character classification device, characterized in that, include: First extraction module: used to input an image containing a cartoon character to be classified into a feature extraction network model to obtain the image features of the cartoon character to be classified; The second extraction module is used to input multiple cartoon character relationship networks into a feature relationship network model to obtain the central node features corresponding to each of the multiple cartoon character relationship networks. This includes: constructing multiple relationship networks between nodes of each cartoon character, using each cartoon character as a node; obtaining the relationship information contained in each cartoon character, and using the feature extraction network model to extract the image features corresponding to each cartoon character, where the relationship information includes character relationships and plot relationships; inputting the image features corresponding to each cartoon character and the relationship information contained in each cartoon character into the nodes corresponding to each cartoon character to obtain the multiple cartoon character relationship networks; the central node features are used to characterize the image features of each cartoon character in the cartoon character relationship network. The classification module is used to output the classification result of the cartoon character to be classified based on the image features and the central node features corresponding to the multiple cartoon character relationship networks, including: calculating the similarity between the image features and each of the central node features; determining the central node feature with the highest similarity to the image features as the target node feature; determining the work corresponding to the cartoon character relationship network to which the target node feature belongs as the work to which the cartoon character to be classified belongs; and outputting the work to which the cartoon character to be classified belongs.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the cartoon character classification method as described in any one of claims 1-5.
8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cartoon character classification method as described in any one of claims 1-5.