A Product Design Sketch Evaluation Method Based on Capsule Networks

Abstract

This invention relates to the fields of artificial intelligence and industrial design technology, specifically providing a product design sketch evaluation method based on capsule networks. It aims to address the lack of a specific sketch evaluation method tailored to the sketch itself in existing technologies. In this invention, the sketch and target semantics are input into a product design sketch evaluation model. By extracting semantic features from the sketch, the recognition rate of the sketch image is improved through the inverted dot product attention routing mechanism in the capsule network, thereby determining the fit between the sketch and the target semantics. This invention achieves the evaluation of the sketch; by applying capsule networks to the field of product design evaluation, it provides a sketch evaluation method that combines capsule networks and targets the sketch itself, possessing a certain degree of accuracy and feasibility in sketch evaluation.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and industrial design technology, and specifically relates to a product design sketch evaluation method based on capsule networks. Background Technology

[0002] Industrial design is an emerging interdisciplinary field that integrates art and science, each with its own characteristics, focusing on the mechanized production of industrial products. As a crucial component of innovative design, industrial design helps establish a link between the multiple physical attributes of a product and the user's subjective perception, promoting the realization of the value of innovative achievements. Although many digital design methods exist today, many designers still choose sketches during the product design phase for analysis and communication, primarily because the speed and efficiency of sketching are widely accepted. However, while sketches are widely used in industrial design, they are often rudimentary and vague. Due to the varying backgrounds and drawing methods of designers, sketching is highly subjective and cannot express all design details, making it difficult to develop a concrete evaluation method.

[0003] With the widespread application of artificial intelligence in product design, domestic scholars have also extensively used related technologies to improve the accuracy and efficiency of product design sketch evaluation. For example, domestic scholars have implemented parameterization, recognition, and final image generation of sketch images from the perspectives of topological constraints, joint feature learning, and generative adversarial networks. However, the above-mentioned research only improves the accuracy of sketch evaluation to a certain extent by transforming the sketch into another type of data, and there is still a lack of a specific sketch evaluation method for the purpose of evaluating the sketch itself. Summary of the Invention

[0004] This invention provides a product design sketch evaluation method based on capsule networks, aiming to solve the problem that there is no specific sketch evaluation method in the prior art that is tailored to the sketch itself.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a product design sketch evaluation method based on capsule networks, the specific steps of which include:

[0007] S100. Collect sketches and input the 448×448 pixel sketches into the product design sketch evaluation model;

[0008] S200. Input the target semantics of the product design into the product design sketch evaluation model; the product design sketch evaluation model includes a feature extraction module and a capsule network module configured with an inverted dot product attention routing mechanism;

[0009] S300, The feature extraction module performs feature extraction on the sketch with a pixel size of 448×448 to obtain a feature vector with a size of 14×14×768;

[0010] S400. The feature vector with a size of 14×14×768 is input into the capsule network module, and the capsule network module outputs the fit between the sketch and the target semantics.

[0011] The capsule network module has primary capsule layers arranged from bottom to top. l 1. Convolutional capsule layer l 2. Convolutional capsule layer l 3. Fully connected capsule layer l 4 and fully connected capsule layers l 5;

[0012] The feature vector is input to the primary capsule layer. l In 1, the primary capsule layer l 1. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l Update of the capsule pose in 2, the convolutional capsule layer l 2. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l The capsule pose is updated in step 3 until the fully connected capsule layer. l 4. The fully connected capsule layer is processed using an algorithm based on the inverted dot product attention routing mechanism. l Update to the capsule pose in 5;

[0013] The fully connected capsule layer l In step 5, each capsule corresponds to a semantic label category, and the fit between each capsule and the corresponding semantic label is calculated using the logits function of the softmax function.

[0014] Based on the above scheme, the basic unit of the capsule network module is a capsule containing feature vectors, which can indicate the probability of feature existence and the direction of different features. It can identify the subordinate relationship of features in the image, thereby further improving the image recognition capability of the product design sketch evaluation model.

[0015] A further solution: In step S300, the feature extraction module performs feature extraction as follows:

[0016] S301. After downsampling the 448×448 pixel sketch with a convolution kernel of 4×4 and a stride of 4, normalize it to obtain a feature vector with a size of 112×112×3.

[0017] S302. Perform the first ConvNeXt Block stacking on the feature vector with size 112×112×3. The depth of the first ConvNeXt Block stacking is 96 and the number of stacking times is 3, to obtain a feature vector with size 112×112×3.

[0018] S303. After downsampling the feature vector with size 112×112×3, perform a second ConvNeXtBlock stacking. The depth of the second ConvNeXt Block stacking is 192 and the stacking number is 3, resulting in a feature vector with size 56×56×192.

[0019] The downsampling operation involves first normalizing the feature vector, and then performing a convolution with a kernel of 2×2 and a stride of 2.

[0020] S304. After downsampling the feature vector with size 56×56×192, perform a third ConvNeXtBlock stacking. The depth of the third ConvNeXt Block stacking is 384 and the number of stacking times is 27, resulting in a feature vector with size 28×28×384.

[0021] S305. After downsampling the feature vector with size 28×28×384, perform a fourth ConvNeXtBlock stacking. The depth of the fourth ConvNeXt Block stacking is 768 and the stacking number is 3, resulting in a feature vector with size 14×14×768.

[0022] A further approach: The specific process for each ConvNeXt Block stacking includes:

[0023] Step 1: After passing the feature vector through a convolutional layer with a kernel size of 7×7, a stride of 1, and padding of 3, normalize it.

[0024] The second step is to pass the normalized feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, and then activate it through an activation function, increasing the depth to 4 times the depth in the ConvNeXt Block stack.

[0025] Step 3: After passing the activated feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, the depth is restored by sequentially passing the Layer Scale function and DropPath operation, and the regularized feature vector is output.

[0026] Based on the above scheme, the semantic features of the sketch are extracted by the feature extraction module. On the one hand, this reduces the amount of data stored and the computational complexity of the computer. On the other hand, it enables the capsule network module to perform targeted semantic evaluation of the sketch, further improving the evaluation efficiency and accuracy of the product design sketch evaluation model.

[0027] A further option: from the primary capsule layer l 1 to the fully connected capsule layer l 5. The algorithm steps of the inverted dot product attention routing mechanism between adjacent capsule layers include:

[0028] S401, Computational Capsule and capsules Consistency probability between The consistency probability The calculation formula is:

[0029] ;

[0030] in, l This represents the layer index of the capsule layer, and l 1≤ l ≤ l 5; Indicates capsule layer l Chinese capsules The i-th capsule of the matrix, Indicates capsule layer l +1 capsule The j-th capsule of the matrix, Indicates capsule layer l Chinese capsules The i-th to j-th capsules of the matrix, The capsule With the capsule Connection weights;

[0031] S402, Obtain the capsule Posture update protocol The posture update protocol The calculation formula is:

[0032] in, Indicates matrix transpose;

[0033] S403, the capsule The posture is updated via the posture update protocol. Update; the capsule The formula for calculating the pose update is:

[0034] ;

[0035] ;

[0036] in, ∑z represents the exponential operation with the base e of the natural logarithm; ∑z represents the summation symbol starting from the z-th capsule; The capsule and the capsules The probability of the inverted dot product attention routing mechanism between them; This represents the normalization algorithm.

[0037] Based on the above scheme, the inverted dot product attention routing mechanism improves the accuracy of the capsule network module in recognizing sketches by updating the poses of higher-level capsule layers through lower-level capsule layers; at the same time, the inverted dot product attention routing mechanism also ensures the accuracy of the capsule network module in evaluating sketches when the sketches contain relatively little semantic feature information.

[0038] A further solution: The training process of the product design sketch evaluation model includes:

[0039] Step 1: Collect sketches and preprocess them to construct a sketch image dataset with a pixel size of 448×448;

[0040] Step 2: Perform contour drawing, image segmentation, and contour extraction on the sketch image dataset, and perform semantic annotation to obtain the annotated training dataset;

[0041] Step 3: Input the training dataset into the feature extraction module to extract features and obtain a feature vector of size 14×14×768;

[0042] Step 4: Input the feature vector with a size of 14×14×768 into the capsule network module for training, and obtain the trained product design sketch evaluation model through the algorithm of the inverted dot product attention routing mechanism.

[0043] A further approach: In step one, the preprocessing of the sketch includes denoising, image enhancement, scaling, and / or cropping.

[0044] Based on the above scheme, preprocessing the sketches ensures that the product design sketch evaluation model can be trained normally and obtain the best training results.

[0045] A further approach: In step two, contour lines are drawn and extracted from a portion of the sketches in the training dataset; the remaining sketches in the training dataset are automatically segmented and their contours extracted through a learning network.

[0046] Based on the above scheme, preprocessing the sketches ensures that the product design sketch evaluation model can be trained normally and obtain the best training results.

[0047] In step two, contour lines are drawn and extracted from a portion of the sketches in the training dataset; the remaining sketches in the training dataset are automatically segmented and their contours extracted through a learning network.

[0048] Based on the above scheme, since the training dataset contains several hundred images, the outlines of a portion of the sketches are first drawn and extracted, and the remaining sketches are automatically processed by the learning network, which improves the efficiency of sketch outline extraction.

[0049] In a second aspect, the present invention provides an electronic device, comprising:

[0050] At least one memory for storing programs;

[0051] At least one processor for executing the program stored in the memory;

[0052] Input devices are used to input programs into memory;

[0053] An output device for displaying the results of the processor executing the program stored in the memory;

[0054] When the program stored in the memory is executed, the processor is used to execute a product design sketch evaluation method based on capsule networks as described in the first aspect.

[0055] This invention provides a computer-readable storage medium storing computer instructions that cause the computer to implement a product design sketch evaluation method based on capsule networks as described in the first aspect.

[0056] The beneficial effects of this invention are as follows:

[0057] This invention applies capsule networks to the field of product design evaluation, providing a sketch evaluation method that combines capsule networks with sketches themselves. By extracting semantic features from the sketch and using the inverted dot product attention routing mechanism in the capsule network to improve the recognition rate of the sketch image, the method achieves sketch evaluation and improves the accuracy of sketch evaluation. Compared to existing convolutional neural networks that only recognize local features in an image, the capsule network in this invention also recognizes the hierarchical relationships of features in the image. Its basic unit is a capsule containing feature vectors, which can not only indicate the probability of feature existence but also represent the direction of different features, further improving image recognition performance. In addition, the capsule network in this invention only needs to learn a small portion of data to achieve good image recognition results.

[0058] In this invention, the inverted dot product attention routing mechanism updates the pose of higher-level capsule layers through lower-level capsule layers. This reduces the overfitting phenomenon caused by excessive training parameters in the product design sketch evaluation model when the capsule network identifies the dependency relationships of features in an image, i.e., the part-to-whole relationship. Therefore, this invention applies the inverted dot product attention routing mechanism to capsule networks, significantly improving the accuracy of image recognition and the performance of the product design sketch evaluation model, as well as the feasibility of the capsule network-based product design sketch evaluation method for sketch evaluation. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of a product design sketch evaluation method based on capsule networks according to the present invention;

[0061] Figure 2 This is a flowchart of the feature extraction module of the present invention;

[0062] Figure 3 This is a flowchart of the ConvNeXt Block operation in the extraction module of this invention;

[0063] Figure 4 This is a flowchart of the training process for the product design sketch evaluation model of the present invention;

[0064] Figure 5 This is a schematic diagram of the training structure of the product design sketch evaluation model of the present invention;

[0065] Figure 6 These are table lamp design sketches collected from specific examples of this invention;

[0066] Figure 7 This is a schematic diagram of the structure of an electronic device according to the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the protection scope of the present invention. Example

[0068] like Figure 1 As shown, this embodiment provides a product design sketch evaluation method based on capsule networks, the specific steps of which include:

[0069] S100. Collect sketches and input the 448×448 pixel sketches into the product design sketch evaluation model;

[0070] S200. Input the target semantics of the product design into the product design sketch evaluation model; the product design sketch evaluation model includes a feature extraction module and a capsule network module configured with an inverted dot product attention routing mechanism;

[0071] In this context, different sketches contain different semantic meanings related to form. For example, product design sketches may contain semantics such as simple-complex, popular-individualistic, elegant-refined, geometric-streamlined, and business-artistic. The target semantic meaning refers to the specific semantic meaning that the form in the sketch is expected to conform to.

[0072] S300, The feature extraction module performs feature extraction on the sketch with a pixel size of 448×448 to obtain a feature vector with a size of 14×14×768;

[0073] S400. The feature vector with a size of 14×14×768 is input into the capsule network module, and the capsule network module outputs the fit between the sketch and the target semantics.

[0074] Specifically, the degree of fit between the sketch and the target semantics derived from the product design sketch evaluation model is represented by a score of 0 to 100.

[0075] This invention uses the Python programming language and PyTorch as the deep learning framework, deploying the program to run on a GPU. PyTorch is an open-source Python machine learning library that provides features including tensor computation with strong GPU acceleration, and deep neural networks built on a tape-based auto-tuning system.

[0076] One of the improved solutions is as follows:

[0077] like Figure 2 As shown, in step S300, the feature extraction module performs feature extraction as follows:

[0078] S301. After downsampling the 448×448 pixel sketch with a convolution kernel of 4×4 and a stride of 4, normalize it to obtain a feature vector with a size of 112×112×3.

[0079] S302. Perform the first ConvNeXt Block stacking on the feature vector with size 112×112×3. The depth of the first ConvNeXt Block stacking is 96 and the number of stacking times is 3, to obtain a feature vector with size 112×112×3.

[0080] S303. After downsampling the feature vector with size 112×112×3, perform a second ConvNeXtBlock stacking. The depth of the second ConvNeXt Block stacking is 192 and the stacking number is 3, resulting in a feature vector with size 56×56×192.

[0081] The downsampling operation involves first normalizing the feature vector, and then performing a convolution with a kernel of 2×2 and a stride of 2.

[0082] S304. After downsampling the feature vector with size 56×56×192, perform a third ConvNeXtBlock stacking. The depth of the third ConvNeXt Block stacking is 384 and the number of stacking times is 27, resulting in a feature vector with size 28×28×384.

[0083] S305. After downsampling the feature vector with size 28×28×384, perform a fourth ConvNeXtBlock stacking. The depth of the fourth ConvNeXt Block stacking is 768 and the stacking number is 3, resulting in a feature vector with size 14×14×768.

[0084] like Figure 3 As shown, the specific process of each ConvNeXt Block stacking includes:

[0085] Step 1: After passing the feature vector through a convolutional layer with a kernel size of 7×7, a stride of 1, and padding of 3, normalize it.

[0086] The second step is to pass the normalized feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, and then activate it through an activation function, increasing the depth to 4 times the depth in the ConvNeXt Block stack.

[0087] Step 3: After passing the activated feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, the depth is restored by sequentially passing the Layer Scale function and DropPath operation, and the regularized feature vector is output.

[0088] Specifically, the feature extraction module uses the ConvNeXt network. Overall, the ConvNeXt network changes the stacking number of convolutional and pooling layers in the ResNet network (a traditional convolutional neural network), employing depthwise convolution with a one-to-one correspondence between the number of channels and the number of convolutions, thereby increasing the number of channels in the ConvNeXt network. The number of channels is adjusted from 64 in the ResNet network to 96, and each convolutional module is designed as an inverse bottleneck structure, wide in the middle and narrow at both ends, to avoid information loss caused by channel shrinkage. The size of the convolutional kernel is increased from the traditional 3×3 to 7×7. To use fewer activation functions, the GeLU function (Gaussian Error Linear Unit) is used in this invention. To use fewer normalization layers, all normalization operations use Layer Normalization (Layer Normalization) to reduce the error that may be caused by normalization operations when there is little sample data.

[0089] The Layer Scale function scales or shifts feature vectors, typically performed after Layer Norm normalization to improve the stability of feature vectors.

[0090] The DropPath operation randomly deactivates the multi-branch structure in the ConvNeXt Block stack to prevent overfitting.

[0091] like Figure 5 As shown, the product design sketch evaluation method based on capsule networks according to claim 1 is characterized in that: in step S400, the capsule network module is provided with primary capsule layers from bottom to top. l 1. Convolutional capsule layer l 2. Convolutional capsule layer l 3. Fully connected capsule layer l 4 and fully connected capsule layers l 5;

[0092] The feature vector is input to the primary capsule layer. l In 1, the primary capsule layer l 1. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l Update of the capsule pose in 2, the convolutional capsule layer l 2. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l The capsule pose is updated in step 3 until the fully connected capsule layer. l 4. The fully connected capsule layer is processed using an algorithm based on the inverted dot product attention routing mechanism. l Update to the capsule pose in 5;

[0093] The fully connected capsule layer l Each capsule in 5 corresponds to a category of the semantic tag, and the fit between each capsule and the corresponding semantic tag is calculated using the logits of the softmax function.

[0094] From the primary capsule layer l 1 to the fully connected capsule layer l 5. The algorithm steps of the inverted dot product attention routing mechanism between adjacent capsule layers include:

[0095] S401, Computational Capsule and capsules Consistency probability between The consistency probability The calculation formula is:

[0096] ;

[0097] in, l This represents the layer index of the capsule layer, and l 1≤ l ≤ l 5; Indicates capsule layer l Chinese capsules The i-th capsule of the matrix, Indicates capsule layer l +1 capsule The j-th capsule of the matrix, Indicates capsule layer l Chinese capsules The i-th to j-th capsules of the matrix, The capsule With the capsule Connection weights;

[0098] S402, Obtain the capsule Posture update protocol The posture update protocol The calculation formula is:

[0099] in, Indicates matrix transpose;

[0100] S403, the capsule The posture is updated via the posture update protocol. Update; the capsule The formula for calculating the pose update is:

[0101] ;

[0102] ;

[0103] in, ∑z represents the exponential operation with the base e of the natural logarithm; ∑z represents the summation symbol starting from the z-th capsule; The capsule and the capsules The probability of the inverted dot product attention routing mechanism between them; This represents the normalization algorithm.

[0104] like Figure 4 and Figure 5 As shown, the training process of the product design sketch evaluation model includes:

[0105] Step 1: Collect sketches and preprocess them to construct a sketch image dataset with a pixel size of 448×448;

[0106] Step 2: Perform contour drawing, image segmentation, and contour extraction on the sketch image dataset, and perform semantic annotation to obtain the annotated training dataset;

[0107] Step 3: Input the training dataset into the feature extraction module to extract features and obtain a feature vector of size 14×14×768;

[0108] Step 4: Input the feature vector with a size of 14×14×768 into the capsule network module for training, and obtain the trained product design sketch evaluation model through the algorithm of the inverted dot product attention routing mechanism.

[0109] In step one, the preprocessing of the sketch includes denoising, image enhancement, scaling, and / or cropping.

[0110] In step two, contour lines are drawn and extracted from a portion of the sketches in the training dataset; the remaining sketches in the training dataset are automatically segmented and their contours extracted through a learning network.

[0111] Specifically, the outline of a portion of the sketch is first drawn and extracted using Photoshop software. Then, the Deeplabv3+ network proposed by Google is used to automatically perform image segmentation and outline extraction on the remaining sketch.

[0112] The invention will now be further illustrated with specific examples:

[0113] Taking a desk lamp design sketch as an example, the product design sketch evaluation model is used to evaluate the desk lamp design sketch; the specific steps include:

[0114] S100, such as Figure 6 As shown, six table lamp design sketches A1, A2, A3, B1, B2 and B3 were collected, and the six table lamp design sketches with a pixel size of 448×448 were input into the product design sketch evaluation model;

[0115] S200. Input the target semantics of the product design into the product design sketch evaluation model;

[0116] Among them, the target semantics of A1 is generous, the target semantics of A2 is gorgeous, the target semantics of A3 is exquisite, the target semantics of B1 is popular, the target semantics of B2 is simple, and the target semantics of B1 is minimalist.

[0117] S300, The feature extraction module extracts features from the table lamp design sketch to obtain a feature vector;

[0118] S400. The feature vector is input into the capsule network module, and the capsule network module obtains the fit between the lamp design sketch and the target semantics through the algorithm of the inverted dot product attention routing mechanism.

[0119] Table 1. Alignment between the table lamp design sketch and the target semantics

[0120] Table lamp design sketch Expert rating Model scoring error(%) A1 78.7 66.9 14.99 A2 84.0 89.8 -6.90 A3 86.7 76.2 12.11 B1 69.6 50.2 27.87 B2 85.3 89.7 -5.16 B3 90.3 94.9 -5.09

[0121] As shown in Table 1, by comparing the semantic evaluation of the six lamp design sketches by product design experts with the semantic evaluation of the six lamp design sketches by the product design sketch evaluation model, it can be concluded that the error between the product design sketch evaluation model's score and the expert's score is mostly less than 15%. This indicates that the product design sketch evaluation model can provide some assistance in judging the fit between the product design sketch and the target semantics, thus proving the feasibility of the capsule network-based product design sketch evaluation method provided by this invention in sketch evaluation.

[0122] Example 2:

[0123] This embodiment provides an electronic device, including:

[0124] At least one memory for storing programs;

[0125] At least one processor for executing the program stored in the memory;

[0126] Input devices are used to input programs into memory;

[0127] An output device for displaying the results of the processor executing the program stored in the memory;

[0128] When the program stored in the memory is executed, the processor is used to execute a product design sketch evaluation method based on capsule networks as described in Embodiment 1.

[0129] Specifically, the input device can be a keyboard, mouse, camera, etc., and in this embodiment, the input device is a keyboard; the output device can be an audio device, monitor, or printer, and in this embodiment, the input device is a monitor.

[0130] The memory, also known as RAM, is the storage space that the processor can directly address. The memory includes Random Access Memory (RAM), Read Only Memory (ROM), Cache, and External Storage.

[0131] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The central processing unit includes a control unit (CU) and an arithmetic logic unit (ALU); the control unit is the command system of the device, performing command functions for the electronic equipment; the arithmetic logic unit performs arithmetic and logical operations.

[0132] like Figure 7 As shown, the relationship between the memory, the processor, the input device, and the output device is as follows:

[0133] The input device inputs a program, which is stored in the memory. The processor retrieves the program from the memory and executes it. The output device displays the result of the processor executing the program.

[0134] Example 3:

[0135] This embodiment provides a computer-readable storage medium storing computer instructions that enable the computer to implement a product design sketch evaluation method based on capsule networks as described in Embodiment 1.

[0136] Specifically, when the computer instructions are executed on a computer, all or part of the product design sketch evaluation method based on capsule networks as described in Embodiment 1 above is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, or microwave). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive), etc.

[0137] This invention is not limited to the above-described optional embodiments. Anyone can derive other various forms of products under the guidance of this invention. However, regardless of any changes made in their shape or structure, any technical solution that falls within the scope of the claims of this invention shall be protected by this invention.

Claims

1. A product design sketch evaluation method based on capsule networks, characterized in that, The specific steps include: S100. Collect sketches and input the 448×448 pixel sketches into the product design sketch evaluation model; S200. Input the target semantics of the product design into the product design sketch evaluation model; the product design sketch evaluation model includes a feature extraction module and a capsule network module configured with an inverted dot product attention routing mechanism; S300, The feature extraction module performs feature extraction on the sketch with a pixel size of 448×448 to obtain a feature vector with a size of 14×14×768; S400. The feature vector with a size of 14×14×768 is input into the capsule network module, and the capsule network module outputs the fit between the sketch and the target semantics. The capsule network module has primary capsule layers arranged from bottom to top. l 1. Convolutional capsule layer l 2. Convolutional capsule layer l 3. Fully connected capsule layer l 4 and fully connected capsule layers l 5; The feature vector is input to the primary capsule layer. l In 1, the primary capsule layer l 1. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l Update of the capsule pose in 2, the convolutional capsule layer l 2. An algorithm using the inverted dot product attention routing mechanism is applied to the convolutional capsule layer. l The capsule pose is updated in step 3 until the fully connected capsule layer. l 4. The fully connected capsule layer is processed using an algorithm based on the inverted dot product attention routing mechanism. l Update to the capsule pose in 5; The fully connected capsule layer l Each capsule in 5 corresponds to a category of the semantic tag, and the fit between each capsule and the corresponding semantic tag is calculated using the logits of the softmax function.

2. The product design sketch evaluation method based on capsule networks according to claim 1, characterized in that, In step S300, the feature extraction module performs feature extraction as follows: S301. After downsampling the 448×448 pixel sketch with a convolution kernel of 4×4 and a stride of 4, normalize it to obtain a feature vector with a size of 112×112×3. S302. Perform the first ConvNeXt Block stacking on the feature vector with size 112×112×3. The depth of the first ConvNeXt Block stacking is 96 and the number of stacking times is 3, to obtain a feature vector with size 112×112×3. S303. After downsampling the feature vector with size 112×112×3, perform a second ConvNeXtBlock stacking. The depth of the second ConvNeXt Block stacking is 192 and the stacking number is 3, resulting in a feature vector with size 56×56×192. The downsampling operation involves first normalizing the feature vector, and then performing a convolution with a kernel of 2×2 and a stride of 2. S304. After downsampling the feature vector with size 56×56×192, perform a third ConvNeXtBlock stacking. The depth of the third ConvNeXt Block stacking is 384 and the number of stacking times is 27, resulting in a feature vector with size 28×28×384. S305. After downsampling the feature vector with size 28×28×384, perform a fourth ConvNeXtBlock stacking. The depth of the fourth ConvNeXt Block stacking is 768 and the stacking number is 3, resulting in a feature vector with size 14×14×768.

3. The product design sketch evaluation method based on capsule networks according to claim 2, characterized in that, The specific process of each ConvNeXt Block stacking includes: Step 1: After passing the feature vector through a convolutional layer with a kernel size of 7×7, a stride of 1, and padding of 3, normalize it. The second step is to pass the normalized feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, and then activate it through an activation function, increasing the depth to 4 times the depth in the ConvNeXt Block stack. Step 3: After passing the activated feature vector through a convolutional layer with a kernel of 1×1 and a stride of 1, the depth is restored by sequentially passing the Layer Scale function and DropPath operation, and the regularized feature vector is output.

4. The product design sketch evaluation method based on capsule networks according to claim 3, characterized in that: From the primary capsule layer l 1 to the fully connected capsule layer l 5. The algorithm steps of the inverted dot product attention routing mechanism between adjacent capsule layers include: S401, Computational Capsule and capsules Consistency probability between The consistency probability The calculation formula is: ； in, l This represents the layer index of the capsule layer, and l 1≤ l ≤ l 5; Indicates capsule layer l Chinese capsules The i-th capsule of the matrix, Indicates capsule layer l +1 capsule The j-th capsule of the matrix, Indicates capsule layer l Chinese capsules The i-th to j-th capsules of the matrix, The capsule With the capsule Connection weights; S402, Obtain the capsule Posture update protocol The posture update protocol The calculation formula is: in, Indicates matrix transpose; S403, the capsule The posture is updated via the posture update protocol. Update; the capsule The formula for calculating the pose update is: ； ； in, ∑z represents the exponential operation with the base e of the natural logarithm; ∑z represents the summation symbol starting from the z-th capsule; The capsule and the capsules The probability of the inverted dot product attention routing mechanism between them; This represents the normalization algorithm.

5. The product design sketch evaluation method based on capsule networks according to claim 1, characterized in that, The training process of the product design sketch evaluation model includes: Step 1: Collect sketches and preprocess them to construct a sketch image dataset with a pixel size of 448×448; Step 2: Perform contour drawing, image segmentation, and contour extraction on the sketch image dataset, and perform semantic annotation to obtain the annotated training dataset; Step 3: Input the training dataset into the feature extraction module to extract features and obtain a feature vector of size 14×14×768; Step 4: Input the feature vector with a size of 14×14×768 into the capsule network module for training, and obtain the trained product design sketch evaluation model through the algorithm of the inverted dot product attention routing mechanism.

6. The product design sketch evaluation method based on capsule networks according to claim 5, characterized in that: In step one, the preprocessing of the sketch includes denoising, image enhancement, scaling, and / or cropping.

7. The product design sketch evaluation method based on capsule networks according to claim 5, characterized in that: In step two, contour lines are drawn and extracted from a portion of the sketches in the training dataset; the remaining sketches in the training dataset are automatically segmented and their contours extracted through a learning network.

8. An electronic device, characterized in that, include: At least one memory for storing programs; At least one processor for executing the program stored in the memory; Input devices are used to input programs into memory; An output device for displaying the results of the processor executing the program stored in the memory; When the program stored in the memory is executed, the processor is used to execute the product design sketch evaluation method based on capsule networks as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions that enable the computer to implement the product design sketch evaluation method based on capsule networks as described in any one of claims 1-7.

Images