Convolutional neural network structure for explainable 3D shape learning, and method and system for 3D shape learning utilizing the same.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- INDUSTRY UNIVERSITY COOPERATION FOUNDATION HANYANG UNIVERSITY
- Filing Date
- 2023-08-03
- Publication Date
- 2026-08-03
Smart Images
Figure 0007898784000109 
Figure 0007898784000110 
Figure 0007898784000111
Abstract
Description
[Technical Field]
[0001] The following explanation concerns 3D shape learning technology. [Background technology]
[0002] Three-dimensional shape analysis using deep learning is mainly comprised of multi-view and voxel methods, but recently there has been growing interest in models using graph and mesh methods.
[0003] Multiview and voxel-based models transform irregular 3D shapes into regular representations for processing by neural networks. While such methods can perform general convolution operations, the loss of topological connectivity and reduction in data density that occur during the representation transformation make them sensitive to noise, leading to unstable results and high computational complexity.
[0004] Graph-based methods perform convolution operations in non-Euclidean space using geodesic-based features such as paths between graph nodes and edges. Mesh-based methods, on the other hand, perform convolution operations tailored to 3D shapes using geometric features such as the position and orientation of nodes and faces. These methods have limitations in handling all types of 3D shapes because they empirically use low-level features to process specific 3D shape datasets.
[0005] Furthermore, deep learning cannot explain the reasoning behind its inferences due to its internal nonlinearity. To address this, explainable artificial intelligence (eXplainable Artificial Intelligence) is being actively researched. However, while much research has been done on model explanation techniques for 2D images, general methods cannot be applied to 3D shapes, and only specialized techniques for models and data are being researched little by little. [Overview of the project] [Problems that the invention aims to solve]
[0006] This invention provides a method and system for extracting geodetic and geometric features from 3D shape data and classifying objects by performing convolution operations on the extracted geodetic and geometric features based on the surfaces that constitute the 3D shape data.
[0007] This provides a convolutional-based neural network model for explainable 3D shape learning that perceives geodetic and geometric features of 3D shape data while maintaining spatial information. [Means for solving the problem]
[0008] A method for shape learning performed by a shape learning system may include the steps of extracting geodetic and geometric features from three-dimensional shape data, and classifying objects by convolution operations on the extracted geodetic and geometric features based on the surfaces that make up the three-dimensional shape data.
[0009] The shape learning method may further include a step of visualizing the results of classifying objects from 3D shape data, using a convolution-based neural network model for explainable 3D shape learning.
[0010] A convolution-based neural network model for explainable 3D shape learning may consist of a descriptor layer that extracts geodetic and geometric features from 3D shape data using each target surface and adjacent surfaces, and a convolution layer that performs convolution operations using the extracted geodetic and geometric features.
[0011] The 3D shape data may consist of nodes, edges, and faces that make up the 3D shape data, with an irregular structure.
[0012] The extraction step may include a step of generating a data structure in which, for each face constituting the 3D shape data, identification information for multiple adjacent faces is constructed in list form, and information about the nodes and edges of each face is obtained based on the identification information constructed in list form.
[0013] The extraction stage may include a step of extracting geodetic features from 3D shape data using order-invariant kernel mapping.
[0014] The extraction step may include extracting geometric features from 3D shape data based on the position and orientation of the target surface and surrounding surfaces.
[0015] The extraction step may include extracting geometric features from 3D shape data based on the distance ratio and difference angle information between the target surface and surrounding surfaces.
[0016] The classification step may include inputting feature vectors for each surface, which integrate geodetic and geometric features, into a convolutional layer.
[0017] The classification stage may include a step of performing a one-dimensional convolution using a feature vector that inherits each face based on the feature vector for each face, and a feature vector that aggregates the relationships between a particular face and its adjacent faces.
[0018] The extraction step may include a step in which a Global Average Pooling (GAP) operation is performed to obtain a score for each class by outputting the result such that the number of faces is maintained through the performed one-dimensional convolution, and then classifying the objects based on the obtained scores.
[0019] The system may include a computer program, which is recorded on a non-temporary computer-readable recording medium, to cause the shape learning system described above to perform a method for shape learning.
[0020] The shape learning system may include a feature extraction unit that extracts geodesic features and geometric features from three-dimensional shape data, and an object classification unit that classifies an object by performing a convolution operation on the extracted geodesic features and the extracted geometric features based on the surfaces constituting the three-dimensional shape data.
[0021] The shape learning system may visualize the result of classifying an object from three-dimensional shape data by using a convolution-based neural network model for explainable three-dimensional shape learning.
Advantages of the Invention
[0022] By extracting geodesic features and geometric features from three-dimensional shape data and performing a convolution operation on the extracted geodesic features and the extracted geometric features based on the surfaces constituting the three-dimensional shape data, the object classification performance can be improved.
[0023] By using a convolution-based neural network model for explainable three-dimensional shape learning, the inference result can be visualized without deforming the model.
Brief Description of the Drawings
[0024] [Figure 1] It is a block diagram for explaining the configuration of a shape learning system in one embodiment. [Figure 2] It is a flowchart for explaining a method for shape learning in one embodiment. [Figure 3] It is a diagram for explaining the operation of an explainable model in one embodiment. [Figure 4] It is a diagram for explaining the geodesic path between a target surface and an adjacent surface. [Figure 5] It is a diagram for explaining the geometric relationship between a target surface and an adjacent surface in one embodiment. [Figure 6] It is a diagram for explaining the structure of the convolution layer of an explainable model in one embodiment. [Figure 7] This figure illustrates an example of the results of an explainable model in one embodiment. [Figure 8] This figure illustrates an example of the results of an explainable model in one embodiment. [Figure 9] This figure illustrates an example of the results of an explainable model in one embodiment. [Figure 10] This figure illustrates an example of the results of an explainable model in one embodiment. [Modes for carrying out the invention]
[0025] The embodiments will be described in detail below with reference to the attached drawings.
[0026] Figure 1 is a block diagram illustrating the configuration of a shape learning system in one embodiment, and Figure 2 is a flowchart illustrating a method for shape learning in one embodiment.
[0027] The processor of the shape learning system 100 may include a feature extraction unit 110 and an object classification unit 120. The components of such a processor may be representations of different functions that are executed by the processor according to control instructions provided by program code recorded in the shape learning system. The processor and its components may control the shape learning system to perform steps 210 to 220 included in the method for shape learning in Figure 2. In this case, the processor and its components may be implemented to execute instructions from operating system code contained in memory and code from at least one program.
[0028] The processor may load program code recorded in a program file for a method of shape learning into memory. For example, when a program is executed in the shape learning system, the processor may control the shape learning system to load program code from the program file into memory, in accordance with the control of the operating system. At this time, the feature extraction unit 110 and the object classification unit 120 may be different functional representations of the processor for executing the instructions of the corresponding parts of the program code loaded into memory to perform the subsequent stages 210 to 220.
[0029] In step 210, the feature extraction unit 110 may extract geodetic and geometric features from the 3D shape data. In graph theory, geodetic information represents the distance between two nodes as the shortest path connecting the two points. Adjacent faces in 3D shape data (hereinafter referred to as "adjacent faces") have the same nodes and edges, and therefore have the same geodetic information. When data is represented in a vector space, nodes and edges may be represented as vectors and calculations may be performed. Geodetic convolution is a calculation method designed to take advantage of the fact that nodes and edges are represented as vectors and calculations are performed, but unstable results may occur depending on the order of nodes and edges. For this reason, the feature extraction unit 110 may use order-invariant kernel mapping to extract geodetic features from the 3D shape data. The feature extraction unit 110 can extract geodetic features without loss of spatial information by calculating the node and edge information of the target face and surrounding faces regardless of the order. Furthermore, the feature extraction unit 110 may extract geometric features from the 3D shape data based on the position and orientation of the target surface and the surrounding surface. By performing calculations in any order based on the distance ratio and difference angle information between the target surface and the surrounding surface, the feature extraction unit 110 can obtain robust results that are independent of the data size.
[0030] In step 220, the object classification unit 120 may classify objects by performing convolution operations on extracted geodetic and geometric features based on the faces that constitute the 3D shape data. The object classification unit 120 may integrate information on the target face and adjacent faces by performing face-by-face operations. The object classification unit 120 may use one-dimensional convolution operations to apply model explanation techniques used in 2D image classification problems.
[0031] Figure 3 is a diagram illustrating the operation of an explainable model in one embodiment.
[0032] Three-dimensional shape data is actively used in computer graphics. However, due to the irregular nature of the data, it is difficult to train neural networks on it. Techniques that convert three-dimensional shape data into simpler structures for processing make it difficult to explain neural network inference due to the loss of spatial information. In this embodiment, the aim is to efficiently process the data through an explainable model (ExMeshCNN)300 that senses the geodesic and geometric properties of the three-dimensional shape data while maintaining spatial information, and to visualize the explanation.
[0033] The explainable model 300 refers to a convolution-based neural network model for explainable 3D shape learning. The explainable model 300 may consist of a descriptor layer 310 and a convolutional layer 320. The descriptor layer 310 may include a geodetic descriptor 311 that extracts geodetic features from 3D shape data using each symmetric and adjacent face, and a geometric descriptor 312 that extracts geometric features from 3D shape data using each symmetric and adjacent face. The convolutional layer 320 may perform a convolution operation using the extracted geodetic and geometric features.
[0034] The first layer of the explainable model 300 may be equipped with learnable edge-based geodetic descriptors 311 and face-based geometric descriptors 312 to obtain higher-level features from faces realized by a 1D-CNN operation. Next, the convolutional layer 320, which performs convolution on each face, follows the descriptor layer 310 and aims to learn regional features within adjacent faces. Finally, a GAP operation and a softmax layer are applied so that the output is a logit. As a result, the explainable model 300 allows for model interpretability by applying the input attribute method by learning the mesh in an end-to-end manner while maintaining spatial information from input to output.
[0035] The shape learning system may classify objects from 3D shape data using an explainable model 300. However, since the 3D shape data has an irregular structure of nodes, edges, and faces, it cannot be used as input to a neural network model. Therefore, the shape learning system may perform data preprocessing. Specifically, the shape learning system may generate identification information for each face and for three adjacent faces in list form from the 3D shape data. Based on the identification information, the shape learning system may generate a data structure that allows obtaining information about the nodes and edges of each face.
[0036] Figure 4 is a diagram illustrating the geodetic path between the target surface and the adjacent surface in one embodiment.
[0037] Shape learning systems can extract geodetic and geometric features from 3D shape data and classify objects using convolutional operations based on the faces that make up the 3D shape data. Furthermore, shape learning systems are designed to easily apply common techniques used to explain 2D image classification models to 3D dimensions, thereby explaining the model's behavior.
[0038] More specifically, the descriptor layer is the first layer that attempts to mine geodetic and geometric features from the faces that make up the mesh. Here, we propose two types of learnable descriptors to capture geodetic and geometric features.
[0039] Geodetic descriptors aim to extract geodetic regional features of a surface that have sometimes been ignored by conventional mesh-based models because it is difficult to define geodetic relationships between two surfaces. Generally, the geodetic distance between two vertices is defined by the number of edges on the shortest path connecting the two vertices. Shape learning systems perform feature extraction and convolution at the surface level, so they can use geodetic distance to show relationships between surfaces. However, to simply solve the problem of applying geodetic distance between surfaces, we introduce geodetic descriptors.
[0040] Target surface Let's call it JPEG0007898784000001.jpg87. Next, To estimate the geodetic relationship between JPEG0007898784000002.jpg87 and the adjacent surface, Consider the aligned vertices and edges in JPEG0007898784000003.jpg87. The center point of JPEG0007898784000004.jpg87 Let it be named JPEG0007898784000005.jpg78. j∈{1,2,3} is one of the three vertices in JPEG0007898784000006.jpg87 Let's call it JPEG0007898784000007.jpg99. And, JPEG0007898784000008.jpg78 and The relationship with JPEG0007898784000009.jpg99 Let's call it JPEG0007898784000010.jpg924. Also, JPEG0007898784000011.jpg99 (k∈1,2) Excluding the vertices belonging to JPEG0007898784000012.jpg87, Among the adjacent faces of JPEG0007898784000013.jpg87 This vertex is geodesically connected to JPEG0007898784000014.jpg99. JPEG0007898784000015.jpg99 and This refers to the relationship with JPEG0007898784000016.jpg99. This shows JPEG0007898784000017.jpg1129. In the end, the target surface JPEG0007898784000018.jpg87 and its adjacent surfaces (i.e., The geodetic distance between JPEG0007898784000019.jpg98 is, Center point of JPEG0007898784000020.jpg87 JPEG0007898784000021.jpg78 and This represents the number of edges on the shortest path between vertices belonging to JPEG0007898784000022.jpg98.
[0041] However, one problem arises. As shown in Figure 4, The shortest path length between JPEG0007898784000023.jpg87 and its adjacent surface is 2. For example, the center point JPEG0007898784000024.jpg78 and JPEG0007898784000025.jpg910, and center point JPEG0007898784000026.jpg78 and The length of the shortest geodetic path between JPEG0007898784000027.jpg910 is 2, and is actually the same in all other cases. In conclusion, geodetic distance does not contain any differentiating and significant information as a surface feature. As an alternative to geodetic distance, geodetic convolution can be considered, which extracts features while additionally considering the curvature of the path. This can solve the above problem because even if the length is the same, it will differ depending on the curvature of the shortest path (for example, path 1 and path 2 shown in Figure 1). In geodetic convolution, vertices The normal vector of JPEG0007898784000028.jpg99 is the center point From JPEG0007898784000029.jpg78 JPEG0007898784000030.jpg99 is located midway along the path and is necessary to consider curvature. Normal vector JPEG0007898784000031.jpg917 is a vertex This can be easily obtained by averaging the normal vectors of the faces surrounding JPEG0007898784000032.jpg99.
[0042] Unfortunately, there is some ambiguity in the geodetic convolution of geodetic paths due to the fact that there is no defined order. For example, as shown in Figure 4, JPEG0007898784000033.jpg87 contains two adjacent vertices. JPEG0007898784000034.jpg910 and It has JPEG0007898784000035.jpg98, ( JPEG0007898784000036.jpg1015) and ( A path is generated for JPEG0007898784000037.jpg1015). In this case, since the order of the two paths is not defined, the result of the convolution operation involving the two paths will differ each time.
[0043] To solve these problems, instead of using geodetic distance and geodetic convolution, We propose the following descriptor to map JPEG0007898784000038.jpg65 to a feature space.
[0044]
number
[0045] Figure 5 is a diagram illustrating the geometric relationship between a target surface and an adjacent surface in one embodiment.
[0046] Commonly used geometric features for mesh data surfaces include surface normal vectors, centers, and edge angles. However, these are somewhat heuristic and low-level features; therefore, we propose a learnable descriptor that can automatically capture higher-level representations.
[0047] The center point indicating the location of JPEG0007898784000045.jpg87 The file is JPEG0007898784000046.jpg78, and the center point of its adjacent surface is Let's call it JPEG0007898784000047.jpg912. In this case, JPEG0007898784000048.jpg78 and The Euclidean distance between JPEG0007898784000049.jpg912 may be a low-level feature. Next, Normal vector of JPEG0007898784000050.jpg87 JPEG0007898784000051.jpg88 and the normal vector of the adjacent surface To get the angle between two faces from JPEG0007898784000052.jpg99, use the cross product. JPEG0007898784000053.jpg88× JPEG0007898784000054.jpg99 may be calculated, and this may also be a low-level feature. In addition to low-level features, high-level features may be obtained using kernel mappings such as the following.
[0048]
number
[0049] Therefore, we propose the following descriptor, which is not affected by order.
[0050]
number
[0051] In summary, the first layer of the explainable model (ExMeshCNN) is each surface In JPEG0007898784000060.jpg87 and its adjacent plane, geodetic descriptors and geometric descriptors realized by a one-dimensional convolution operation ( JPEG0007898784000061.jpg1115 and It consists of JPEG0007898784000062.jpg814). The results of the two descriptors may be concatenated and input to the next convolutional layer as follows.
[0052]
number
[0053] Figure 6 is a diagram illustrating the structure of a convolutional layer in an explainable model in one embodiment.
[0054] Let N be the number of faces that make up the input mesh. After passing through the descriptor layer, the mesh is represented by face feature vectors. It may be represented by a matrix composed of JPEG0007898784000064.jpg928. Define JPEG0007898784000065.jpg939 as the dimension of the surface feature vector (or the number of input channels for the following CNN). Doing so will result in a matrix size of N × C1.
[0055] Next, a series of 1D-CNN layers may be designed in an explainable model, through which the output of the descriptor layer passes. For each layer, the input matrix consists of two types of feature vectors. JPEG0007898784000066.jpg65 and It may be extended by combinations of JPEG0007898784000067.jpg85. Specifically, JPEG0007898784000068.jpg811 and JPEG0007898784000069.jpg78 may be defined as follows:
[0056]
number
[0057] Next, a 1-D convolution specific to the above input format may be performed. Let JPEG0007898784000075.jpg942 be used as an arbitrary convolution filter. Then, activate each filter. JPEG0007898784000076.jpg914 can be calculated as follows:
[0058]
number
[0059] In this way, each of the l-th layers receives input (N × C l ) consumes size, and this input is size (2N × C) l After being extended to ), the output (2N × C) is filtered by a 1-D convolutional filter. l+1 The size is converted to n. As a result, the number of channels changes for each layer, but the width is maintained in the same way as the number of faces, N. This mechanism makes it easy to understand the prominence of each face for decision-making.
[0060] In the final CNN layer, the number of output channels must be equal to the number of classes K. Next, a score for each class may be obtained by a GAP (Global Average Pooling) operation, and finally, classification may be performed based on the scores. GAP not only helps to significantly reduce the number of parameters while maintaining accuracy, but also more easily back-projects the contribution of each aspect to the classification.
[0061] Figures 7-10 illustrate an example of a explainable model result in one embodiment.
[0062] The structure of the explainable model is considered to be a face-level convolution and a final GAP operation that preserves the number of input faces. Despite not being specialized for 3D mesh data, it can highlight remarkable representations for classification by working in conjunction with conventional visual attribute methods. Figures 7-10 show how representative visual attribute methods such as Layer-wise Relevance Propagation (LRP) and Gradient-weighted Class Activation Mapping (Grad-CAM) work in the explainable model.
[0063] First, let's explain LRP. LRP adopts a decomposition strategy for explanation. After considering the final output of the model as the relevance score, the contribution of each neuron to the relevance score is backpropagated from the output layer to the input layer. At this time, there is a constraint that the sum of the contributions of neurons belonging to each layer must be the same for all layers. In order to apply LRP, relevance JPEG0007898784000085.jpg810 may be assigned to softmax output as follows:
[0064]
number
[0065]
number
[0066]
number
[0067] Finally, in the first descriptor layer, each of the following: The relevance may be allocated to the surface-level relevance score displayed in JPEG0007898784000096.jpg97.
[0068]
number
[0069] Next, Grad-CAM will be described. While LRP attempts to redistribute the relevance scores of the output into the input space, Grad-CAM focuses on the information of the last layer. Specifically, after calculating the gradient of each class flowing into the last convolutional layer, a rough localization map indicating the important regions of the input is generated.
[0070] A k ∈R N is taken as the feature map of the k th -th channel in the last CNN layer. By doing so, the weight value of the neuron importance for each class c JPEG0007898784000100.jpg810 may be calculated as follows.
[0071] JPEG0007898784000101.jpg1538 This captures the importance of the feature map of the k-th channel for the target class c. JPEG0007898784000102.jpg1115 means the GAP operation. Note that the size of the feature map N is the same as the number of faces.
[0072] Finally, the class-discriminative localization map of size N for the target class c represented by JPEG0007898784000103.jpg108 may be calculated as follows.
Equation
[0073] According to the embodiment, ModelNet40, Manifold40, SHREC11, and Cube are used as datasets to evaluate the classification accuracy of explainable models, and COSEG is used as a dataset to evaluate segmentation performance. Manifold40 is a dataset with the 2-Manifold condition added to ModelNet40, and SHREC11 is a small dataset that tests with training data set to 10 and 16, respectively. Cube is a dataset in the form of objects thinly sliced off one face of a hexahedron.
[0074] Tables 1-4 show the performance of the model for each dataset. Tables 1 and 2 compare the classification performance with the conventional model, Table 3 compares the segmentation performance with the conventional model, and Table 4 compares the number of parameters with the conventional model.
[0075] To compare the performance of the method proposed in this embodiment with conventional methods, we compared it with graph and mesh-based models using similar methods. As the experimental results show, the method proposed in this embodiment demonstrated higher accuracy across a large number of datasets compared to other conventional methods. It also showed high accuracy in segmentation. The model proposed in this embodiment was confirmed to exhibit superior performance despite being the lightest model among those shown in the table below.
[0076] [Table 1]
[0077] [Table 2]
[0078] [Table 3]
[0079] [Table 4]
[0080] Figures 7-10 show the results of applying LRP and Grad-CAM, commonly used in explaining 2D image classification models, to an explainable model. While conventional methods involved modifying the entire model or using additional methodologies to apply explanatory techniques, in this embodiment, the model's inference results can be explained using explanatory techniques without any modifications. The red areas shown in Figures 7-10 represent key parts of the model's inference results.
[0081] Figure 7 shows the results obtained by running LRP on a trained explainable model, and Figure 8 shows the results of applying Grad-CAM to the explainable model. Figures 9 and 10 also show the results of rotating the same mesh at various angles to observe all surface protrusions displayed on the 3D mesh. We can reaffirm that the explainable model learned several class-differentiating features to make a final decision, and that LRP and Grad-CAM successfully captured the features. For example, when viewing a car from below, the tires are found to be an important feature, and when viewed from the front, the windshield is also an important feature. In the case of a bed, the frame and scattered bedding are important features, but the flat floor of the bed does not seem to be considered. In conclusion, we can confirm that the explainable model appropriately captures class-differentiating features from mesh data, and that visualization can be easily achieved by running LRP and Grad-CAM on the explainable model.
[0082] The above-described apparatus may be implemented by hardware components, software components, and / or combinations of hardware and software components. For example, the apparatus and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or various devices capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications running on the OS. The processing unit may also respond to software execution, access data, record, manipulate, process, and generate data. For convenience of understanding, it may be described as if a single processing unit is used, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.
[0083] Software may include computer programs, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or which may instruct the processing unit independently or collectively. Software and / or data may be embodied in any kind of machine, component, physical device, virtual equipment, computer recording medium, or device for interpretation based on the processing unit or for providing instructions or data to the processing unit. Software may be distributed across a network of computer systems, and may be recorded or executed in a distributed manner. Software and data may be recorded on one or more computer-readable recording media.
[0084] The methods according to the embodiment may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the medium may be specifically designed for the embodiment or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to record and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that generated by a compiler, but also high-level language code that is executed by a computer using an interpreter or the like.
[0085] As described above, embodiments have been explained based on limited embodiments and drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the described technique may be performed in a different order than described, and / or the components of the described system, structure, apparatus, circuit, etc. may be combined or assembled in a different manner than described, or opposed or replaced by other components or equivalents, and still achieve suitable results.
[0086] Therefore, even if the embodiment is different, it falls within the scope of the attached claims if it is equivalent to the claims.
Claims
1. A method for shape learning performed by a shape learning system, The stage of extracting geodetic and geometric features from 3D shape data, A step of classifying objects by convolution operations on the extracted geodetic features and extracted geometric features based on the surfaces constituting the three-dimensional shape data, and A step in which a convolution-based neural network model for explainable 3D shape learning is used to visualize the results of classifying objects from the 3D shape data. Includes, The aforementioned convolution-based neural network model for explainable 3D shape learning is: A method for shape learning, comprising a descriptor layer that extracts geodetic and geometric features from the aforementioned three-dimensional shape data using each target surface and adjacent surfaces, and a convolutional layer that performs a convolution operation using the extracted geodetic and geometric features.
2. The three-dimensional shape data is characterized in that the nodes, edges, and faces constituting the three-dimensional shape data are composed of an irregular structure. The method for learning shape according to claim 1.
3. The extraction step is as follows: This step involves generating a data structure that obtains information about the nodes and edges of each face based on the identification information configured in the list, with each face representing a face that constitutes the three-dimensional shape data, and then creating a list of identification information for multiple adjacent faces. A method for shape learning according to claim 1, including the method described in claim 1.
4. The extraction step is as follows: This stage involves extracting geodetic features from 3D shape data using Order-Invariant Kernel Mapping. A method for shape learning according to claim 1, including the method described in claim 1.
5. The extraction step is as follows: Steps to extract geometric features from the aforementioned 3D shape data based on the position and orientation of the target surface and surrounding surfaces. A method for shape learning according to claim 1, including the method described in claim 1.
6. The extraction step is as follows: The step of extracting geometric features from the aforementioned 3D shape data based on the distance ratio and difference angle information between the target surface and the surrounding surface. A method for shape learning according to claim 5, including the method described in claim 5.
7. The aforementioned classification stage is, The step of inputting feature vectors for each surface, which integrate the aforementioned geodetic and geometric features, into the convolutional layer. A method for shape learning according to claim 1, including the method described in claim 1.
8. The aforementioned classification stage is, This step involves performing a one-dimensional convolution using a feature vector that inherits each face based on the feature vector for each face, and a feature vector that aggregates the relationships between a specific face and its adjacent faces. A method for shape learning according to claim 7, including the method described in claim 7.
9. The extraction step is as follows: The result is output such that the number of faces is maintained by the one-dimensional convolution performed above, and a GAP (Global Average Pooling) operation is performed to obtain a score for each class, and object classification is performed based on the obtained scores. A method for shape learning according to claim 8, including the method described in claim 8.
10. A computer program recorded on a non-temporary computer-readable recording medium for causing the shape learning system to execute the method for shape learning described in any one of claims 1 to 9.
11. A shape learning system, A feature extraction unit that extracts geodetic and geometric features from 3D shape data, and Object classification unit classifies objects based on the surfaces constituting the three-dimensional shape data by performing convolution operations on the extracted geodetic features and extracted geometric features. Includes, The shape learning system is Using a convolution-based neural network model for explainable 3D shape learning, the results of classifying objects from the 3D shape data are visualized. The aforementioned convolution-based neural network model for explainable 3D shape learning is: A shape learning system comprising a descriptor layer that extracts geodetic and geometric features from the aforementioned three-dimensional shape data using each target surface and adjacent surfaces, and a convolutional layer that performs a convolution operation using the extracted geodetic and geometric features.