Shape classification method based on multi-dimensional and multi-channel convolutional neural network

Abstract

The present invention relates to a shape classification method based on a multidimensional multi-channel convolutional neural network. The purpose of the present invention is to solve the problem of low shape classification accuracy in existing methods when the image contour is incomplete due to image noise and the lack of a small sample data set. The process is as follows: 1. Calculate the contour angle and contour unit curvature, and perform contour angle encoding and contour unit curvature encoding based on the contour angle and contour unit curvature; calculate descriptors at different scale levels based on the contour angle encoding and contour unit curvature encoding; 2. Establish a training set and a test set; 3. Establish a multidimensional multi-channel network model; 4. Input the training set and the test set into the multidimensional multi-channel network model to obtain a trained multidimensional multi-channel network model; 5. Input the image to be tested into the trained multidimensional multi-channel network model to complete image classification. The present invention is used in the field of image classification.

Description

Technical Field

[0001] The present invention relates to a shape classification method. Background Art

[0002] Humans can recognize objects based on their shapes. Compared with traditional textures, patterns, and colors, shape is considered to be one of the best features for object recognition and classification. In image processing and computer vision, computer recognition and classification of shapes is a core and challenging problem. It mainly obtains shape contour features to achieve the purpose of identifying objects. In traditional two-dimensional shape recognition, traditional single closed shapes are used as the main research object, and all contour information can be extracted. However, in real life, most images exist in the form of multiple contours, and the contour information may be partially cropped or occluded. In image processing, there is also the problem of partial distortion of the image contour due to image noise. For example Figure 2a is the original image, 2b is the contour image, and 2c is the contour image of the missing part.

[0003] Researchers have been studying shape recognition and classification for over 20 years. During this period, several classic classification algorithms have emerged, but these algorithms have limitations in classifying partial contours. The IDSC+DP descriptor offers high accuracy but is extremely slow. Its accuracy is high when classifying complete contours, but it drops significantly when contours are missing. Running the entire process on the MPEG dataset presented in this study takes over 18 hours, a significant time-consuming task. SC+DP creates a shape histogram for each sample point and calculates the spatial distribution of the remaining points relative to a reference point. When contours are missing, the incompleteness of the sample points significantly degrades classification performance. AP&BAP (angle mode and binary angle mode) uses contour points to form a sequence of multiple angles, then applies binary encoding to the angle sequence to generate feature values. These features are used to determine differences between images, and the KNN algorithm is used for classification. However, images with randomly missing contours introduce significant noise, resulting in poor classification performance. CBW reflects contour features by studying leaf shapes and integrating shape features into chords. This paper proposes a new Hausdorff matching method, but this matching method is not applicable to multi-contour datasets because it is not applicable to multi-contour datasets when the inside-outside distance ratio of the shape is large.

[0004] In convolutional neural network classification, a lack of datasets is also a major factor contributing to low classification accuracy. In shape classification, researchers have used a linear SVM classifier for human detection as an example to study the problem of human recognition based on a feature set. This algorithm presupposes the ability to capture all human features, and classification requires a large training set. With a small dataset like the one in this paper, and limited methods for dataset expansion, classification is extremely difficult. Researchers later used the Sketch-a-Net network model to recognize and classify hand-drawn sketch shapes. This model achieved high accuracy, but it only works well when the dataset is sufficient and the contour information is fully captured. Accuracy drops significantly when this information is incomplete. Researchers proposed a network model, ShapeNet, that expands the dataset by rotating and flipping the image. When the dataset is small, rotation and flipping only expand the dataset by a factor of 72, which is too small for the dataset in this paper, and the classification accuracy is low. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem of low shape classification accuracy of existing methods when the data set is small and the image contour is incomplete due to image noise, and to propose a shape classification method based on a multi-dimensional and multi-channel convolutional neural network.

[0006] The specific process of the shape classification method based on multi-dimensional multi-channel convolutional neural network is as follows:

[0007] Step 1: Calculate the contour angle and contour unit curvature, and perform contour angle encoding and contour unit curvature encoding based on the contour angle and contour unit curvature; and calculate descriptors at different scale levels based on the contour angle encoding and contour unit curvature encoding;

[0008] Step 2: Create training set and test set;

[0009] Step 3: Establish a multi-dimensional and multi-channel network model;

[0010] Step 4: Input the training set and the test set into the multi-dimensional multi-channel network model to obtain a trained multi-dimensional multi-channel network model;

[0011] Step 5: Input the image to be tested into the trained multi-dimensional multi-channel network model to complete image classification.

[0012] The beneficial effects of the present invention are:

[0013] The present invention contributes to solving the problem of partial contour shape recognition and classification in three main aspects.

[0014] First, the present invention proposes a new shape descriptor unit contour angle sequence and unit contour curvature sequence, which converts traditional two-dimensional image information into one-dimensional information through encoding at multiple scale levels, thereby achieving the expansion from global information to local information, and the encoded descriptor has translation, scaling, rotation and mirror invariance.

[0015] Secondly, the present invention adopts a cyclic point dropping method to expand the data set when the current data set is insufficient, so that shape classification can be performed even when the data set is insufficient.

[0016] Third, the present invention proposes a new network model; it introduces a multi-scale encoding of sampling point angle sequences and contour unit curvature from global information to local information, each encoding forming a one-dimensional signal. Each set of one-dimensional signals is independently equipped with a one-dimensional convolutional neural network. Network parameters such as the size and number of convolution kernels within the convolutional network are set according to the specific circumstances of the encoding method. For these two encoding methods, two convolution results are generated, which are sequentially expanded and combined into a one-dimensional feature, which is then fed into a fully connected layer for final classification. The use of two channels can improve the classification accuracy of partial contour shapes.

[0017] This paper combines traditional shape descriptors with deep learning to form a new shape classification method, which has been proven to be effective in multiple data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flowchart of the present invention;

[0019] Figure 2a is the original picture;

[0020] Figure 2b is the contour map of the original image;

[0021] Figure 2c It is the contour image after losing a contour segment;

[0022] Figure 3a It is a single contour map;

[0023] Figure 3b is the sampling point map of a single contour;

[0024] Figure 4a It is the sampling point diagram when the scale level r = 0;

[0025] Figure 4b It is the sampling point diagram when the scale level r = 1;

[0026] Figure 4c This is the sampling point diagram when the scale level r = 2;

[0027] Figure 4dThis is the sampling point diagram when the scale level r = 3;

[0028] Figure 4e This is the sampling point diagram when the scale level r = 4;

[0029] Figure 4f This is a graph showing that 10% of the contour points are lost starting from point 1 at scale level 0;

[0030] Figure 4g This is a graph showing that 10% of the contour points are lost starting from the 100th point at scale level 0;

[0031] Figure 4h This is a graph showing that 10% of the contour points are lost starting from the 200th point at scale level 0;

[0032] Figure 5 This is a structural diagram of a multi-dimensional multi-channel network model;

[0033] Figure 6 For each of the 13 categories of the LOGO dataset

[0034] Figure 7 for Figure 6 All images in the first category of the 13 categories;

[0035] Figure 8a Display graph for contour fragments;

[0036] Figure 8b for Figure 8a A display diagram showing a missing segment in the middle contour segment;

[0037] Figure 8c for Figure 8a A display diagram showing a missing segment in the middle contour segment;

[0038] Figure 8d for Figure 8a A display diagram showing a missing segment in the middle contour segment;

[0039] Figure 8e for Figure 8a A display diagram showing a missing segment in the middle contour segment;

[0040] Figure 9 One image for each of the first 18 categories in the MPEG dataset;

[0041] Figure 10 for Figure 9 All pictures in the first category of MPEG;

[0042] Figure 11 The graphs are of the same class in the two datasets. DETAILED DESCRIPTION

[0043] Specific embodiment 1: The specific process of the shape classification method based on the multi-dimensional multi-channel convolutional neural network in this embodiment is as follows:

[0044] The present invention contributes to solving the problem of partial contour shape recognition and classification in three main aspects.

[0045] First, the present invention proposes a new shape descriptor unit contour angle sequence and unit contour curvature sequence, which converts traditional two-dimensional image information into one-dimensional information through encoding at multiple scale levels, thereby achieving the expansion from global information to local information, and the encoded descriptor has translation, scaling, rotation and mirror invariance.

[0046] Secondly, in response to the insufficiency of the current data set, the present invention adopts a cyclic point dropping method to expand the data set, so that perfect shape classification can be performed even when the data set is insufficient.

[0047] Third, the present invention proposes a new network model; the present invention introduces a multi-scale level sampling point angle sequence encoding and contour unit curvature encoding from global information to local information, each encoding forming a one-dimensional signal. Each set of one-dimensional signals is independently equipped with a one-dimensional convolutional neural network, and the network parameters such as the size and number of convolution kernels within the convolutional network are set according to the specific circumstances of the encoding method. For these two encoding methods, two convolution results will be generated, which will be sequentially expanded and combined into a one-dimensional feature, and then enter the fully connected layer for final classification.

[0048] Step 1: Calculate the contour angle and contour unit curvature, and perform contour angle encoding and contour unit curvature encoding based on the contour angle and contour unit curvature; and calculate descriptors at different scale levels based on the contour angle encoding and contour unit curvature encoding;

[0049] Step 2: Create training set and test set;

[0050] Step 3: Establish a multi-dimensional and multi-channel network model;

[0051] Step 4: Input the training set and the test set into the multi-dimensional multi-channel network model to obtain a trained multi-dimensional multi-channel network model;

[0052] Step 5: Input the image to be tested into the trained multi-dimensional multi-channel network model to complete image classification.

[0053] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that, in step 1, the unit contour angle and the contour unit curvature are calculated, and the unit contour angle encoding and the contour unit curvature encoding are performed based on the contour angle and the contour unit curvature; based on the unit contour angle encoding and the unit contour curvature encoding, descriptors at different scale levels are calculated; the specific process is as follows:

[0054] Angle sequence coding is a lossy encoding method, but it can greatly extract the shape information of open contours. Furthermore, angle sequence is a one-dimensional encoding method that can be reconstructed, and the reconstruction effect is also good. The main reason for using angle sequence coding is that it maximizes the preservation of contour shape information and is invariant to translation, scaling, rotation, and mirroring, making it highly robust. Contour unit curvature coding complements contour unit angle coding and can obtain more comprehensive and advanced shape information.

[0055] Given an RGB image, process the given RGB image into a binary grayscale image;

[0056] A binary grayscale image can be composed of a series of contour lines, and by selecting a sufficient number of sampling points, the shape can be described as closely as possible;

[0057] Therefore, a binary image can be described by a series of sampling points Ω = {p i =(x i ,y i ),i=1,2,3,...,N},

[0058] Where Ω is a binary image, p i is the i-th sampling point, x i ,y i is the coordinate corresponding to the sampling point, and N is the total number of sampling points;

[0059] Since the sampling contour is a closed curve, p N+i =p i ;

[0060] The sampling points are sampled in a clockwise direction; Figure 3a It is a single contour map; Figure 3b This is the picture after taking the points;

[0061] Take three points p on the contour line i+1 =(x i+1 ,y i+1 ), p i =(x i ,y i ), p i-1 =(x i-1 ,y i-1 ), calculate with p i =(x i ,y i ) point is the corner vertex, and the two adjacent points p i+1 =(x i+1 ,y i+1 ) and p i-1 =(x i-1 ,y i-1), i.e., the angle of the unit contour formed by the three points in the calculation curve;

[0062] The calculation formula is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, a i For p i+1 With p i The length between two points, b i For p i With p i-1 The length between two points, c i For p i+1 With p i-1 The length between two points, θ i is the unit contour angle formed by three points within the curve;

[0068] Based on formulas (1), (2), (3), and (4), an angle sequence {θ i};

[0069] Calculate the p i =(x i ,y i ) point is the center point, and the two adjacent points p i+1 =(x i+1 ,y i+1 ) and p i-1 =(x i-1 ,y i-1 ) is the sum of the lengths formed by the three points in the curve, that is, the curvature of the unit contour formed by the three points in the curve is calculated; the formula is:

[0070] l i =a i +b i (5)

[0071] Among them, l i is the curvature of the contour of a single unit;

[0072] A good descriptor should satisfy the invariance of translation, scaling, rotation and mirroring; therefore, the formula is used to i Normalize

[0073]

[0074] in, is the result after normalizing the curvature of the contour of a single unit, min{l s} is the minimum value of the unit contour curvature formed by all points, max{l s} is the maximum value of the unit contour curvature formed by all points;

[0075] After formula (6), the interference of the descriptor caused by the change of the number of sampling points and the change of scale level can be reduced.

[0076] From the above, the unit contour angle code and the unit contour curvature code can be defined as:

[0077]

[0078] Multi-scaling is a method of smoothing each sampling point through adjacent points using median filtering (the present invention has a total of 5 scale levels). Multi-scaling can more effectively and completely expand the shape information of the contour from local information to global information.

[0079] Take the contour sampling points {p1,p2,...,p N}, let each scale level be r, and the coordinates of the contour sampling points at different scale levels are:

[0080]

[0081] in, is the coordinate of the corresponding sampling point at scale level r;

[0082] Figure 4a 、 4b ,4c, 4d, and 4e are the pictures showing the five scale levels, r = 0, 1, 2, 3, and 4, respectively;

[0083] When r = 0, it is the original point on the contour; after the above calculation, the coordinates of the contour points at different scale levels are formed;

[0084] The descriptors at different scale levels can be calculated from the coordinates of the contour points at different scale levels:

[0085]

[0086] in, is the unit contour angle at scale level r; is the unit contour curvature at scale level r, It is the unit contour angle encoding and unit contour curvature encoding at scale level r.

[0087] Other steps and parameters are the same as those in the first embodiment.

[0088] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the training set and the test set are established in step two; the specific process is as follows:

[0089] In convolutional neural network shape classification, obtaining a sufficient training set using a relatively small number of image samples is a challenge. Currently, individual shape datasets typically contain no more than 3,000 images, which is far too small for deep learning. To address this shortfall, a method using cyclic point dropping to generate local contours is proposed to generate training sets.

[0090] Get a dataset, use 70% of the dataset as a training set and 30% as a test set;

[0091] The total number of sampling points of the contour in a single binary image in the training set is N, where a small number of continuous contour points are discarded in each cycle, and the remaining majority of contour points are taken as the entire contour;

[0092] The discarded small portion of continuous contour points accounts for 5%, 10% and 15% of the total number N;

[0093] The number of points in most of the remaining contours is a constant value of 256;

[0094] In each cycle, a small number of continuous contour points are discarded, and the remaining majority of contour points are used as the entire contour; the specific process is:

[0095] In a single binary image in the training set, the first contour sampling point is taken as the starting point, 5% of the contour sampling points are continuously discarded, and the remaining 95% of the contour sampling points are used as all contour sampling points; the first contour sampling point is taken as the starting point, 10% of the contour sampling points are continuously discarded, and the remaining 90% of the contour sampling points are used as all contour sampling points; the first contour sampling point is taken as the starting point, 15% of the contour sampling points are continuously discarded, and the remaining 85% of the contour sampling points are used as all contour sampling points; and so on, in a single binary image, the Nth contour sampling point is taken as the starting point, 5% of the contour sampling points are continuously discarded, and the remaining 95% of the contour sampling points are used as all contour sampling points; the Nth contour sampling point is taken as the starting point, 10% of the contour sampling points are continuously discarded, and the remaining 90% of the contour sampling points are used as all contour sampling points; the Nth contour sampling point is taken as the starting point, 15% of the contour sampling points are continuously discarded, and the remaining 85% of the contour sampling points are used as all contour sampling points;

[0096] This significantly increases the training set and ensures robustness and generalization when test set segments are lost (in this experiment, the number of remaining points after losing segment contour points was kept constant at 256). The number of lost segments in this experiment accounted for 5%, 10%, and 15% of the total; excessive lost segments can easily cause shape distortion.

[0097] In order to adapt to the input data size of the network model to 5×1×256 (5 dimensions, 1 channel, 256 data), a single binary image must have a constant value of 256 remaining contour points when losing different percentages of contour sampling points j. Therefore, the total number of contour sampling points N in the corresponding single binary image is j Satisfies the following formula:

[0098] N j =256 / g j ,(j=5%,10%,15%,g j =1-j) (10)

[0099] where g j is the percentage of the remaining contour sampling points to the total number when the percentage of different contour sampling points is lost; where N j is the total number of contour sampling points in a single binary image when the percentage of missing contour sampling points is j;

[0100] New contour points formed by a single binary image after losing points at different scales as follows:

[0101]

[0102] in, is the coordinate of the corresponding sampling point at scale level r;

[0103] Calculate the unit contour angle code and unit contour curvature code for each new contour point at different scale levels for:

[0104]

[0105] in, is the unit contour angle at scale level r; is the unit contour curvature at scale level r, is the unit contour angle code and unit contour curvature code at scale level r;

[0106] It can be seen from the coordinates of the contour sampling points formed at the above scale level that the coordinate sequence of the contour sampling points of a single binary image can be formed new contour sampling point coordinate sequences, so a single binary image dataset is expanded times;

[0107] Each image dataset is expanded by multiple binary images in the training set times to form an expanded training set.

[0108] Figure 4fThis is a graph showing that 10% of the contour points are lost starting from point 1 at scale level 0. Figure 4g This is a graph showing that 10% of the contour points are lost starting from the 100th point at scale level 0. Figure 4h This figure shows a graph at scale level 0, where 10% of contour points are lost starting at point 200. The test set is used to verify the feasibility of the algorithm when contours are lost. This experiment randomly drops some continuous contour segments. The amount of lost segments varies across datasets. In the multi-contour LOGO dataset, contour segments are dropped, while in the single-contour dataset, points are dropped as contour points (the specific method is described in the dataset experiment below).

[0109] Other steps and parameters are the same as those in the first or second embodiment.

[0110] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that a multi-dimensional multi-channel network model is established in step 3; the specific process is as follows:

[0111] For multi-channel shape classification, a multi-dimensional multi-channel network model is proposed, and its network structure model is shown in the figure Figure 5 shown.

[0112] There are two channels during input, namely the unit contour angle channel and the unit contour curvature channel;

[0113] The same points in the element profile angle channel and the element profile curvature channel:

[0114] 1. The maximum pooling kernel size is 1×2 and the stride is 2.

[0115] 2. Batch normalization is performed after each convolution layer to prevent drift in deep networks.

[0116] 3. The activation function of all layers is ReLu function.

[0117] The multi-dimensional multi-channel network model includes an input layer, a unit contour angle channel, a unit contour curvature channel, a first fully connected layer, a second fully connected layer, a third fully connected layer, a fourth fully connected layer, and an output layer in sequence;

[0118] The input layer is connected to the unit contour angle channel and the unit contour curvature channel respectively. The outputs of the unit contour angle channel and the unit contour curvature channel are merged and input into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer in sequence. The output result of the fourth fully connected layer is output through the output layer.

[0119] The other steps and parameters are the same as those in the first to third embodiments.

[0120] Specific embodiment five: This embodiment is different from any one of specific embodiments one to four in that the unit contour angle channel includes, in sequence, a first convolutional layer, a first batch normalization layer, a first maximum pooling layer, a second convolutional layer, a second batch normalization layer, a second maximum pooling layer, a third convolutional layer, a third batch normalization layer, a third maximum pooling layer, a fourth convolutional layer, a fourth batch normalization layer, and a fourth maximum pooling layer;

[0121] The unit contour curvature channel sequentially includes a fifth convolutional layer, a fifth batch normalization layer, a fifth maximum pooling layer, a sixth convolutional layer, a sixth batch normalization layer, a sixth maximum pooling layer, a seventh convolutional layer, a seventh batch normalization layer, a seventh maximum pooling layer, an eighth convolutional layer, an eighth batch normalization layer, and an eighth maximum pooling layer;

[0122] The outputs of the fourth maximum pooling layer and the eighth maximum pooling layer are merged and input into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer in sequence, and the output result of the fourth fully connected layer is output through the output layer.

[0123] The other steps and parameters are the same as those in the first to fourth embodiments.

[0124] Specific embodiment six: This embodiment differs from any one of specific embodiments one to five in that the maximum pooling kernel sizes of the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer, and the fourth maximum pooling layer are all 1×2, with a step size of 2.

[0125] The other steps and parameters are the same as those in the first to fifth embodiments.

[0126] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that in step four, the training set and the test set are input into the multi-dimensional multi-channel network model to obtain a trained multi-dimensional multi-channel network model; the specific process is as follows:

[0127] In the Element Profile Angle channel:

[0128] The 5×1×256 (5-dimensional, 1-channel, 256 data) one-dimensional contour angle encoding in the training set is input into the first convolutional layer to obtain a 6×1×250 convolution result. After the first batch normalization layer and the first maximum pooling layer, a 6×1×125 pooled result is formed;

[0129] The number of convolution kernels in the first convolution layer is 6, and the convolution kernel size is 1×7;

[0130] The pooled result of the first maximum pooling layer is input into the second convolutional layer to obtain a 30×1×119 convolution result. After the second batch normalization layer and the second maximum pooling layer, a 30×1×59 pooled result is formed.

[0131] The number of convolution kernels in the second convolution layer is 30, and the convolution kernel size is 1×7;

[0132] The first and second convolutions are used to obtain more global features;

[0133] The pooled result of the second maximum pooling layer is input into the third convolutional layer to obtain a convolution result of 60×1×57. After the third batch normalization layer and the third maximum pooling layer, a pooled result of 60×1×28 is formed.

[0134] The number of convolution kernels in the third convolution layer is 60, and the convolution kernel size is 1×3;

[0135] The pooled result of the third maximum pooling layer is input into the fourth convolutional layer to obtain a convolution result of 180×1×26. After the fourth batch normalization layer and the fourth maximum pooling layer, a pooled result of 180×1×13 is formed.

[0136] The number of convolution kernels in the fourth convolution layer is 180, and the convolution kernel size is 1×3;

[0137] Expand the pooled result of 180×1×13 (the pooled result of 180×1×13 is the feature matrix) to obtain the one-dimensional unit contour angle;

[0138] In the Element Profile Curvature channel:

[0139] The one-dimensional contour curvature code of 5×1×256 (5 dimensions, 1 channel, 256 data) in the training set is input into the fifth convolutional layer to obtain a 10×1×248 convolution result. After the fifth batch normalization layer and the fifth maximum pooling layer, a 10×1×124 pooled result is formed;

[0140] The number of convolution kernels in the fifth convolutional layer is 10, and the convolution kernel size is 1×9;

[0141] The pooled result of the fifth maximum pooling layer is input into the sixth convolutional layer to obtain a convolution result of 24×1×118. After the sixth batch normalization layer and the sixth maximum pooling layer, a pooled result of 24×1×59 is formed.

[0142] The number of convolution kernels in the sixth largest convolution layer is 24, and the convolution kernel size is 1×7;

[0143] The first and second convolutions are used to obtain more high-level local features;

[0144] The pooled result of the sixth maximum pooling layer is input into the seventh convolutional layer to obtain a convolution result of 100×1×55. After the seventh batch normalization layer and the seventh maximum pooling layer, a pooled result of 100×1×27 is formed.

[0145] The number of convolution kernels in the seventh convolution layer is 100, and the convolution kernel size is 1×5;

[0146] The pooled result of the seventh maximum pooling layer is input into the eighth convolutional layer to obtain a convolution result of 240×1×25. After the eighth batch normalization layer and the eighth maximum pooling layer, a pooled result of 240×1×12 is formed.

[0147] The number of convolution kernels in the eighth convolutional layer is 240, and the convolution kernel size is 1×3;

[0148] Expand the pooled result of 240×1×12 (the pooled result of 240×1×12 is the feature matrix) to obtain the one-dimensional unit contour curvature;

[0149] Merge the one-dimensional unit contour angle and one-dimensional unit contour curvature into a one-dimensional feature with a size of 5520, and input it into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer in sequence. After passing through four fully connected layers, classification is performed and finally N is output, where N is the actual number of classifications to be performed.

[0150] Get a pre-trained multi-dimensional multi-channel network model;

[0151] The test set is input into the pre-trained multi-dimensional multi-channel network model. If the requirements are met, a trained multi-dimensional multi-channel network model is obtained. If the requirements are not met, the training set is used to continue training the multi-dimensional multi-channel network model until the requirements are met, and a trained multi-dimensional multi-channel network model is obtained.

[0152] The other steps and parameters are the same as those in the first to sixth embodiments.

[0153] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that the optimizer of the multi-dimensional multi-channel network model is an Adam optimizer.

[0154] Select Adam optimization for the optimizer. This optimizer combines the advantages of the AdaGrad and RMSProp optimization algorithms. As one of the best default optimizers, it has the following advantages: simple implementation, efficient calculation, low memory requirements, and strong adaptability to noise caused by fragments in multi-channel unit contour angles and unit contour curvatures.

[0155] The other steps and parameters are the same as those in the first to seventh embodiments.

[0156] The following examples are used to verify the beneficial effects of the present invention:

[0157] Example 1:

[0158] To evaluate the effectiveness of this method, we conducted experiments on three datasets: the multi-contour LOGO dataset, the MPEG-7 dataset, and a dataset consisting of both LOGO and MPEG-7. This experiment was conducted in MATLAB 2020 and PyTorch, powered by an R7-5800H processor, a 3060 graphics card, and 32GB of RAM. For all datasets, 70% of the images were used as training sets, and 30% as test sets.

[0159] 1) Experimental results on the LOGO dataset

[0160] In the LOGO dataset, there are 13 categories in the dataset, with a total of 486 images. The number of images in each category is more than 30, and the number of images in each category is: 53, 51, 50, 48, 44, 38, 37, 35, 34, 33, 32, 31, 30.

[0161] like Figure 6 Representative images of each of the 13 categories in the logo dataset are shown. Figure 7 for Figure 6 53 images of the first category of LOGOs shown;

[0162] The vast majority of logo images are multi-contour images, making this dataset quite challenging. The test set for this experiment utilizes a multi-contour segment loss method. Using the Canny algorithm, after extracting the contours, all the contour segments of the image are lost one at a time to create the test set.

[0163] The training set and test set of the data set are input into the network model for classification to obtain the classification accuracy;

[0164] Figure 8a Display graph for contour fragments; Figure 8b for Figure 8a A display diagram showing a missing segment in the middle contour segment; Figure 8c for Figure 8a A display diagram showing a missing segment in the middle contour segment; Figure 8d for Figure 8a A display diagram showing a missing segment in the middle contour segment; Figure 8e for Figure 8a A diagram showing a contour segment missing a segment; the experimental result is 0.6864.

[0165] II) Experimental results on MPEG-7 dataset

[0166] The MPEG-7 dataset is a widely used dataset for shape evaluation. It contains 70 categories, each with 20 images, for a total of 1,400 images. Because this dataset is a single-contour dataset, the test set is constructed by dropping consecutive points from the contour. Similar to the training set, the test set uses a cyclic point dropping method, maintaining a constant number of 256 points after dropping. Ten images are evenly selected from the complete contour at each loss of 4%, 8%, 12%, 16%, and 20%, respectively, for the test set. This allows 51 datasets to be generated from one image, for a total of 21,420 test images.

[0167] The training set and test set of the data set are input into the network model for classification to obtain the classification accuracy;

[0168] Figure 9 For the first picture in each of the first 18 categories in part of MPEG-7, Figure 10 for Figure 9 Schematic diagram of 20 images in the first category; its experimental result is 0.8521:

[0169] III) Experimental results after merging the LOGO dataset and the MPEG-7 dataset

[0170] There are two categories of images in the 13 categories of LOGO images and MPEG-7 images. Therefore, the two categories are merged into the same category. After merging the same categories, there are a total of 81 categories of images, totaling 1886 images. The same category of images are apple images and rooster images. The first and third are images in the LOGO dataset, and the second and fourth are MPEG-7 images. Figure 11 shown.

[0171] In this experiment, the test set selects and discards fragments based on the characteristics of the image. The LOGO dataset is still discarded in the form of fragments, and MPEG-7 is still discarded using the second method.

[0172] The training set and test set of the data set are input into the network model for classification to obtain the classification accuracy; the experimental result accuracy is 0.8282.

[0173] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A shape classification method based on a multi-dimensional multi-channel convolutional neural network, characterized by: The method includes: Step 1: Calculate the contour angle and contour unit curvature, and perform contour angle encoding and contour unit curvature encoding based on the contour angle and contour unit curvature; and calculate descriptors at different scale levels based on the contour angle encoding and contour unit curvature encoding; Step 2: Create training set and test set; Step 3: Establish a multi-dimensional and multi-channel network model; Step 4: Input the training set and the test set into the multi-dimensional multi-channel network model to obtain a trained multi-dimensional multi-channel network model; Step 5: Input the image to be tested into the trained multi-dimensional multi-channel network model to complete image classification; In the step 1, the unit contour angle and the contour unit curvature are calculated, and the unit contour angle encoding and the contour unit curvature encoding are performed based on the contour angle and the contour unit curvature; based on the unit contour angle encoding and the unit contour curvature encoding, descriptors at different scale levels are calculated; the specific process is: Given an RGB image, process the given RGB image into a binary grayscale image; A binary grayscale image consists of a series of contour lines; Therefore, a binary image is described by a series of sampling points Ω={p i =(x i ,y i ),i=1,2,3,...,N}, Where Ω is a binary image, p i is the i-th sampling point, x i ,y i is the coordinate corresponding to the sampling point, and N is the total number of sampling points; Since the sampling contour is a closed curve, p N+i =p i ; The sampling points are sampled in a clockwise direction; Take three points p on the contour line i+1 =(x i+1 ,y i+1 ), p i =(x i ,y i ), p i-1 =(x i-1 ,y i-1 ), calculate with p i =(x i ,y i ) point is the corner vertex, and the two adjacent points p i+1 =(x i+1 ,y i+1 ) and p i-1 =(x i-1 ,y i-1 ), i.e., the angle of the unit contour formed by the three points in the calculation curve; The calculation formula is as follows: Among them, a i For p i+1 With p i The length between two points, b i For p i With p i-1 The length between two points, c i For p i+1 With p i-1 The length between two points, θ i is the unit contour angle formed by three points within the curve; Based on formulas (1), (2), (3), and (4), an angle sequence {θ i }; Calculate the p i =(x i ,y i ) point is the center point, and the two adjacent points p i+1 =(x i+1 ,y i+1 ) and p i-1 =(x i-1 ,y i-1 ) is the sum of the lengths formed by the three points in the curve, that is, the curvature of the unit contour formed by the three points in the curve is calculated; the formula is: l i =a i +b i (5) Among them, l i is the contour curvature of a single unit; Right i Normalize in, is the result after normalizing the curvature of the contour of a single unit, min{l s } is the minimum value of the unit contour curvature formed by all points, max{l s } is the maximum value of the unit contour curvature formed by all points; The unit contour angle coding and unit contour curvature coding are defined as follows: Take the contour sampling points {p1,p2,...,p N }, let each scale level be r, and the coordinates of the contour sampling points at different scale levels are: in, is the coordinate of the corresponding sampling point at scale level r; The descriptors at different scale levels are calculated from the coordinates of the contour points at different scale levels: in, is the unit contour angle at scale level r; is the unit contour curvature at scale level r, It is the unit contour angle encoding and unit contour curvature encoding at scale level r.

2. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 1, characterized in that: In step 2, a training set and a test set are established; the specific process is as follows: Get a dataset, use 70% of the dataset as a training set and 30% as a test set; The total number of sampling points of the contour in a single binary image in the training set is N, where a small number of continuous contour points are discarded in each cycle, and the remaining majority of contour points are taken as the entire contour; The discarded small portion of continuous contour points accounts for 5%, 10% and 15% of the total number N; The number of points in most of the remaining contours is a constant value of 256; In each cycle, a small number of continuous contour points are discarded, and the remaining majority of contour points are used as the entire contour; the specific process is: In a single binary image in the training set, the first contour sampling point is taken as the starting point, 5% of the contour sampling points are continuously discarded, and the remaining 95% of the contour sampling points are used as all contour sampling points; the first contour sampling point is taken as the starting point, 10% of the contour sampling points are continuously discarded, and the remaining 90% of the contour sampling points are used as all contour sampling points; the first contour sampling point is taken as the starting point, 15% of the contour sampling points are continuously discarded, and the remaining 85% of the contour sampling points are used as all contour sampling points; and so on, in a single binary image, the Nth contour sampling point is taken as the starting point, 5% of the contour sampling points are continuously discarded, and the remaining 95% of the contour sampling points are used as all contour sampling points; the Nth contour sampling point is taken as the starting point, 10% of the contour sampling points are continuously discarded, and the remaining 90% of the contour sampling points are used as all contour sampling points; the Nth contour sampling point is taken as the starting point, 15% of the contour sampling points are continuously discarded, and the remaining 85% of the contour sampling points are used as all contour sampling points; In order to adapt to the size of the network model input data 5×1×256, when a single binary image loses different percentages of contour sampling points j, the remaining majority of the contour points must be a constant value of 256, so the total number of contour sampling points in the corresponding single binary image N j Satisfies the following formula: N j =256 / g j ,(j=5%,10%,15%,g j =1-j) (10) where g j is the percentage of the remaining contour sampling points to the total number when the percentage of different contour sampling points is lost; where N j is the total number of contour sampling points in a single binary image when the percentage of missing contour sampling points is j; New contour points formed by a single binary image after losing points at different scales as follows: in, is the coordinate of the corresponding sampling point at scale level r; Calculate the unit contour angle code and unit contour curvature code for each new contour point at different scale levels for: in, is the unit contour angle at scale level r; is the unit contour curvature at scale level r, is the unit contour angle code and unit contour curvature code at scale level r; It can be seen from the coordinates of the contour sampling points formed at the above scale level that the coordinate sequence of the contour sampling points of a single binary image can be formed new contour sampling point coordinate sequences, so a single binary image dataset is expanded times; Each image dataset is expanded by multiple binary images in the training set times to form an expanded training set.

3. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 2, characterized in that: In step 3, a multi-dimensional multi-channel network model is established; the specific process is: The multi-dimensional multi-channel network model includes an input layer, a unit contour angle channel, a unit contour curvature channel, a first fully connected layer, a second fully connected layer, a third fully connected layer, a fourth fully connected layer, and an output layer in sequence; The input layer is connected to the unit contour angle channel and the unit contour curvature channel respectively. The outputs of the unit contour angle channel and the unit contour curvature channel are merged and input into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer in sequence. The output result of the fourth fully connected layer is output through the output layer.

4. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 3, characterized in that: The unit contour angle channel includes, in sequence, a first convolutional layer, a first batch normalization layer, a first maximum pooling layer, a second convolutional layer, a second batch normalization layer, a second maximum pooling layer, a third convolutional layer, a third batch normalization layer, a third maximum pooling layer, a fourth convolutional layer, a fourth batch normalization layer, and a fourth maximum pooling layer; The unit contour curvature channel sequentially includes a fifth convolutional layer, a fifth batch normalization layer, a fifth maximum pooling layer, a sixth convolutional layer, a sixth batch normalization layer, a sixth maximum pooling layer, a seventh convolutional layer, a seventh batch normalization layer, a seventh maximum pooling layer, an eighth convolutional layer, an eighth batch normalization layer, and an eighth maximum pooling layer; The outputs of the fourth maximum pooling layer and the eighth maximum pooling layer are merged and input into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer in sequence, and the output result of the fourth fully connected layer is output through the output layer.

5. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 4, characterized in that: The maximum pooling kernel sizes of the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer, and the fourth maximum pooling layer are all 1×2, with a step size of 2.

6. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 5, characterized in that: In the step 4, the training set and the test set are input into the multi-dimensional multi-channel network model to obtain a trained multi-dimensional multi-channel network model; the specific process is: In the Element Profile Angle channel: The 5×1×256 one-dimensional contour angle code in the training set is input into the first convolutional layer to obtain a 6×1×250 convolution result. After the first batch normalization layer and the first maximum pooling layer, a 6×1×125 pooled result is formed. The number of convolution kernels in the first convolution layer is 6, and the convolution kernel size is 1×7; The pooled result of the first maximum pooling layer is input into the second convolutional layer to obtain a 30×1×119 convolution result. After the second batch normalization layer and the second maximum pooling layer, a 30×1×59 pooled result is formed. The number of convolution kernels in the second convolution layer is 30, and the convolution kernel size is 1×7; The pooled result of the second maximum pooling layer is input into the third convolutional layer to obtain a convolution result of 60×1×57. After the third batch normalization layer and the third maximum pooling layer, a pooled result of 60×1×28 is formed. The number of convolution kernels in the third convolution layer is 60, and the convolution kernel size is 1×3; The pooled result of the third maximum pooling layer is input into the fourth convolutional layer to obtain a convolution result of 180×1×26. After the fourth batch normalization layer and the fourth maximum pooling layer, a pooled result of 180×1×13 is formed; The number of convolution kernels in the fourth convolution layer is 180, and the convolution kernel size is 1×3; Expand the pooled result of 180×1×13 to obtain the one-dimensional unit contour angle; In the Element Profile Curvature channel: The 5×1×256 one-dimensional contour curvature code in the training set is input into the fifth convolutional layer to obtain a 10×1×248 convolution result. After the fifth batch normalization layer and the fifth maximum pooling layer, a 10×1×124 pooled result is formed; The number of convolution kernels in the fifth convolutional layer is 10, and the convolution kernel size is 1×9; The pooled result of the fifth maximum pooling layer is input into the sixth convolutional layer to obtain a convolution result of 24×1×118. After the sixth batch normalization layer and the sixth maximum pooling layer, a pooled result of 24×1×59 is formed. The number of convolution kernels in the sixth convolution layer is 24, and the convolution kernel size is 1×7; The pooled result of the sixth maximum pooling layer is input into the seventh convolutional layer to obtain a convolution result of 100×1×55. After the seventh batch normalization layer and the seventh maximum pooling layer, a pooled result of 100×1×27 is formed. The number of convolution kernels in the seventh convolution layer is 100, and the convolution kernel size is 1×5; The pooled result of the seventh maximum pooling layer is input into the eighth convolutional layer to obtain a convolution result of 240×1×25. After the eighth batch normalization layer and the eighth maximum pooling layer, a pooled result of 240×1×12 is formed. The number of convolution kernels in the eighth convolutional layer is 240, and the convolution kernel size is 1×3; Expand the pooled result of 240×1×12 to obtain the one-dimensional unit contour curvature; The one-dimensional unit contour angle and one-dimensional unit contour curvature are merged into a one-dimensional feature with a size of 5520, which is sequentially input into the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer. After passing through the four fully connected layers, classification is performed and N is finally output; Get a pre-trained multi-dimensional multi-channel network model; The test set is input into the pre-trained multi-dimensional multi-channel network model. If the requirements are met, a trained multi-dimensional multi-channel network model is obtained. If the requirements are not met, the training set is used to continue training the multi-dimensional multi-channel network model until the requirements are met, and a trained multi-dimensional multi-channel network model is obtained.

7. The shape classification method based on a multi-dimensional multi-channel convolutional neural network according to claim 6, characterized in that: The optimizer of the multi-dimensional multi-channel network model is the Adam optimizer.

Images