A visual design image classification method based on a resnet-50 network model

Abstract

The application relates to the technical field of image classification, and discloses a visual design image classification method based on a resnet-50 network model, which comprises the following steps: obtaining a visual design image set P and pre-processing P to obtain a set, which is divided into a construction set, a training set and a test set; performing image segmentation on all images in the construction set to obtain w sub-images; obtaining m image shape feature vectors, n image color feature vectors and p image texture feature vectors, performing vector interleaving processing, and mapping to a resnet-50 network model to form a visual design image network model; training the model, learning and optimizing model parameters through loss function calculation loss; obtaining a visual design image category; after image segmentation, the images of the test set are input into the trained model to obtain the corresponding visual design image category; the shape, color and texture feature vectors of the visual design image are obtained, vector interleaving processing is performed, and the three are fused to guarantee the accurate classification of the visual design image.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image classification, and particularly relates to a visual design image classification method based on a resnet-50 network model. BACKGROUND

[0002] Visual design includes logo design, packaging design, image design, advertisement design and interface design and the like, and is a subjective form of performance means and result for visual function of users. The image design refers to using visual design means, and making an enterprise's business philosophy, behavior concept, management feature, product packaging style, marketing criterion and strategy form a whole image through modeling of a logo and specific color and the like performance methods. The advertisement design refers to various handwork or computer drawing methods or image technology, and creative image design by using a composite method.

[0003] In the visual design process, inspiration may be drawn from existing images, and there are a large number of visual design images, and users have difficulty in quickly searching for images of a required category in the numerous images and creating on the basis of the images, so that classification of the visual design images becomes a problem to be solved. SUMMARY

[0004] The application aims to solve the problem that there are a large number of visual design images, users have difficulty in quickly searching for images of a required category in the numerous images and creating on the basis of the images, and provides a visual design image classification method based on a resnet-50 network model.

[0005] The technical scheme for solving the above technical problem is as follows:

[0006] A visual design image classification method based on a resnet-50 network model comprises the following steps:

[0007] S1, a visual design image set P is acquired, and P is preprocessed to obtain , is divided into a construction set , a training set and a test set ;

[0008] S2, image segmentation is performed on all images in the construction set , to obtain w sub-images ;

[0009] S3, image shape description mining, image color description mining and image texture description mining are performed on the w sub-images , to respectively obtain m visual design image shape feature vectors , n visual design image color feature vectors , p visual design image texture feature vectors , ;

[0010] S4, vector interleaving processing is performed on m visual design image shape feature vectors , n visual design image color feature vectors and p visual design image texture feature vectors , and the visual design image network model is formed by mapping to a resnet-50 network model;

[0011] S5, a cross-entropy loss function is constructed so that the distance between visual design images of different categories is farther;

[0012] S6, the model is trained, and the image set in S1 is input into the visual design image network model in S4, the model is trained using an SGD optimizer, and the model parameters are learned and optimized through loss function calculation in S5;

[0013] S7: obtain the visual design image category, and input the images in the test set in S1 after image segmentation into the trained model in S6 to obtain the corresponding visual design image category.

[0014] On the basis of the above technical solutions, the application can also be improved as follows.

[0015] Preferably, the specific steps of image shape description mining in S3 are as follows:

[0016] First, the sub-image is subjected to image enhancement processing and gray scale conversion, and then histogram equalization is introduced to reassign the gray scale values of the image pixels, obtain an image contour map, use a shape descriptor to quantify the shape features of the objects in the image, use boundary feature method to obtain the image shape features in the image contour map, use principal component analysis to convert the high-dimensional feature vectors in the image shape features into low-dimensional feature vectors, encode the image shape features and convert them into shape numerical vectors, remove redundant shape numerical vectors, and finally obtain the visual design image shape feature vector .

[0017] Preferably, the specific steps of image color description mining in S3 are as follows:

[0018] The color space of the sub-image is obtained, the sub-image color space into several bins, and then divides the image into several regions by using color automatic segmentation technology, indexes each region by using each color component of the quantized color space, forms a binary color index set, converts the sub-image into a color histogram, maps each color in the image to an index value in the color index set, and counts the frequency of occurrence of each color in the image to obtain a color histogram, divides the pixels of each bin of the color histogram into aggregated pixels and non-aggregated pixels, obtains a color aggregation vector by calculating the proportion and distribution of the aggregated pixels, selects the most representative features in the color aggregation vector to form a color aggregation vector set, and performs dimensionality reduction processing on the high-dimensional color aggregation vectors in the color aggregation vector set by using principal component analysis, and finally converts the color aggregation vector encoding into a color numerical vector, and the color numerical vector is the color feature vector of the visual design image .

[0019] Preferably, the specific steps of image texture description mining in S3 are as follows:

[0020] The sub-image is subjected to grayscale processing to obtain a grayscale image, the energy, contrast, entropy and inverse gap of the grayscale image are obtained by using a gray level co-occurrence matrix, and the energy, contrast, correlation, entropy and inverse gap of the grayscale image are used as texture feature vectors, and the high-dimensional feature vectors are subjected to dimensionality reduction operation by using principal component analysis, and finally the visual design image texture feature vector is obtained .

[0021] Preferably, the image enhancement processing includes the following steps:

[0022] The sub-image is processed by a spatial domain enhancement method

[0023]

[0024] wherein f(x, y) is the original image, g(x, y) is the processed image, T is a spatial domain operation on the pixels in the neighborhood of the original image (x, y) position, and the neighborhood of the point (x, y) is defined as a square or rectangular region with (x, y) as the center point.

[0025] Preferably, the histogram equalization includes the following two formulas:

[0026] The transformation function is:

[0027]

[0028] Wherein, r is the original gray value, s is the transformed gray value, pr(r) is the probability density function of the original gray value, L is the gray level number, and the value is 256;

[0029] Probability density function transformation:

[0030]

[0031] Wherein, ps(s) is the probability density function of the changed gray value, pr(r) is the probability density function of the original gray value, L is the gray level number, and the value is 256.

[0032] Preferably, the construction color aggregation vector formula is as follows:

[0033]

[0034] Wherein, CCV is the color aggregation vector, is the number of aggregated pixels of the Nth color, is the number of non-aggregated pixels of the N colors.

[0035] Preferably, the calculation formula of energy, contrast, entropy and inverse distance is as follows:

[0036] Energy:

[0037] Wherein, e and f represent the index of gray level, and P(e,f) is the element value in the gray level co-occurrence matrix;

[0038] Contrast:

[0039] Wherein, e and f represent the index of gray level, and GLGM(e,f) represents the value of the corresponding position in the gray level co-occurrence matrix;

[0040] Entropy:

[0041] Wherein, e and f represent the index of gray level, and P(e,f) is the element value in the gray level co-occurrence matrix;

[0042] Inverse distance:

[0043] Wherein, e and f represent the index of gray level, and P(e,f) is the element value in the gray level co-occurrence matrix.

[0044] Preferably, the S4 specific steps are as follows:

[0045] First, use the vector interleaving technique to generate a drawing command set, which first draws the color feature vector Second, draw the color feature vector Unoccupied position draws texture feature vector , finally draws color feature vector and color feature vector Unoccupied position draws shape feature vector , finally maps to resnet-50 network model through residual mapping formula; the residual mapping formula is as follows:

[0046]

[0047] Wherein, F (x) is initial input feature, W λ It is the linear transformation matrix of visual design image color feature vector, and lambda is the mapping weight of visual design image color feature vector, W ω It is the linear transformation matrix of visual design image texture feature vector, and omega is the mapping weight of visual design image texture feature vector, W θ It is the linear transformation matrix of visual design image shape feature vector, and theta is the mapping weight of visual design image shape feature vector.

[0048] Preferably, the cross-entropy loss function in S5 is:

[0049]

[0050] Wherein, the value of l (y b =d) is 0 or 1, and the value of 1 indicates that y b is correctly classified as d class, and the value of 0 indicates that y b is not correctly classified as d class, N indicates the number of pictures, C indicates the involved picture class, a is a random number in (2, 3) interval, Indicates the probability that the bth image is judged as d, and L is the cross-entropy loss.

[0051] Compared with the prior art, the technical scheme of the present application has the following beneficial technical effects:

[0052] 1. The cross-entropy loss used in the present application can ensure that the images of different categories have a large inter-class distance during the training process of the model, thereby improving the classification ability of the model.

[0053] 2. The shape, color and texture feature vectors of the visual design image are obtained, and vector interleaving processing is performed to fuse the three to ensure accurate classification of the visual design image, so that the priority of the shape feature vector is the highest, the priority of the texture feature vector is the second, and the priority of the color feature vector is the lowest, thereby ensuring the feature priority in the visual design image network model.

[0054] 3. The application maps three different vectors of shape, color and texture feature vectors of a visual design image to a resnet-50 network model through residual mapping, solving the problem of inconsistent scales of the three different vectors in the mapping process. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The flowchart of the application is shown. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work are within the protection scope of the application.

[0057] A visual design image classification method based on a resnet-50 network model, comprising the following steps:

[0058] S1, obtaining a visual design image set P and pre-processing P to obtain , dividing into a construction set , a training set and a test set ;

[0059] S2, performing image segmentation on all images in the construction set to obtain w sub-images ;

[0060] S3, performing image shape description mining, image color description mining and image texture description mining on the w sub-images to obtain m visual design image shape feature vectors , n visual design image color feature vectors , and p visual design image texture feature vectors , ;

[0061] S4, performing vector interleaving processing on the m visual design image shape feature vectors , the n visual design image color feature vectors and the p visual design image texture feature vectors and mapping to a resnet-50 network model to form a visual design image network model;

[0062] S5, construct a cross-entropy loss function so that the visual design image distance between different categories is farther;

[0063] S6, train the model, the image set in S1 training set is input into the visual design image network model of S4, the model is trained using the SGD optimizer, and the loss function of S5 is used to calculate the loss to learn and optimize the model parameters;

[0064] S7: obtain the visual design image category, after the image in S1 test set is segmented, it is input into the trained model of S6 to obtain the corresponding visual design image category.

[0065] The preprocessing step includes removing duplicate, invalid or irrelevant images; denoising, using filters such as mean filtering, Gaussian filtering, etc. to remove noise in the image; image enhancement, by adjusting the contrast and brightness of the image and other parameters to highlight important information in the image; normalization, scaling the image pixel value to a fixed range for subsequent processing; size adjustment, adjusting the image to a uniform size for batch processing and model input, the size adjustment parameter is set to 448x448, and the training set , training set and test set The ratio of the training set is 7:2:1.

[0066] Generally, visual design images are classified according to shape into geometric shape images and free shape images, according to color into monochrome images and color images, and according to texture into natural texture images, artificial texture images and mixed texture images, among which geometric shape images are further divided into circular, rectangular, triangular, star-shaped, etc., monochrome images are further divided into red, orange, yellow, green, cyan, blue, purple, white, black, etc., color images include random combinations of multiple monochromatic colors, and natural texture images are further divided into cloud, smoke, fog, wood grain, rock texture, etc. Therefore, the specific classification of visual design image categories is a combination of the above classifications, such as free shape white cloud texture image (as the image classification is numerous, it is difficult to list one by one, therefore, it is illustrated in the form of example).

[0067] The specific steps of image shape description mining in S3 are as follows:

[0068] First, the sub-image Image enhancement and grayscale conversion are performed, followed by histogram equalization to redistribute the grayscale values ​​of image pixels, resulting in an image contour map. Shape descriptors are used to quantify the shape features of objects in the image. Boundary feature analysis is employed to obtain the image shape features from the contour map. Principal component analysis is then used to convert high-dimensional feature vectors in the image shape features into low-dimensional feature vectors. The image shape features are encoded and converted into shape numerical vectors. Redundant shape numerical vectors are removed, ultimately yielding the visual design image shape feature vector. .

[0069] The specific steps for image color description mining in S3 are as follows:

[0070] Get sub-image The color space of the sub-image The color space is quantized into several handles, and then the image is divided into several regions using automatic color segmentation technology. Each region is indexed by each color component of the quantized color space, forming a binary color index set. The sub-images are then further segmented. The process involves converting the image to a color histogram, mapping each color to an index value in a color index set, and counting the frequency of each color in the image. The pixels in each handle of the color histogram are then divided into aggregated and non-aggregated pixels. By calculating the proportion and distribution of aggregated pixels, color aggregate vectors are obtained. The most representative features are selected from these aggregate vectors to form a color aggregate vector set. Principal component analysis is then used to reduce the dimensionality of the high-dimensional color aggregate vectors in this set. Finally, the color aggregate vectors are encoded and converted into color numerical vectors, which are the color feature vectors of the visual design image. .

[0071] The specific steps for image texture description mining in S3 are as follows:

[0072] Pair images Grayscale processing is performed to obtain a grayscale image. The energy, contrast, entropy, and inverse difference of the grayscale image are obtained using the gray-level co-occurrence matrix. These parameters are then used as texture feature vectors. Principal component analysis is used to reduce the dimensionality of these high-dimensional feature vectors, ultimately yielding the texture feature vectors of the visual design image. .

[0073] Image enhancement processing includes the following steps:

[0074] Sub-images are processed using spatial domain enhancement. Image pixels, enhanced image, its expression is as follows:

[0075]

[0076] In the formula, f(x, y) is the original image, g(x, y) is the processed image, T is a spatial domain operation on the pixels in the neighborhood of the original image (x, y) position, and the neighborhood of the point (x, y) is defined as a square or rectangular region with (x, y) as the center point.

[0077] Histogram equalization includes the following two formulas:

[0078] Transformation function:

[0079]

[0080] Wherein, r is the original gray value, s is the transformed gray value, pr(r) is the probability density function of the original gray value, L is the gray level number, and the value is 256;

[0081] Probability density function transformation:

[0082]

[0083] Wherein, ps(s) is the probability density function of the changed gray value, pr(r) is the probability density function of the original gray value, L is the gray level number, and the value is 256.

[0084] Preferably, the color aggregation vector formula is constructed as follows:

[0085]

[0086] Wherein, CCV is the color aggregation vector, is the number of aggregated pixels of the Nth color, is the number of non-aggregated pixels of the N colors.

[0087] Preferably, the calculation formulas of energy, contrast, entropy and inverse distance are as follows:

[0088] Energy:

[0089] Wherein, e and f represent the index of gray level, and P(e, f) is the element value in the gray level co-occurrence matrix;

[0090] Contrast:

[0091] Wherein, e and f represent the index of gray level, and GLGM(e, f) represents the value at the corresponding position in the gray level co-occurrence matrix;

[0092] Entropy:

[0093] Wherein, e and f represent the index of gray level, and P(e, f) is the element value in the gray level co-occurrence matrix;

[0094] Inverse gap:

[0095] Where e and f represent the indices of the gray levels, and P(e,f) is the element value in the gray-level co-occurrence matrix.

[0096] The specific steps for S4 are as follows:

[0097] First, a drawing command set is generated using vector interleaving techniques. This command set first draws color feature vectors. Secondly, in the color feature vector Draw texture feature vectors in unoccupied positions Finally, the color feature vector is drawn. and color feature vector Draw shape feature vectors in unoccupied positions Finally, the residuals are mapped to the ResNet-50 network model using the residual mapping formula, which is as follows:

[0098]

[0099] Where F(x) is the initial input feature, W λ W is the linear transformation matrix of the color feature vector of the visual design image, where λ is the mapping weight of the color feature vector of the visual design image, and W is the linear transformation matrix of the color feature vector of the visual design image. ω W is the linear transformation matrix of the visual design image texture feature vector, where ω is the mapping weight of the visual design image texture feature vector, and W is the linear transformation matrix. θ It is the linear transformation matrix of the shape feature vector of the visual design image, and θ is the mapping weight of the shape feature vector of the visual design image.

[0100] It should be noted that this invention focuses on visual design of image shape feature vectors. Next is the visual design image texture feature vector. Finally, there is the visual design image color feature vector. Therefore, under normal circumstances, θ > ω > λ, and θ is generally taken as... ω is generally taken as λ is generally taken as

[0101] The cross-entropy loss function in S5 is:

[0102]

[0103] Where, l(y) b =d) takes the value of 0 or 1. A value of 1 indicates that when y b Correctly classified into class d, with a value of 0 representing y. bNot correctly classified as d, N denotes the number of pictures, C denotes the picture class involved, a is a random number in the interval (2, 3), Pbdenotes the probability that the b-th image is classified as d, L is the cross-entropy loss.

[0104] It is to be understood that the phraseology or terminology such as "first" and "second", etc., used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0105] While the embodiments of the present application have been illustrated and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the spirit and scope of the application, which is defined by the appended claims and their equivalents.

Claims

1. A visual design image classification method based on a resnet-50 network model, characterized in that, The method comprises the following steps: S1, acquire a visual design image set P and pre-process P to obtain , divide P into a training set Ptrain and a test set Ptest . , divide Ptrain into a construction set Pcons and a validation set Pval . ​ S2, image segmentation is performed on all images in the construction set to obtain w sub-images ; S3, to w sub-images Carrying out image shape description mining, image color description mining and image texture description mining, respectively obtaining m visual design image shape feature vectors , n visual design image color feature vectors , and p visual design image texture feature vectors , ; S4, vector interleaving processing m visual design image shape feature vectors , n visual design image color feature vectors and p visual design image texture feature vectors and mapping to a resnet-50 network model to form a visual design image network model; S5, constructing a cross-entropy loss function to make the visual design image distance between different categories further apart; S6, training model, training set in S1 image set , input to S4 visual design image network model, using SGD optimizer to train the model, through S5 loss function to calculate the loss of the model parameters learning optimization; S7: Obtain the visual design image category, input the image in the test set in S1 after image segmentation into the model trained in S6 to obtain the corresponding visual design image category; S7: Obtain the visual design image category, input the image in the test set in S1 after image segmentation into the model trained in S6 to obtain the corresponding visual design image category; The specific steps of S4 are as follows: First, a vector interleaving technique is used to generate a rendering command set that first renders the color feature vector , second renders the texture feature vector in the unoccupied positions , and finally renders the shape feature vector and color feature vector in the unoccupied positions , and finally maps to the resnet-50 network model through a residual mapping formula; the residual mapping formula is as follows: ； wherein F(x) is the initial input feature, W λ is the visual design image color feature vector linear transformation matrix, λ is the visual design image color feature vector mapping weight, W ω is the visual design image texture feature vector linear transformation matrix, ω is the visual design image texture feature vector mapping weight, W θ is the visual design image shape feature vector linear transformation matrix, θ is the visual design image shape feature vector mapping weight.

2. The visual design image classification method based on the resnet-50 network model according to claim 1, characterized in that: The specific steps of image shape description mining in S3 are as follows: Firstly, the sub-image is subjected to image enhancement processing and gray scale conversion, and then histogram equalization is introduced to re-allocate the gray scale values of the image pixels to obtain an image contour map. A shape descriptor is used to quantify the shape features of the objects in the image. The boundary feature method is used to obtain the image shape features in the image contour map. Principal component analysis is used to convert the high-dimensional feature vectors in the image shape features into low-dimensional feature vectors. The image shape features are encoded and converted into shape numerical vectors. Redundant shape numerical vectors are removed, and finally the visual design image shape feature vector is obtained. ​ 3. The visual design image classification method based on the resnet-50 network model according to claim 1, characterized in that: The specific steps of image color description mining in S3 are as follows: Acquiring a sub-image of the color space, quantizing the color space of the sub-image into several bins, then using a color automatic segmentation technique to divide the image into several regions, using each region to index each color component of the quantized color space to form a binary color index set, then converting the sub-image into a color histogram, mapping each color in the image to a certain index value in the color index set and counting the frequency of each color in the image to obtain a color histogram, dividing the pixels of each bin of the color histogram into aggregated pixels and non-aggregated pixels, obtaining a color aggregation vector by calculating the proportion and distribution of the aggregated pixels, selecting the most representative features in the color aggregation vector to form a color aggregation vector set, and using principal component analysis to reduce the dimension of the high-dimensional color aggregation vector in the color aggregation vector set, and finally converting the color aggregation vector encoding into a color numerical vector, which is the color feature vector of the visual design image .

4. The visual design image classification method based on the resnet-50 network model according to claim 1, characterized in that: The specific steps of image texture description mining in S3 are as follows: Sub-image The gray-scale processing is performed to obtain a gray-scale image, the energy, contrast, entropy and inverse gap of the gray-scale image are obtained by using a gray-scale co-occurrence matrix, and the energy, contrast, correlation, entropy and inverse gap of the gray-scale image are taken as a texture feature vector, the high-dimensional feature vector is reduced by using principal component analysis, and finally a visual design image texture feature vector is obtained .

5. The visual design image classification method based on the resnet-50 network model according to claim 2, characterized in that: The image enhancement processing comprises the following steps: Processing sub-images by spatial domain enhancement The image pixel, the enhanced image, its expression as follows: In the formula, f(x, y) is the original image, g(x, y) is the processed image, T is a spatial domain operation on the pixels in the neighborhood of the original image (x, y) position, and the neighborhood of the point (x, y) is defined as a square or rectangular region with (x, y) as the center point.

6. The visual design image classification method based on the resnet-50 network model according to claim 2, characterized in that: The histogram equalization comprises the following two formulas: The transformation function is: ； Wherein, r is the original gray value, s is the transformed gray value, pr(r) is the probability density function of the original gray value, L is the gray level, and the value is 256; The probability density function transformation is: ； Wherein, ps(s) is the probability density function of the changed gray value, pr(r) is the probability density function of the original gray value, L is the gray level, and the value is 256.

7. The visual design image classification method based on the resnet-50 network model according to claim 3, characterized in that: The color aggregation vector formula is as follows: ； wherein CCV is a color clustering vector, is the number of clustered pixels for the Nth color, is the number of non-clustered pixels for the N colors.

8. The visual design image classification method based on the resnet-50 network model according to claim 4, characterized in that: The calculation formulas of energy, contrast, entropy and inverse distance are as follows: Energy: ; Wherein, e and f represent the index of gray level, and P(e, f) is the element value in the gray level co-occurrence matrix; Contrast: ; Wherein, e and f represent the index of gray level, and GLGM(e, f) represents the value of the corresponding position in the gray level co-occurrence matrix; Entropy: ; Wherein, e and f represent the index of gray level, and P(e, f) is the element value in the gray level co-occurrence matrix. Disparity Inverse: ; Wherein, e and f represent the index of gray level, and P(e, f) is the element value in the gray level co-occurrence matrix.

9. The visual design image classification method based on the resnet-50 network model according to claim 1, characterized in that: The cross-entropy loss function in S5 is: ； wherein l(y b = d) is 0 or 1, and 1 means that y b is correctly classified as d, and 0 means that y b is not correctly classified as d, N represents the number of pictures, C represents the involved picture category, a is a random number in the interval of (2, 3), represents the probability that the bth image is determined as d, and L is the cross-entropy loss.

Images