A device for constructing a bottleneck edge model, a system for constructing a neuron network, and a method for constructing the same

Abstract

The present application relates to a kind of bottle neck edge shrink model, neuron network and its construction method, bottle neck edge shrink model is arranged in bottle neck edge shrink module, the bottle neck edge shrink model at least includes two 1*1 convolution layer and a layer m*m convolution layer, wherein m*m convolution layer is arranged between two 1*1 convolution layer, the edge shrink method of the bottle neck edge shrink module includes: G (x) = c_2 (sigma (conv (sigma (c_1 (x))))) ) + r (x) ;Wherein, conv (X) indicates that m*m convolution transformation is carried out to image matrix x without padding, and m>1;C_1 (x) and c_2 (x) indicate that 1*1 convolution transformation is carried out to image matrix x, sigma (x) indicates that nonlinear transformation is carried out to image matrix x;R (x) indicates that edge is shrunk to image matrix x.The bottle neck edge shrink model of the present application reduces the channel number of image matrix by 1×1 convolution layer c_1, so that the parameter number and the amount of calculation of 3×3 convolution kernel are greatly reduced, so that the parameter number of bottle neck edge shrink module is less than the corresponding basic edge shrink module, while the corresponding network depth is increased by 50%, by reducing the parameter number and improving the stability of entire neuron network.

Description

[0001] The present application is a divisional of the patent application No. 201910894420.1, with the title of "Image recognition method and device including surrounding environment", filed on September 20, 2019, and the application type of invention. TECHNICAL FIELD

[0002] The present application relates to the field of information technology, in particular to a bottleneck edge reduction model construction device, a neuron network model construction system and a construction method thereof. BACKGROUND

[0003] Image recognition by deep learning is the most popular development direction and the most successful application direction of artificial intelligence today. In identity authentication, security, autonomous driving, and even integrated circuit design and garbage classification, there is great value in using deep learning to recognize images.

[0004] The specific method of using deep learning to recognize images is to construct a convolutional neural network, collect a large amount of images (data) and their classification labels, use these data and labels to train the convolutional neural network, establish a corresponding model, and use the established model to recognize input images in practical applications.

[0005] In recent years, with the improvement and optimization of the structure of convolutional neural networks, the image recognition ability of deep learning has made great progress. One of the most famous convolutional neural network structures is the residual network (ResNet) proposed by He Kaiming of Microsoft Research Institute in China in late 2015, which has exceeded the level of human recognition in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). The residual network is composed of residual modules. The core idea of the residual network is to replace the convolution layer H(x) in the convolutional neural network with the residual F(x) = H(x) - x that processes the convolution and the input, which can increase the stability of the system and avoid the degradation phenomenon when the neural network is too deep (too many layers). With this idea, residual modules (including basic residual modules and bottleneck residual modules that further reduce parameters and increase levels) can be constructed, and a residual neural network with more network levels than previous neural network models can be constructed based on residual modules, achieving a significant improvement in image recognition.

[0006] When identifying an object in the center of an image, the value of each pixel on the image is not the same. To establish a high-quality convolutional neural network to identify objects in the image, the weight of the pixels of the object itself in the center of the image on the parameters of the neural network should be greater than that of the corresponding pixels of the background around the object.

[0007] However, existing convolutional neural networks, including the residual network by He Kaiming et al., assign the same weight to all pixels in the image on the network parameters. This results in either insufficient parameters to capture enough features of objects in the image, or too many parameters leading to overfitting. At the same time, the computational cost of modeling and recognition is also greater.

[0008] However, simply cropping the image edges and removing the corresponding pixels of the surrounding background results in insufficient image information, making recognition difficult. As shown in Figures 4 and 5, the original image with the background... Figure 4a and Figure 5a As shown, after cropping the image edges, as Figure 4b and Figure 5b As shown, objects in the image are more difficult to identify than in the original image with a background.

[0009] For example, Chinese patent CN108985236A discloses a face recognition method based on a deep separable convolutional model, including the following steps: First, reading a dataset of face image samples; second, establishing a deep separable convolutional model, wherein multiple residual bottleneck modules are cascaded between adjacent convolutional modules; third, updating the parameters of the deep separable convolutional model using a gradient descent algorithm; fourth, performing face recognition using the updated deep separable convolutional model. This patent can improve recognition speed while maintaining accuracy. However, this patent does not differentiate between the weights of face images and background images, resulting in a large computational load.

[0010] For example, Chinese patent 108921788A discloses an image super-resolution method based on deep residual CNN, including: constructing a deep residual CNN model; inputting a low-resolution image into the deep residual CNN model for recognition to obtain a recognized image; and inputting the recognized image into a VGG loss network for recognition performance evaluation to obtain a super-resolution recognized image. This patent evaluates the recognition performance of the obtained image to obtain a higher-resolution recognized image, but it does not pre-distinguish the weights between the recognized image and the background image, resulting in a large computational load.

[0011] For example, Chinese patent CN 110163260 A discloses a residual network-based image recognition method, which includes: constructing a residual network, training the residual network with preset training samples; obtaining an image to be recognized; preprocessing the image to be recognized; dividing the preprocessed image to be recognized into two non-overlapping blocks, sequentially inputting the two blocks into the residual network as input, and obtaining the prediction value of each block after the residual network; and outputting a recognition result according to the prediction values of the two blocks, the recognition result including that the image to be recognized is a target image and the image to be recognized is a non-target image. This patent extracts key features through a residual network, can reduce noise interference, and can determine a target image with high accuracy. However, this patent still cannot reduce the computational complexity of modeling and recognition.

[0012] For example, Chinese patent 108765449A discloses a convolutional neural network-based image background segmentation and recognition method, which uses a convolutional neural network to learn sample images, trains a background segmentation model and a recognition classification model, and then performs image background segmentation and recognition classification according to the model. This patent applies a convolutional neural network to image recognition classification, considers the influence of the image background on recognition classification, and replaces the fully connected layer in the convolutional neural network with a convolutional layer in the background segmentation model. This patent uses a convolutional neural network model to realize image background segmentation, and then uses the image for image recognition classification to improve the accuracy of recognition classification. However, after the background is segmented, the information with relative significance to the background is ignored, resulting in inaccurate image recognition.

[0013] For example, Chinese patent CN102694961B discloses a method and device for single-sided edge shrinking of an image, which includes: synthesizing an original image with a key signal of the original image, aligning the object positions in the original image and the key signal during the synthesis, obtaining a processed foreground after the synthesis; determining the position of a black border in the processed foreground; keeping the position of the original image unchanged, moving the key signal of the original image relative to the original image by a specified movement distance in the direction opposite to the black border position; synthesizing the original image with the moved key signal again; and performing overall edge shrinking on the image obtained after the re-synthesis. This patent enhances the display effect of the image by performing single-sided edge shrinking on the image. However, this patent describes a method and device for image enhancement, and does not involve image recognition, or it is an image enhancement means performed after the foreground and background are distinguished by other image recognition devices. Furthermore, the action of edge shrinking in this patent refers to directly removing the pixels at the edge of the image after image processing, rather than the characteristic elements at the edge of the matrix in the neural network. In the neural network, except for the matrix of the input layer which is image information, the matrices of other layers cannot be called images, but are characteristic elements.

[0014] In addition, on the one hand, there are differences in the understanding of those skilled in the art; on the other hand, a large number of literatures and patents are studied when the present application is made, but due to the limitation of the volume, all the details and contents are not listed in detail, but this does not mean that the present application does not have these prior art characteristics, on the contrary, the present application has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art in the background art. SUMMARY

[0015] In view of the deficiencies of the prior art, the present application provides an image recognition method including a surrounding environment, characterized in that the method comprises the following steps: extracting an image matrix based on a convolution layer; performing at least one edge reduction processing on the image matrix; performing at least one non-edge reduction residual processing on the image matrix after edge reduction processing; and outputting recognized image data based on a full connection layer. Compared with the neuron network in the prior art, the neuron network of the present application has the advantage of better image recognition effect due to the setting of the edge reduction module. First, the introduction of the edge reduction module extracts the feature information of the image edge part representing the surrounding environment pixels, which helps to improve the recognition quality of the objects in the image. Second, the introduction of the edge reduction module makes the number of parameters corresponding to the image edge pixel feature information less than the number of parameters of the image middle part about the pixel feature information of the object to be identified, thereby reducing the number of parameters of the entire neuron network without reducing the number of parameters of the image middle part about the pixel feature information of the object to be identified, improving the stability and recognition quality of the network. Third, the introduction of the edge reduction module makes the weight of the image edge pixel feature information less than the weight of the image middle part about the pixel feature information of the object to be identified, thereby improving the recognition rate of the objects in the image middle part and reducing the misidentification of irrelevant objects in the image edge part.

[0016] According to a preferred embodiment, the method further comprises the step of constructing an edge reduction module; the edge reduction method of the edge reduction module comprises: extracting the middle and / or edge feature elements of the image matrix based on the convolution layer, removing the feature elements of the edge k layers of the image matrix, and after edge reduction, the size of the convolution kernel of the convolution layer is: m=k*2+1, wherein m and k are positive integers. The advantage of the edge reduction method of the present application is that the feature elements of the middle and edge pixels of the input image matrix x can be extracted through the padding-free convolution transformation, which not only reduces the calculation amount, but also makes the weights of the middle and edge pixels different: the influence weight of the edge pixel is lower than that of the middle pixel.

[0017] According to a preferred embodiment, the edge shrinking module is a basic edge shrinking module, the basic edge shrinking module comprises at least two m*m convolution layers, and the edge shrinking method of the basic edge shrinking module comprises: G(x) = conv_2(σ(conv_1(x)))+r(x),

[0018] wherein conv_1(x) and conv_2(x) represent m*m convolution transformation of the image matrix x without padding, m>1; σ(x) represents nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x. The basic edge shrinking module has the advantage that the edge two layers of pixels can be removed, thereby reducing the influence weight of the edge pixels and making the influence weight of the edge pixels lower than the influence weight of the middle pixels.

[0019] According to a preferred embodiment, the edge shrinking module is a bottleneck edge shrinking module, the bottleneck edge shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers, and the edge shrinking method of the bottleneck edge shrinking module comprises:

[0020] G(x) = c_2(σ(conv(σ(c_1(x)))))+r(x)

[0021] wherein conv(x) represents m*m convolution transformation of the image matrix x without padding, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation of the image matrix x; σ(x) represents nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x. The bottleneck edge shrinking module has the advantage or technical effect that the channel number of the image matrix is reduced by the 1*1 convolution layer c_1, the parameter number and the calculation amount of the 3*3 convolution kernel are greatly reduced, thereby making the parameter number of the bottleneck edge shrinking module less than that of the corresponding basic edge shrinking module, while the network depth is increased by 50%.

[0022] According to a preferred embodiment, the function of r(x) for removing the feature elements of the edge k layers of the image matrix is:

[0023] that is, the first to k rows, the first to k columns, the (n-k+1) to n rows and the (n-k+1) to n columns of the matrix x are removed.

[0024] According to a preferred embodiment, the m*m convolution layer of the basic edge shrinking module is a convolution layer comprising a 3*3 convolution kernel; and the m*m convolution layer of the bottleneck edge shrinking module is a convolution layer comprising a 3*3 convolution kernel.

[0025] According to a preferred embodiment, the method further comprises: performing at least one pooling processing and / or convolution processing on the image matrix to change the number of channels before or during the non-edge-shrinking residual processing.

[0026] The application further provides an image recognition device comprising a surrounding environment, characterized in that the device comprises at least: a convolution module configured to extract an image matrix based on a convolution layer; an edge-shrinking unit configured to perform at least one edge-shrinking processing on the image matrix; a residual unit configured to perform at least one non-edge-shrinking residual processing on the image matrix after the edge-shrinking processing; and a fully connected layer module configured to output recognized image data based on a fully connected layer. The edge-shrinking unit is arranged between the convolution module and the residual unit, and the residual unit is connected to the fully connected layer module.

[0027] According to a preferred embodiment, the edge-shrinking unit comprises a plurality of edge-shrinking modules, and the edge-shrinking method of the edge-shrinking modules comprises: extracting middle and / or edge feature elements of the image matrix based on the convolution layer, and removing feature elements of edge k layers of the image matrix. After the edge-shrinking, the size of the convolution kernel of the convolution layer is m=k*2+1, where m and k are positive integers.

[0028] According to a preferred embodiment, the edge-shrinking module comprises a basic edge-shrinking module and a bottleneck edge-shrinking module. The basic edge-shrinking module comprises at least two m*m convolution layers, and the edge-shrinking method of the basic edge-shrinking module comprises: G(x)=conv_2(σ(conv_1(x)))+r(x),

[0029] wherein conv_1(x) and conv_2(x) represent m*m convolution transformation without padding on the image matrix x, m>1; σ(x) represents nonlinear transformation on the image matrix x; and r(x) represents edge-shrinking on the image matrix x.

[0030] The bottleneck edge-shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer, and the m*m convolution layer is arranged between the two 1*1 convolution layers. The edge-shrinking method of the bottleneck edge-shrinking module comprises: G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0031] wherein conv(x) represents m*m convolution transformation without padding on the image matrix x, m≥1; c_1(x) and c_2(x) represent 1*1 convolution transformation on the image matrix x; σ(x) represents nonlinear transformation on the image matrix x; and r(x) represents edge-shrinking on the image matrix x.

[0032] The application further provides a bottleneck edge shrinking model construction device arranged in a bottleneck edge shrinking module, the bottleneck edge shrinking model comprising at least two 1*1 convolution layers and one m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers, and the edge shrinking method of the bottleneck edge shrinking module comprises:

[0033] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0034] wherein conv(x) represents m*m convolution transformation without padding on the image matrix x, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation on the image matrix x; σ(x) represents nonlinear transformation on the image matrix x; and r(x) represents edge shrinking on the image matrix x.

[0035] Preferably, the function of r(x) for removing the feature elements of the edge k layers of the image matrix is as follows:

[0036]

[0037] that is, the first to k rows, the first to k columns, the (n-k+1) to n rows and the (n-k+1) to n columns of the original matrix x are removed.

[0038] Preferably, the second convolution layer is a 3x3 convolution kernel convolution layer.

[0039] The application further provides a neuron network construction system, which at least comprises:

[0040] a convolution module arranged with an image information input end and used for extracting an image matrix based on a convolution layer;

[0041] an edge shrinking unit comprising at least one edge shrinking module and used for performing at least one edge shrinking process on the image matrix;

[0042] a residual unit comprising at least one residual module and used for extracting relatively abstract features of a to-be-recognized region in a non-edge part of the image;

[0043] a full connection layer module used for outputting recognized image data based on a full connection layer;

[0044] the convolution module, the edge shrinking unit, the residual unit and the full connection layer module are sequentially connected.

[0045] In the neuron network construction system of the application, the edge shrinking module is a bottleneck edge shrinking module,

[0046] the bottleneck edge shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers,

[0047] The necking method of the bottle necking module comprises:

[0048] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0049] Wherein, conv(x) represents m*m convolution transformation without padding on the image matrix x, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation on the image matrix x, sigma(x) represents nonlinear transformation on the image matrix x; r(x) represents necking on the image matrix x.

[0050] In the construction system of the neuron network of the application, the function of r(x) for removing the feature elements of the edge k layers of the image matrix is:

[0051]

[0052] That is, the 1st-kth row, the 1st-kth column, the (n-k+1)th-nth row and the (n-k+1)th-nth column of the original matrix x are removed.

[0053] The application further provides a construction method of a neuron network, which comprises:

[0054] The convolution module is connected with the necking unit;

[0055] The necking unit is a bottle necking unit (20); a first pooling module (31) and a first 1x1 convolution module (41) are arranged between the bottle necking unit (20) and a bottle neck residual unit (50);

[0056] The first pooling module (31) is arranged upstream of the data of the first 1x1 convolution module (41);

[0057] The bottle neck residual unit (50) is connected with a full connection layer module (60);

[0058] The full connection layer module (60) is used for outputting the final image data.

[0059] In the construction method of the neuron network of the application, the bottle necking unit (20) comprises a first bottle necking module (21), a second bottle necking module (22), a third bottle necking module (23) and a fourth bottle necking module (24) connected in sequence;

[0060] Each bottle necking module removes 1 layer of edge feature elements, and the first bottle necking module (21), the second bottle necking module (22), the third bottle necking module (23) and the fourth bottle necking module (24) can remove 4 layers of edge feature elements of the image matrix, so that the number of rows and columns of the image matrix is reduced by 8.

[0061] In the construction method of the neuron network of the present application, the second pooling module (32) and the second 1x1 convolution module (42) are arranged between any two non-edge-reducing bottleneck residual modules in the bottleneck residual unit (50); the second pooling module (32) is arranged upstream of the data of the second 1x1 convolution module (42).

[0062] The role of the pooling module arranged between the residual modules is to change the size of the image matrix.

[0063] In the construction method of the neuron network of the present application, the bottleneck residual module comprises at least three convolution layers, and the expression of the bottleneck residual module can be written as: H(x) = c_2(σ(conv(σ(c_1(x)))))+x, wherein conv(x) represents a 3x3 convolution transformation with padding = 1 on the image matrix x.

[0064] c_1(x) and c_2(x) represent 1x1 convolution transformation on the image matrix x,

[0065] σ(x) represents nonlinear transformation on the image matrix x. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 is a logic diagram of the edge-reducing method of the basic edge-reducing module of the present application;

[0067] Figure 2 is a logic diagram of the edge-reducing method of the bottleneck edge-reducing module of the present application;

[0068] Figure 3 is a logic diagram of a preferred embodiment of the present application;

[0069] Figure 4a is a schematic diagram of one of the present applications with a background original image;

[0070] Figure 4b is a schematic diagram of one of the present applications without an edge image;

[0071] Figure 5a is a schematic diagram of another background original image of the present application; and

[0072] Figure 5b is a schematic diagram of another edgeless image of the present application.

[0073] LIST OF REFERENCE NUMBERS

[0074] 11: first 3x3 convolution module 12: second 3x3 convolution module

[0075] 13: third 3x3 convolution module r(x): edge-reducing module

[0076] 10: 5x5 convolution module 20: bottleneck shave unit

[0077] 21: first bottleneck shave module 22: second bottleneck shave module

[0078] 23: third bottleneck shave module 24: fourth bottleneck shave module

[0079] 31: first pooling module 32: second pooling module

[0080] 41: first 1x1 convolution module 42: second 1x1 convolution module

[0081] 50: bottleneck residual unit 51: first bottleneck residual module

[0082] 52: second bottleneck residual module 53: third bottleneck residual module

[0083] 54: fourth bottleneck residual module 55: fifth bottleneck residual module

[0084] 60: fully connected layer module DETAILED DESCRIPTION

[0085] The present application will be described in detail below with reference to the accompanying drawings.

[0086] In the present application, padding refers to the attribute definition element border and the space between the element content. When the padding inner margin of the element is cleared, the released area will be filled with the background color of the element. The padding attribute alone can change the padding above, below, left and right.

[0087] Residual network: the feature of residual network is easy to optimize, and can improve the accuracy by increasing the depth. The residual block inside uses a jump connection, which alleviates the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0088] Basic residual module: a module in the residual network; the module includes at least two convolution layers, and the expression of the module can be written as: H(x) = conv_2(σ(conv_1(x)))+x, wherein, conv_1(x), conv_2(x) represent 3x3 convolution transformation with padding = 1 on the image matrix x; σ(x) represents nonlinear transformation on the image matrix x.

[0089] Bottleneck residual module: a module in a residual network; the module comprises at least three convolution layers, and an expression of the module can be written as: H(x) = c_2(σ(conv(σ(c_1(x)))))+x, wherein conv(x) represents a 3*3 convolution transformation with padding = 1 on an image matrix x; c_1(x) and c_2(x) represent 1*1 convolution transformations on the image matrix x; and σ(x) represents a nonlinear transformation on the image matrix x.

[0090] Convolution layer: used for extracting features of an image, and weights of a convolution kernel are learnable. Each convolution layer is composed of a plurality of convolution units, and parameters of each convolution unit are obtained by optimizing an algorithm of back propagation. A purpose of the convolution operation is to extract different features of an input, and a first convolution layer can only extract low-level features such as edges, lines and corners, and more layers of the network can iteratively extract more complex and abstract features from the low-level features.

[0091] Pooling layer: used for reducing a size of an input of a next layer, thereby reducing a calculation amount and a number of parameters, so that network overfitting can be prevented.

[0092] Fully connected layer: each neuron in a fully connected layer is fully connected with all neurons of a previous layer. The fully connected layer can integrate local information having a class distinction in a convolution layer or a pooling layer. In order to improve a performance of the CNN network, an excitation function of each neuron in the fully connected layer generally adopts a ReLU function. Output values of a last fully connected layer are transmitted to an output, and a softmax regression can be used for the layer, which can also be called a softmax layer.

[0093] Embodiment 1

[0094] In the prior art, a core convolution layer used in a residual network is previously added with padding, the input and output matrix of the convolution layer are of the same size, and a residual operation F(x) = H(x)-x is performed. Wherein, F(X) is a residual function, indicating a difference between a module output and an output, H(X) is a module output matrix, and X is a module input matrix (an image matrix).

[0095] In the prior art, the technical scheme of "edge shrinking" is only mentioned in Chinese patent CN102694961B, but the "edge shrinking" of this patent refers to directly removing the pixels of the image edge after image processing. This is completely different from the definition of the edge shrinking of the present application, and the purposes of use are also irrelevant. The edge shrinking of the present application refers to removing the feature elements of the edge of the matrix in the neuron network. Among them, in the neuron network, except that the matrix of the input layer is an image, the matrices of other layers cannot be called images, but feature elements. The edge shrinking of the present application appears in cooperation with two or three padding-free convolutions in the neuron network to ensure that the size of the matrix after edge shrinking is the same as that of the matrix after convolution.

[0096] The present application proposes an edge shrinking operation method on the basis of the residual network. The edge shrinking method of the present application refers to not adding padding to the image, so that the output matrix of the convolution layer is smaller in scale than the input matrix. At this time, when performing the residual operation, the input image matrix needs to be edge shrunk. That is, G(x) = H(x)-r(x). Wherein r(x) is the edge shrinking operation on the input image matrix x. The edge shrinking method includes: based on the convolution layer, extracting the middle and / or edge feature elements of the image matrix, and removing the feature element information of the edge k layer of the image matrix. After edge shrinking, the size of the convolution kernel of the convolution layer is: m = k*2+1. Wherein m, k are positive integers. The advantage of the edge shrinking method of the present application is that the middle and edge feature elements of the input image matrix x can be extracted by convolution, so that the weights of the influence of the middle and edge feature elements on the output are different: the influence weight of the edge feature element is lower than that of the middle feature element.

[0097] The device carrying the edge shrinking operation method is defined as an edge shrinking module. The edge shrinking module includes a basic edge shrinking module and a bottleneck edge shrinking module.

[0098] The basic edge shrinking module includes at least two m*m convolution layers. The edge shrinking method of the basic edge shrinking module includes: G(x) = conv_2(σ(conv_1(x)))+r(x). Wherein conv_1(x), conv_2(x) represent m*m convolution transformation of the image matrix x without padding, m≥1. σ(x) represents nonlinear transformation of the image matrix x. For example, σ(x) is a ReLU function. r(x) represents edge shrinking of the image matrix x.

[0099] When the convolutional layer is preferably a convolutional layer containing a 3x3 convolution kernel, conv_1(x), conv_2(x) represent a 3x3 convolution transformation without padding on the image matrix x. r(x) represents removing the feature elements of the two edges of x to ensure that the size of r(x) is the same as that of conv_2(σ(conv_1(x))). The advantage of the basic edge reduction module is that it can remove the feature elements of the two edges, thereby reducing the influence weight of the edge feature elements, so that the influence weight of the edge feature elements is lower than that of the middle feature elements, that is, the influence weight of the edge pixels is lower than that of the middle pixels. The basic edge reduction module of the present application must contain at least two convolutional layers, otherwise its edge reduction and residual operation can be combined with the convolutional layer, so that the operation is linear and cannot improve the recognition effect.

[0100] As shown in Figure 1 , according to one preferred embodiment, the convolutional layer of the basic edge reduction module is preferably a convolutional layer containing a 3x3 convolution kernel. As shown in Figure 1 , the input image matrix x is subjected to a first 3x3 convolution transformation without padding by the first 3x3 convolution module 11, and is subjected to a linear transformation by the ReLU (Rectified Linear Units) activation function and then input into the second 3x3 convolution module 12. The second 3x3 convolution module 12 performs a second 3x3 convolution transformation without padding on the image matrix, and then reduces the edges of the input image matrix by r(x) to remove the feature elements of the two edges. According to one preferred embodiment, the edge reduction module is a bottleneck edge reduction module. The bottleneck edge reduction module includes at least two 1*1 convolutional layers and one m*m convolutional layer. The m*m convolutional layer is arranged between the two 1*1 convolutional layers. The edge reduction method of the bottleneck edge reduction module includes:

[0101] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x).

[0102] Wherein, conv(x) represents an m*m convolution transformation without padding on the image matrix x, m>1. c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x, σ(x) represents a nonlinear transformation on the image matrix x; r(x) represents edge reduction on the image matrix x.

[0103] Preferably, as shown in Figure 2 , in the bottleneck edge reduction module, the second convolutional layer is a 3x3 convolution kernel. Then conv(x) represents a 3*3 convolution transformation without padding on the image matrix x.

[0104] Specifically, as shown in Figure 2As shown, the convolution layer is a third 3x3 convolution module 13 with a 3x3 convolution kernel, which is arranged between the first 1x1 convolution module 41 and the second 1x1 convolution module 42. The image matrix X is input into the first 1x1 convolution module 41 for the first 1x1 convolution transformation without padding. The image matrix output by the first 1x1 convolution module 41 is input into the third 3x3 convolution module 13 after linear transformation by the ReLU (Rectified Linear Units) activation function, for 3x3 convolution transformation without padding. The image matrix output by the third 3x3 convolution module 13 is input into the second 1x1 convolution module 42 after linear transformation by the ReLU (Rectified Linear Units) activation function, for the second 1x1 convolution transformation without padding. The image matrix output by the second 1x1 convolution module 42 is edge-reduced by r(x) to remove the feature elements in the edge layer, so as to ensure that the size of r(x) is the same as that of c_2(σ(conv(σ(c_1(x)))). Compared with the basic edge-reduction module, the advantage or technical effect of the bottleneck edge-reduction module lies in that the channel number of the image matrix is reduced by the 1x1 convolution layer c_1, so that the parameter number and the calculation amount of the 3x3 convolution kernel are greatly reduced, thereby the parameter number of the bottleneck edge-reduction module is less than that of the corresponding basic edge-reduction module, and at the same time, the network depth is increased by 50%. Reducing the parameter number can improve the stability of the entire neural network, and increasing the network depth can improve the image recognition rate. Practice shows that the bottleneck edge-reduction module can replace the basic edge-reduction module to achieve better recognition effect.

[0105] Preferably, in the basic edge-reduction module and the bottleneck edge-reduction module, the convolution kernel of the convolution module is not limited to the 3x3 convolution kernel, but can also be other odd-size convolution kernels, such as 5x5 and 7x7 convolution kernels. The edge-reduction operation of the edge-reduction module changes with the convolution kernel, and the corresponding number of removed feature elements is changed accordingly, so as to ensure that the size of r(x) is appropriate. In the actual edge-reduction operation process, the convolution module with the 3x3 convolution kernel is the most beneficial to avoid overfitting problem, and the calculation efficiency is relatively high.

[0106] Preferably, the function of r(x) for removing the feature elements in the edge k layer of the image matrix is:

[0107]

[0108] That is, the first to kth rows, the first to kth columns, the (n-k+1)th to nth rows and the (n-k+1)th to nth columns of the matrix x are removed. The image matrix X is A nn matrix.

[0109] Preferably, the r(x) function for removing the feature elements in the edge layer can be written as That is, the first row, the first column, the last row and the last column of the original matrix x are removed. Similarly, the r(x) function of removing the two layers of feature elements on the edge is to remove the first, second rows, the first, second columns, the last two rows and the last two columns of the original matrix x. Preferably, the neuron network is constructed by using the basic edge reduction module and the bottleneck edge reduction module, so as to reduce the influence weight of the edge below the bump, thereby obtaining better image recognition effect.

[0110] Preferably, the neuron network at least includes a convolution module, an edge reduction module, a residual module and a full connection module. Preferably, the neuron network further includes a pooling module and other basic linear / nonlinear transformations.

[0111] Preferably, the image information input end in the neuron network is provided with a convolution layer for extracting basic geometric features of the image, such as various direction boundaries and the like. Adjacent to the convolution layer is provided with an edge reduction unit including a plurality of edge reduction modules. The number of the edge reduction modules is determined by the size of the input image and the relative size of the object to be identified and the surrounding environment in the image. The edge reduction unit is used to intercept the edge layer feature elements in the image matrix. Adjacent to the edge reduction unit is provided with a non-edge reduction residual unit. The residual unit includes a plurality of residual modules. The residual module includes a basic residual module and a bottleneck residual module. Adjacent to the residual module is provided with a full connection layer module. The residual module is used after the edge reduction module and before the full connection layer, for extracting relatively abstract features of the to-be-identified region in the non-edge part of the image. The full connection layer module is used as the last layer of the neuron network, for extracting the final classification feature information of the image.

[0112] Compared with the neuron network in the prior art, the neuron network of the present application has the advantages of better image recognition effect, which is due to the setting of the edge reduction module. First, the introduction of the edge reduction module extracts the feature information of the image edge part representing the surrounding environment pixels, which helps to improve the identification quality of the object in the image. Second, the introduction of the edge reduction module makes the number of parameters corresponding to the image edge pixel feature information smaller than the number of parameters of the image middle part about the pixel feature information of the to-be-identified object, thereby reducing the number of parameters of the entire neuron network without reducing the number of parameters of the image middle part about the pixel feature information of the to-be-identified object, improving the stability and identification quality of the network. Third, the introduction of the edge reduction module makes the weight of the image edge pixel feature information smaller than the weight of the image middle part about the pixel feature information of the to-be-identified object, thereby improving the identification rate of the object in the image middle part and reducing the misidentification of irrelevant objects in the image edge part.

[0113] Preferably, in the neuron network, a pooling layer can be arranged between the edge reduction module and the non-edge reduction residual module to reduce the image, or a 1x1 convolution layer can be arranged to change the number of channels.

[0114] Preferably, the edge-shrinking module is arranged before the non-edge-shrinking residual module, which has relative advantages. The pixels corresponding to the peripheral environment are located at the edge of the image matrix, and the feature elements inside the image matrix correspond to the pixels of the object to be identified. The feature elements at the edge of the input image matrix have a low weight on the final output, and after being removed, not only the calculation amount can be reduced, but also the influence of the image edge can be reduced. The feature elements close to the middle and the middle of the input image are also important for the identification of the object in the image, and at this time, the non-edge-shrinking residual module can keep the feature elements close to the middle and the middle of the image matrix unaffected. Therefore, the edge-shrinking module is arranged before the non-edge-shrinking residual module, so that the contrast between the middle information and the edge information of the image region is increased, and the middle image information is highlighted.

[0115] Figure 3 is a specific embodiment of the neuron network of the application. Figure 3 The figure shows an embodiment of identifying the numbers (0-10) in a single-channel image (commonly known as a black and white image) with an image matrix of 32x32. Figure 3 In the figure, M(32x32), CH(1) indicates that the image matrix is 32x32 and the channel is 1. By analogy.

[0116] The 5x5 convolution module 10 with a 5x5 convolution kernel is connected with the bottleneck edge-shrinking unit 20. The first pooling module 31 and the first 1x1 convolution module 41 are arranged between the bottleneck edge-shrinking unit 20 and the bottleneck residual unit 50. Moreover, the first pooling module 31 is arranged upstream of the data of the first 1x1 convolution module 41. The second pooling module 32 and the second 1x1 convolution module 42 can be arranged between any two non-edge-shrinking residual modules in the bottleneck residual unit 50. Moreover, the second pooling module 32 is arranged upstream of the data of the second 1x1 convolution module 42. The bottleneck residual unit 50 is connected with the fully connected layer module 60. The fully connected layer module 60 outputs the last image data.

[0117] Preferably, the bottleneck edge-shrinking unit 20 includes the first bottleneck edge-shrinking module 21, the second bottleneck edge-shrinking module 22, the third bottleneck edge-shrinking module 23 and the fourth bottleneck edge-shrinking module 24 connected in sequence. Each bottleneck edge-shrinking module removes 1 layer of edge feature elements, and the four bottleneck edge-shrinking modules can remove four layers of edge feature elements of the image matrix, that is, the number of rows and columns of the image matrix is reduced by 8 respectively, and the size of the image matrix is changed from 28x28 input by the bottleneck edge-shrinking unit 20 to 20x20 output by the unit.

[0118] Preferably, the residual modules in the non-edge-shrinking bottleneck residual unit 50 are all non-edge-shrinking residual modules. Preferably, the non-edge-shrinking bottleneck residual unit 50 comprises a first bottleneck residual module 51, a second bottleneck residual module 52, a third bottleneck residual module 53, a fourth bottleneck residual module 54 and a fifth bottleneck residual module connected in sequence. The second pooling module 32 and the second 1x1 convolution module 42 are arranged between the third bottleneck residual module 53 and the fourth bottleneck residual module 54. Preferably, the role of arranging the pooling module between the bottleneck residual modules is to change the size of the image matrix. For example, Figure 3 The middle pooling module halves the number of rows and columns of the image matrix, and adds a nonlinear layer in the network. The role of the 1x1 convolution module is to change the number of image channels. For example, Figure 3 The number of image channels is doubled.

[0119] As shown in Figure 3 The image recognition method of the neuron network of the present application includes the following steps:

[0120] S1: extracting an image matrix based on a convolution layer;

[0121] S2: performing at least one edge-shrinking processing on the image matrix;

[0122] S3: performing at least one non-edge-shrinking residual processing on the image matrix after the edge-shrinking processing;

[0123] S4: outputting the recognized image data based on a full connection layer.

[0124] The present application discusses the image recognition method in detail.

[0125] The 5x5 convolution module 10 extracts the image matrix, i.e. the geometric features of the image, from the input image data. The matrix output by the 5x5 convolution module 10 is M(28x28), CH(16). The first bottleneck edge reduction module 21, the second bottleneck edge reduction module 22, the third bottleneck edge reduction module 23 and the fourth bottleneck edge reduction module 24 sequentially reduce the edge features of the image matrix by one layer each time, and a total of four layers of edge features are removed. Accordingly, the image matrix changes from M(28x28), CH(16) to M(20x20), CH(16). The first pooling module 31 reduces the image by pooling, so that the image matrix changes from M(20x20), CH(16) to M(10x10), CH(16). The first 1x1 convolution module 41 performs convolutional transformation on the image matrix to change the number of channels from 16 to 32. The first bottleneck residual module 51, the second bottleneck residual module 52 and the third bottleneck residual module 53 sequentially extract features from the image matrix. The size of the image matrix does not change in this process. The second pooling module 32 reduces the image by pooling, so that the image matrix changes from M(10x10), CH(32) to M(5x5), CH(16). The second 1x1 convolution module 42 performs convolutional transformation on the image matrix to change the number of channels from 32 to 64. At this time, the image matrix is M(5x5), CH(64). The fourth bottleneck residual module 54 and the fifth bottleneck residual module 55 perform bottleneck residual processing on the image matrix, and the processed image matrix is input into the fully connected layer module 60. The fully connected layer module 60 fully connects each neuron in the image matrix with all neurons in the previous layer to extract feature information for final image classification. The fully connected layer module 60 outputs the integrated image recognition information.

[0126] Preferably, after the neuron network is constructed, a plurality of training samples are input into the neuron network for deep learning training. After a large amount of training is completed, the neuron network can accurately recognize images including the environment. Moreover, the calculation amount of image recognition is greatly reduced.

[0127] Embodiment 2

[0128] This embodiment is a further supplement and explanation of the foregoing embodiments, and the repeated contents will not be described again.

[0129] The embodiment provides an image recognition device including a peripheral environment, characterized in that the device at least comprises:

[0130] a convolution module configured to extract an image matrix based on a convolution layer;

[0131] an edge reduction unit configured to perform at least one edge reduction processing on the image matrix;

[0132] a residual unit configured to perform at least one non-shrinking residual processing on the image matrix after the edge shrinking processing;

[0133] a full connection layer module configured to recognize the image data based on the full connection layer output. The edge shrinking unit is arranged between the convolution module and the residual unit. That is, the convolution module is arranged at the data input end and connected with the edge shrinking unit to input the image matrix extracted based on the convolution layer into the edge shrinking unit. The edge shrinking unit is connected with the residual unit to input the image matrix without the edge layer matrix elements into the residual unit. The residual unit is connected with the full connection layer module to extract relatively abstract features of the to-be-recognized region of the non-edge part of the image and input the extracted relatively abstract features of the non-edge part of the image matrix into the full connection layer module. The full connection layer module is used as the last layer of the neuron network to extract the feature information of the final classification of the image and output.

[0134] Compared with the neuron network in the prior art, the neuron network of the present application has the advantages of better image recognition effect due to the arrangement of the edge shrinking module. First, the introduction of the edge shrinking module extracts the feature information of the image edge part representing the surrounding environment pixels, which helps to improve the recognition quality of the object in the image. Second, the introduction of the edge shrinking module makes the number of parameters corresponding to the feature information of the image edge pixels less than the number of parameters of the feature information of the pixels of the to-be-recognized object in the middle part of the image, thereby reducing the number of parameters of the entire neuron network without reducing the number of parameters of the feature information of the pixels of the to-be-recognized object in the middle part of the image, improving the stability and recognition quality of the network. Third, the introduction of the edge shrinking module makes the weight of the feature information of the image edge pixels less than the weight of the feature information of the pixels of the to-be-recognized object in the middle part of the image, thereby improving the recognition rate of the object in the middle part of the image and reducing the misrecognition of the irrelevant objects in the edge part of the image.

[0135] Preferably, the various modules of the present application can be servers, processors and / or integrated chips and the like hardware, or servers loaded with corresponding computing processing methods.

[0136] For example, the convolution module is a server, processor or integrated chip with convolution transformation function. The edge shrinking unit includes a plurality of servers, processors or integrated chips with edge shrinking processing function. The edge shrinking module is a server, processor or integrated chip loaded with edge shrinking computing method. The residual unit includes a plurality of servers, processors or integrated chips with residual processing function. The residual module is a server, processor or integrated chip loaded with residual processing algorithm. The full connection layer module is a server, processor or integrated chip loaded with full connection layer computing method.

[0137] The edge shrinking unit comprises a plurality of edge shrinking modules. The edge shrinking method of the edge shrinking module comprises: extracting intermediate and / or edge feature elements of an image matrix based on a convolution layer, and removing feature elements of edge k layers of the image matrix. After edge shrinking, the size of a convolution kernel of the convolution layer is m=k*2+1.

[0138] Preferably, the edge shrinking module comprises a basic edge shrinking module and a bottleneck edge shrinking module. The basic edge shrinking module comprises at least two m*m convolution layers. The edge shrinking method of the basic edge shrinking module comprises: G(x)=conv_2(sigma(conv_1(x)))+r(x). Wherein, conv_1(x) and conv_2(x) represent m*m convolution transformation of the image matrix x without padding, and m>1; sigma(x) represents nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0139] The bottleneck edge shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer. The m*m convolution layer is arranged between the two 1*1 convolution layers. The edge shrinking method of the bottleneck edge shrinking module comprises: G(x)=c_2(sigma(conv(sigma(c_1(x)))))+r(x). Wherein, conv(x) represents m*m convolution transformation of the image matrix x without padding, and m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation of the image matrix x; sigma(x) represents nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0140] The function of r(x) for removing feature elements of edge k layers of the image matrix is:

[0141] That is, the first to k rows, the first to k columns, the (n-k+1) to n rows and the (n-k+1) to n columns of the original matrix x are removed.

[0142] It should be noted that the above specific embodiments are exemplary, and those skilled in the art can think of various solutions under the inspiration of the disclosure of the present application, and these solutions also belong to the disclosed range of the present application and fall within the protection scope of the present application. Those skilled in the art should understand that the specification and drawings of the present application are illustrative and not constitute a limitation on the claims. The protection scope of the present application is defined by the claims and their equivalents. The specification of the present application contains a plurality of inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally", which all indicate that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application according to each inventive concept.

Claims

1. A device for constructing a bottle neck finish model, characterized by, The bottleneck edge shrinking module is arranged in the neck edge shrinking module, The bottleneck edge shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers, The edge shrinking method of the bottleneck edge shrinking module comprises: G(x) = c_2(sigma(conv(sigma(c_1(x))))) + r(x), conv(x) represents m*m convolution transformation of the image matrix x without padding, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation of the image matrix x; sigma(x) represents nonlinear transformation of the image matrix x; r(x) represents edge shrinking of the image matrix x. The function of r(x) for removing the feature elements of the edge k layers of the image matrix is: That is, the first to k rows, the first to k columns, the (n-k+1) to n rows and the (n-k+1) to n columns of the original matrix x are removed.

2. The apparatus according to claim 1, wherein The second convolution layer is a 3x3 convolution kernel convolution layer.

3. A system for constructing a neuronal network, characterized by At least comprising: A convolution module arranged at an image information input end and used for extracting an image matrix based on a convolution layer; An edge shrinking unit comprising at least one edge shrinking module and used for performing at least one edge shrinking processing on the image matrix; A residual unit comprising at least one residual module and used for extracting relatively abstract features of a to-be-identified region in a non-edge part of the image; A full connection layer module used for outputting identified image data based on a full connection layer; The convolution module, the edge shrinking unit, the residual unit and the full connection layer module are sequentially connected; The edge shrinking module is a bottleneck edge shrinking module, The bottleneck edge shrinking module comprises at least two 1*1 convolution layers and one m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers, The edge shrinking method of the bottleneck edge shrinking module comprises: G(x) = c_2(sigma(conv(sigma(c_1(x))))) + r(x), conv(x) represents m*m convolution transformation of the image matrix x without padding, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation of the image matrix x; sigma(x) represents nonlinear transformation of the image matrix x; r(x) represents edge shrinking of the image matrix x. The function of r(x) for removing the feature elements of the edge k layers of the image matrix is: That is, the first to k rows, the first to k columns, the (n-k+1) to n rows and the (n-k+1) to n columns of the original matrix x are removed.

4. A method of constructing a neuronal network, characterized by The method comprises: Connecting the convolution module and the edge shrinking unit; The edge shrinking unit is a bottleneck edge shrinking unit (20); a first pooling module (31) and a first 1x1 convolution module (41) are arranged between the bottleneck edge shrinking unit (20) and a bottleneck residual unit (50); The first pooling module (31) is arranged upstream of data of the first 1x1 convolution module (41); Connecting the bottleneck residual unit (50) and a full connection layer module (60); The full connection layer module (60) is used for outputting final image data; The bottleneck edge shrinking unit (20) comprises a first bottleneck edge shrinking module (21), a second bottleneck edge shrinking module (22), a third bottleneck edge shrinking module (23) and a fourth bottleneck edge shrinking module (24) connected in sequence. The bottleneck edge shrinking module comprises at least two 1*1 convolution layers and an m*m convolution layer, wherein the m*m convolution layer is arranged between the two 1*1 convolution layers. The edge shrinking method of the bottleneck edge shrinking module comprises: G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x), wherein, conv(x) represents an m*m convolution transformation without padding on the image matrix x, m>1; c_1(x) and c_2(x) represent 1*1 convolution transformation on the image matrix x; σ(x) represents nonlinear transformation on the image matrix x; and r(x) represents edge shrinking on the image matrix x. The bottleneck residual module comprises at least three convolution layers, The expression of the bottleneck residual module can be written as: H(x)=c_2(σ(conv(σ(c_1(x)))))+x, wherein, conv(x) represents a 3*3 convolution transformation with padding=1 on the image matrix x; c_1(x) and c_2(x) represent 1*1 convolution transformation on the image matrix x, σ(x) represents nonlinear transformation on the image matrix x.

5. The method of claim 4, wherein the step of constructing the neuronal network further comprises the step of: Each bottleneck edge shrinking module removes one layer of edge feature elements, and the first bottleneck edge shrinking module (21), the second bottleneck edge shrinking module (22), the third bottleneck edge shrinking module (23) and the fourth bottleneck edge shrinking module (24) can remove four layers of edge feature elements of the image matrix, so that the number of rows and columns of the image matrix is reduced by 8. ​ 6. The method of claim 5, wherein the step of constructing the neural network comprises the step of: Any two non-edge-shrinking bottleneck residual modules in the bottleneck residual unit (50) are provided with a second pooling module (32) and a second 1x1 convolution module (42); The second pooling module (32) is arranged upstream of the data of the second 1x1 convolution module (42); The role of the pooling module arranged between the residual modules is to change the size of the image matrix.

Images