An image shrinking model, method, and apparatus based on matrix edge feature elements.

Abstract

This invention relates to an image edge reduction model, method, and apparatus based on matrix edge feature elements. The image edge reduction model includes a basic edge reduction module, which comprises at least two m*m convolutional layers. The edge reduction method of the basic edge reduction module includes: G(x) = conv_2(σ(conv_1(x))) + r(x), where conv_1(x) and conv_2(x) represent padding-free m*m convolution transformations on the image matrix x, m ≥ 1; σ(x) represents a nonlinear transformation on the image matrix x; and r(x) represents edge reduction of the image matrix x. The image edge reduction model of this invention can extract feature elements of the middle and edge pixels of the input image matrix x through padding-free convolution transformations, reducing computational load while ensuring that the influence weights of the information from the middle and edge pixels on the output are different: the influence weight of edge pixels is lower than that of the middle pixels.

Description

[0001] This invention is a divisional application of patent application number 201910894420.1, entitled "A Method and Apparatus for Image Recognition Including Surrounding Environment", filed on September 20, 2019, and of the type of application being invention. Technical Field

[0002] This invention relates to the field of information technology, and in particular to an image shrinking model, method, and apparatus based on matrix edge feature elements. Background Technology

[0003] Image recognition using deep learning is currently one of the most popular and successful applications of artificial intelligence. Deep learning has immense value in areas such as identity authentication, security, autonomous driving, and even integrated circuit design and waste sorting.

[0004] The specific steps for using deep learning for image recognition are: constructing a convolutional neural network; collecting massive amounts of images (data) and their classification labels; using this data and labels to train the convolutional neural network and build a corresponding model; and using the established model to recognize the input image in practical applications.

[0005] In recent years, with improvements and optimizations to convolutional neural network (CNN) structures, deep learning's image recognition capabilities have advanced rapidly. One of the most famous CNN structures is the Residual Network (ResNet), proposed in late 2015 by He Kaiming et al. from Microsoft Research Asia. Its recognition rate on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) has surpassed human recognition levels. Residual networks are constructed by stacking residual modules. The core idea of ​​residual networks is to replace the convolutional layers H(x) in the CNN with the residual F(x) = H(x) – x that processes the convolution and input. This increases the system's stability and avoids degradation when the neural network is too deep (too many layers). Following this approach, residual modules (including basic residual modules and bottleneck residual modules that further reduce parameters and increase layers) can be constructed. Based on these residual modules, residual neural networks with significantly more layers than previous neural network models can be built, resulting in a substantial improvement in image recognition performance.

[0006] When identifying an object in the center of an image, the value of each pixel in the image is not equal. To build a high-quality convolutional neural network to identify objects in an image, the pixels of the object itself in the center of the image should have a greater weight in influencing the parameters of the neural network than the pixels corresponding to the background around the object.

[0007] However, existing convolutional neural networks, including the residual network by He Kaiming et al., assign the same weight to all pixels in the image on the network parameters. This results in either insufficient parameters to capture enough features of objects in the image, or too many parameters leading to overfitting. At the same time, the computational cost of modeling and recognition is also greater.

[0008] However, simply cropping the image edges and removing the corresponding pixels of the surrounding background results in insufficient image information, making recognition difficult. As shown in Figures 4 and 5, the original image with the background... Figure 4a and Figure 5a As shown, after cropping the image edges, as Figure 4b and Figure 5b As shown, objects in the image are more difficult to identify than in the original image with a background.

[0009] For example, Chinese patent CN108985236A discloses a face recognition method based on a deep separable convolutional model, including the following steps: First, reading a dataset of face image samples; second, establishing a deep separable convolutional model, wherein multiple residual bottleneck modules are cascaded between adjacent convolutional modules; third, updating the parameters of the deep separable convolutional model using a gradient descent algorithm; fourth, performing face recognition using the updated deep separable convolutional model. This patent can improve recognition speed while maintaining accuracy. However, this patent does not differentiate between the weights of face images and background images, resulting in a large computational load.

[0010] For example, Chinese patent 108921788A discloses an image super-resolution method based on deep residual CNN, including: constructing a deep residual CNN model; inputting a low-resolution image into the deep residual CNN model for recognition to obtain a recognized image; and inputting the recognized image into a VGG loss network for recognition performance evaluation to obtain a super-resolution recognized image. This patent evaluates the recognition performance of the obtained image to obtain a higher-resolution recognized image, but it does not pre-distinguish the weights between the recognized image and the background image, resulting in a large computational load.

[0011] For example, Chinese patent CN 110163260 A discloses an image recognition method based on residual networks. The method includes: constructing a residual network and training the residual network using preset training samples; acquiring an image to be recognized; performing preprocessing on the image to be recognized; dividing the preprocessed image to be recognized into two non-overlapping blocks, which are then sequentially input into the residual network to obtain the predicted value of each block after passing through the residual network; and outputting a recognition result based on the predicted values ​​of the two blocks, wherein the recognition result includes whether the image to be recognized is a target image or a non-target image. This patent extracts key features through residual networks, which can reduce noise interference and identify the target image with high accuracy. However, this patent still cannot reduce the computational load of modeling and recognition.

[0012] For example, Chinese Patent 108765449A discloses an image background segmentation and recognition method based on convolutional neural networks (CNNs). This method uses CNNs to learn from sample images, training a background segmentation model and a recognition classification model, and then performs background segmentation and recognition classification based on these models. This patent combines the influence of the image background on recognition classification when applying CNNs to image recognition classification, and in the background segmentation model, the fully connected layers in the CNN are replaced with convolutional layers. This patent achieves image background segmentation using a CNN model, and then uses the image for image recognition classification to improve the accuracy of recognition classification. However, after background segmentation, information that is relatively meaningful to the background is ignored, leading to inaccurate image recognition.

[0013] For example, Chinese patent CN102694961B discloses a method and apparatus for unilaterally shrinking the edges of an image, comprising: synthesizing an original image with a key signal of the original image, wherein during the synthesis process, the original image and the object position in the key signal are aligned, and a processed foreground is obtained after synthesis; determining the position of the black edge in the processed foreground; keeping the position of the original image unchanged, moving the key signal of the original image relative to the original image in the opposite direction to the position of the black edge by a specified moving distance; synthesizing the original image and the moved key signal again; and shrinking the edges of the image obtained after the second synthesis. Although this patent enhances the display effect of the image by unilaterally shrinking the edges of the image, it describes an image enhancement method and apparatus, which does not involve image recognition, or rather, it is an image enhancement method performed after the foreground and background have been distinguished by other image recognition devices. Moreover, the shrinking action in this patent refers to directly removing the pixels at the image edges after image processing, rather than the feature elements of the matrix edges in a neural network. In neural networks, except for the matrix of the input layer which is image information, the matrices of other layers can no longer be called images, but feature elements.

[0014] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0015] To address the shortcomings of existing technologies, this invention provides an image recognition method including the surrounding environment. The method comprises at least the following steps: extracting an image matrix based on a convolutional layer; performing at least one edge-shrinking process on the image matrix; performing at least one non-edge-shrinking residual process on the edge-shrinked image matrix; and outputting the recognized image data based on a fully connected layer. Compared to existing neural networks, the neural network of this invention achieves better image recognition performance due to the inclusion of an edge-shrinking module. First, the edge-shrinking module extracts feature information representing the surrounding environment pixels at the image edges, helping to improve the recognition quality of objects in the image. Second, the edge-shrinking module reduces the number of parameters corresponding to these image edge pixel feature information parameters compared to the number of parameters related to the pixel feature information of the object to be recognized in the image center. This reduces the number of parameters in the entire neural network without reducing the number of parameters related to the pixel feature information of the object to be recognized in the image center, thereby improving the stability and recognition quality of the network. Third, the edge-shrinking module reduces the weight of these image edge pixel feature information parameters compared to the weight of the pixel feature information of the object to be recognized in the image center, thereby improving the recognition rate of objects in the image center and reducing misidentification of irrelevant objects at the image edges.

[0016] According to a preferred embodiment, the method further includes the step of: constructing an edge shrinking module; the edge shrinking method of the edge shrinking module includes: extracting the middle and / or edge feature elements of the image matrix based on the convolutional layer, removing the feature elements of the edge k layers of the image matrix, and after edge shrinking, the size of the convolutional kernel of the convolutional layer is: m = k*2 + 1, where m and k are positive integers. The advantage of the edge shrinking method of the present invention is that it can extract the feature elements of the middle and edge pixels of the input image matrix x through a padding-free convolutional transformation, which reduces the amount of computation and makes the influence weight of the information of the middle and edge pixels on the output different: the influence weight of the edge pixels is lower than the influence weight of the middle pixels.

[0017] According to a preferred embodiment, the edge-shrinking module is a basic edge-shrinking module, which includes at least two m*m convolutional layers. The edge-shrinking method of the basic edge-shrinking module includes: G(x) = conv_2(σ(conv_1(x))) + r(x).

[0018] Here, conv_1(x) and conv_2(x) represent padding-free m*m convolution transformations of the image matrix x, where m ≥ 1; σ(x) represents a non-linear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x. The advantage of the basic edge shrinking module is that it can remove pixels from the two edge layers, thereby reducing the influence weight of edge pixels to be lower than that of middle pixels.

[0019] According to a preferred embodiment, the edge reduction module is a bottleneck edge reduction module, which includes at least two 1*1 convolutional layers and one m*m convolutional layer, wherein the m*m convolutional layer is disposed between the two 1*1 convolutional layers. The edge reduction method of the bottleneck edge reduction module includes:

[0020] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0021] Here, conv(x) represents an m*m convolution transformation on the image matrix x without padding, where m>1; c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x; σ(x) represents a nonlinear transformation on the image matrix x; and r(x) represents edge shrinking on the image matrix x. The advantage or technical effect of the bottleneck edge shrinking module is that by reducing the number of channels in the image matrix through a 1×1 convolutional layer c_1, the number of parameters and computational cost of the 3×3 convolutional kernel are significantly reduced. Thus, the bottleneck edge shrinking module has fewer parameters than the corresponding basic edge shrinking module, while simultaneously increasing the corresponding network depth by 50%.

[0022] According to a preferred embodiment, the function of r(x) for removing the feature elements of the edge k layers of the image matrix is:

[0023] That is, remove rows 1 to k, columns 1 to k, rows (n-k+1) to n, and columns (n-k+1) to n from matrix x.

[0024] According to a preferred embodiment, the m*m convolutional layer of the basic edge shrinking module is a convolutional layer containing a 3x3 convolutional kernel; the m*m convolutional layer in the bottleneck edge shrinking module is a convolutional layer containing a 3x3 convolutional kernel.

[0025] According to a preferred embodiment, the method further includes: performing at least one pooling and / or convolution process on the image matrix to change the number of channels before or during the non-shrinking residual processing.

[0026] The present invention also provides an image recognition device including a surrounding environment. The device includes at least: a convolution module for extracting an image matrix based on a convolutional layer; a shrinking unit for performing at least one shrinking process on the image matrix; a residual unit for performing at least one non-shrinking residual process on the shrinked image matrix; and a fully connected layer module for outputting recognized image data based on the fully connected layer. The shrinking unit is disposed between the convolution module and the residual unit, and the residual unit is connected to the fully connected layer module.

[0027] According to a preferred embodiment, the edge shrinking unit includes several edge shrinking modules; the edge shrinking method of the edge shrinking module includes: extracting the middle and / or edge feature elements of the image matrix based on the convolutional layer, removing the feature elements of the edge k layers of the image matrix, and after edge shrinking, the size of the convolutional kernel of the convolutional layer is: m = k*2 + 1, where m and k are positive integers.

[0028] According to a preferred embodiment, the edge shrinking module includes a basic edge shrinking module and a bottleneck edge shrinking module. The basic edge shrinking module includes at least two m*m convolutional layers, and the edge shrinking method of the basic edge shrinking module includes: G(x) = conv_2(σ(con_1(x))) + r(x).

[0029] Where conv_1(x) and con_2(x) represent padding-free m*m convolution transformations of the image matrix x, where m>1; σ(x) represents a nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0030] The bottleneck shrinking module includes at least two 1*1 convolutional layers and one m*m convolutional layer, wherein the m*m convolutional layer is placed between the two 1*1 convolutional layers. The shrinking method of the bottleneck shrinking module includes: G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0031] Where conV(x) represents an m*m convolution transformation of the image matrix x without padding, where m≥1; c_1(X) and c_2(x) represent 1*1 product transformations of the image matrix x; σ(x) represents a nonlinear transformation of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0032] The present invention also provides an image shrinking model based on matrix edge feature elements, the image shrinking model including a basic shrinking module, the basic shrinking module including at least two m*m convolutional layers.

[0033] The edge-shrinking method of the basic edge-shrinking module includes: G(x) = com_2(σ(conv_1(x))) + r(x),

[0034] Where conv_1(x) and conv_2(x) represent padding-free m*m convolution transformations of the image matrix x, where m≥1; σ(x) represents nonlinear transformations of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0035] Preferably, the image shrinking model further includes a bottleneck shrinking module, which includes at least two 1*1 convolutional layers and one m*m convolutional layer, wherein the m*m convolutional layer is positioned between the two 1*1 convolutional layers.

[0036] The edge reduction method of the bottleneck edge reduction module includes:

[0037] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0038] Where conv(x) represents an m*m convolution transformation without padding on the image matrix x, where m>1; c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x; σ(x) represents a nonlinear transformation on the image matrix x; and r(x) represents a shrinking transformation on the image matrix x.

[0039] Preferably, the function r(x) for removing the feature elements of the edge k layers of the image matrix is:

[0040]

[0041] That is, remove rows 1 to k, columns 1 to k, rows (n-k+1) to n, and columns (n-k+1) to n from the original matrix x.

[0042] The present invention also provides an image shrinking method based on matrix edge feature elements. The method includes at least: extracting the middle and / or edge feature elements of the image matrix based on the convolutional layer, removing the feature elements of the edge k layers of the image matrix, and after shrinking, the size of the convolutional kernel of the convolutional layer is: m = k*2 + 1, where m and k are positive integers.

[0043] Preferably, the method further includes:

[0044] The basic edge shrinking module is used for edge shrinking.

[0045] The basic edge-shrinking module includes at least two m*m convolutional layers.

[0046] The edge shrinking method of the basic edge shrinking module includes: G(x) = conv_2(σ(conv_1(x))) + r(x),

[0047] Where conv_1(x) and conv_2(x) represent padding-free m*m convolution transformations of the image matrix x, where m≥1; σ(x) represents nonlinear transformations of the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0048] Preferably, the method further includes:

[0049] The bottleneck shrinking module is used for edge shrinking.

[0050] The bottleneck shrinking module includes at least two 1*1 convolutional layers and one m*m convolutional layer, wherein the m*m convolutional layer is positioned between the two 1*1 convolutional layers.

[0051] The edge reduction method of the bottleneck edge reduction module includes:

[0052] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0053] Where conv(x) represents an m*m convolution transformation without padding on the image matrix x, where m>1; c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x; σ(x) represents a nonlinear transformation on the image matrix x; and r(x) represents a shrinking transformation on the image matrix x.

[0054] Preferably, the function r(x) that removes the feature elements of the edge k layers of the image matrix is:

[0055]

[0056] That is, remove rows 1 to k, columns 1 to k, rows (n-k+1) to n, and columns (n-k+1) to n from the original matrix x.

[0057] Preferably, the method further includes: after performing at least one edge-shrinking process on the image matrix extracted based on the convolutional layer, performing at least one non-edge-shrinking residual process on the edge-shrinking image matrix; and outputting the recognized image data based on the fully connected layer.

[0058] The present invention also provides an image shrinking device based on matrix edge feature elements, the device comprising at least:

[0059] The convolution module is used to extract the image matrix based on the convolutional layer;

[0060] Shrinking unit, used to perform shrinking processing on the image matrix at least once;

[0061] The residual unit is used to perform at least one non-shrink residual process on the edge-shrinked image matrix.

[0062] The fully connected layer module is used to output recognized image data based on the fully connected layer.

[0063] Preferably, in the image shrinking device, the shrinking module is a bottleneck shrinking module, which includes at least two 1*1 convolutional layers and one m*m convolutional layer, wherein the m*m convolutional layer is disposed between the two 1*1 convolutional layers, and the shrinking method of the bottleneck shrinking module includes:

[0064] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x)

[0065] Where conv(x) represents an m*m convolution transformation without padding on the image matrix x, where m>1; c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x; σ(x) represents a nonlinear transformation on the image matrix x; and r(x) represents a shrinking transformation on the image matrix x. Attached Figure Description

[0066] Figure 1 This is a logical schematic diagram of the edge-shrinking method of the basic edge-shrinking module of the present invention;

[0067] Figure 2 This is a schematic diagram of the edge reduction method of the bottleneck edge reduction module of the present invention;

[0068] Figure 3 This is a logical schematic diagram of a preferred embodiment of the present invention;

[0069] Figure 4a This is a schematic diagram of one of the original background images of the present invention;

[0070] Figure 4b This is a schematic diagram of one of the edgeless images of the present invention;

[0071] Figure 5a This is another schematic diagram of the present invention with a background image; and

[0072] Figure 5b This is a schematic diagram of another edgeless image according to the present invention.

[0073] List of reference numerals

[0074] 11: First 3x3 convolutional module 12: Second 3x3 convolutional module

[0075] 13: Third 3x3 convolution module r(x): Shrinking module

[0076] 10: 5x5 convolutional module; 20: bottleneck shrinking unit.

[0077] 21: First bottleneck shrinking module; 22: Second bottleneck shrinking module

[0078] 23: Third bottleneck shrinking module 24: Fourth bottleneck shrinking module

[0079] 31: First pooling module 32: Second pooling module

[0080] 41: First 1x1 convolutional module 42: First 1x1 convolutional module

[0081] 50: Bottleneck Residual Unit; 51: First Bottleneck Residual Module

[0082] 52: Second bottleneck residual module 53: Third bottleneck residual module

[0083] 54: Fourth Bottleneck Residual Module 55: Fifth Bottleneck Residual Module

[0084] 60: Fully Connected Layer Module Detailed Implementation

[0085] The following is a detailed explanation with reference to the accompanying drawings.

[0086] In this invention, padding refers to the space between the element's border and its content defined by the padding property. When the element's padding is cleared, the released area will be filled with the element's background color. The padding property alone can be used to change the padding on the top, bottom, left, and right sides.

[0087] Residual networks: Residual networks are characterized by their ease of optimization and the ability to improve accuracy by increasing their depth considerably. Their internal residual blocks utilize skip connections, mitigating the vanishing gradient problem that arises when increasing depth in deep neural networks.

[0088] Basic residual module: A module in a residual network; a module consists of at least two convolutional layers, and the module can be expressed as: H(x)=conv_2(σ(conv_1(x)))+x, where conv_1(x) and conv_2(x) represent 3×3 convolution transformations with padding=1 on the image matrix x; σ(x) represents nonlinear transformations on the image matrix x.

[0089] Bottleneck Residual Module: A module in the residual network; the module consists of at least three convolutional layers, and the module can be expressed as: H(x)=c_2(σ(conv(σ(c_1(x)))))+x, where conv(x) represents a 3×3 convolution transformation with padding=1 on the image matrix x; c_1(x) and c_2(x) represent a 1×1 convolution transformation on the image matrix x, and σ(x) represents a nonlinear transformation on the image matrix x.

[0090] Convolutional layers: These are used to extract features from an image, and the weights of the convolutional kernels are learnable. Each convolutional layer consists of several convolutional units, and the parameters of each unit are optimized through backpropagation. The purpose of convolution is to extract different features from the input. The first convolutional layer may only extract low-level features such as edges, lines, and corners, while more layers can iteratively extract more complex and abstract features from these low-level features.

[0091] Pooling layers: used to reduce the input size of the next layer, thereby reducing the computational cost and the number of parameters, which can prevent network overfitting.

[0092] Fully connected layers: Each neuron in a fully connected layer is fully connected to all neurons in the layer preceding it. Fully connected layers can integrate local information with class discriminative power from convolutional or pooling layers. To improve CNN network performance, the activation function for each neuron in a fully connected layer is typically the ReLU function. The output of the last fully connected layer is passed to an output, which can be achieved using softmax logistic regression; this layer is also known as a softmax layer.

[0093] Example 1

[0094] In existing technologies, the core convolutional layer used in residual networks pre-padding the image. The input and output matrices of this convolutional layer are of the same size, and the residual operation F(x) = H(x) - x is performed. Here, F(x) is the residual function, which refers to the difference between the module outputs, H(x) is the output matrix of the module, and X is the input matrix (image matrix) of the module.

[0095] In existing technologies, only Chinese patent CN102694961B mentions "edge shrinking," but this patent's "edge shrinking" refers to directly removing pixels from the image edges after image processing. This is completely different from the edge shrinking definition of this invention, and their purpose is also unrelated. The edge shrinking of this invention refers to removing feature elements from the edges of the matrix in a neural network. In this neural network, except for the input layer matrix which is an image, the matrices of other layers can no longer be considered images but rather feature elements. The edge shrinking of this invention is used in conjunction with two or three padding-free convolutions in the neural network to ensure that the size of the edge-shrinking matrix is ​​the same as the size of the convolutional matrix.

[0096] This invention proposes a shrinking method based on residual networks. The shrinking method of this invention refers to not adding padding to the image, making the output matrix of this convolutional layer smaller than the input matrix. In this case, during the residual operation, the input image matrix needs to be shrunk. That is, G(x) = H(x) - r(x), where r(x) is the shrinking operation performed on the input image matrix x. The shrinking method includes: extracting the middle and / or edge feature elements of the image matrix based on the convolutional layer, and removing the feature element information of the edge k layers of the image matrix. After shrinking, the size of the convolutional kernel of the convolutional layer is: m = k*2 + 1, where m and k are positive integers. The advantage of the shrinking method of this invention is that it can extract the middle and edge feature elements of the input image matrix x through convolution, so that the weights of the middle and edge feature elements on the output are different: the influence weight of the edge feature elements is lower than that of the middle feature elements.

[0097] A device carrying a shrinking operation method is defined as a shrinking module. Shrinking modules include a basic shrinking module and a bottleneck shrinking module.

[0098] The basic edge-shrinking module consists of at least two m*m convolutional layers. The edge-shrinking method of the basic edge-shrinking module is: G(x) = conv_2(σ(conv_1(x))) + r(x). Here, conv_i(x) and conv_2(x) represent padding-free m*m convolution transformations on the image matrix x, where m ≥ 1. σ(x) represents a non-linear transformation on the image matrix x. For example, σ(x) is the ReLU function. r(x) represents edge-shrinking of the image matrix x.

[0099] When the convolutional layer is preferably a convolutional layer containing a 3x3 convolutional kernel, conv_1(x) and conv_2(x) represent performing a 3x3 convolution transformation on the image matrix x without padding. r(x) represents removing the feature elements of its two edge layers to ensure that the scale of r(x) is the same as the scale of conv_2(σ(conv_1(x))). The advantage of the basic edge shrinking module is that it can remove the feature elements of the two edge layers, thereby reducing the influence weight of edge feature elements, making the influence weight of edge feature elements lower than that of middle feature elements, that is, the influence weight of edge pixels is lower than that of middle pixels. The basic edge shrinking module of this invention must contain at least two convolutional layers; otherwise, its edge shrinking and residual operations can be merged with the convolutional layers, making the operation linear and failing to improve the recognition effect.

[0100] like Figure 1 As shown, according to a preferred embodiment, the convolutional layer of the basic edge-shrinking module is preferably a convolutional layer containing a 3x3 convolutional kernel. Figure 1 As shown, the input image matrix x undergoes a first 3x3 convolution transformation without padding in the first 3x3 convolution module 11, followed by ReLU (Rectified Linear Unit) transformation. s After a linear transformation by the activation function, the image matrix is ​​input into the second 3x3 convolution module 12. The second 3x3 convolution module 12 performs a second, padding-free 3x3 convolution transformation on the image matrix, and then uses r(x) to shrink the edges of the input image matrix, removing the feature elements of the two edge layers. According to a preferred embodiment, the edge-shrinking module is a bottleneck edge-shrinking module. The bottleneck edge-shrinking module includes at least two 1x1 convolutional layers and one mxm convolutional layer. The mxm convolutional layer is positioned between the two 1x1 convolutional layers. The edge-shrinking method of the bottleneck edge-shrinking module includes:

[0101] G(x)=c_2(σ(conv(σ(c_1(x)))))+r(x).

[0102] Where conv(x) represents an m*m convolution transformation without padding on the image matrix x, where m>1. c_1(x) and c_2(x) represent a 1*1 convolution transformation on the image matrix x, σ(x) represents a nonlinear transformation on the image matrix x, and r(x) represents edge shrinking on the image matrix x.

[0103] Preferred, such as Figure 2 As shown, in the bottleneck shrinking module, the second convolutional layer is a convolutional layer with a 3x3 kernel. Therefore, conv(x) represents a 3x3 convolution transformation without padding on the image matrix x.

[0104] Specifically, such as Figure 2As shown, a third 3x3 convolutional module 13 with a 3x3 kernel is positioned between the first 1x1 convolutional module 41 and the second 1x1 convolutional module 42. The image matrix X is input to the first 1x1 convolutional module 41 for the first 1x1 convolutional transformation without padding. The image matrix output from the first 1x1 convolutional module 41 is then processed by ReLU (Rectified Linear Unit)... s) After a linear transformation by the activation function, the image matrix is ​​input into the third 3x3 convolutional module 13 for a padding-free 3x3 convolutional transformation. The image matrix output from the third 3x3 convolutional module 13 is then linearly transformed by the ReLU (Rectified Linear Units) activation function and input into the second 1x1 convolutional module 42 for a second padding-free 1x1 convolutional transformation. The image matrix output from the second 1x1 convolutional module 42 is edge-shrinked by r(x), removing feature elements from the edge layer to ensure that the scale of r(x) is the same as the scale of c_2(σ(conv(σ(c_1(x)))))). Compared with the basic edge-shrinking module, the advantage or technical effect of the bottleneck edge-shrinking module is that by reducing the number of channels in the image matrix through a 1×1 convolutional layer c_1, the number of parameters and computational cost of the 3×3 convolutional kernel are significantly reduced, thus making the bottleneck edge-shrinking module have fewer parameters than the corresponding basic edge-shrinking module, while increasing the corresponding network depth by 50%. Reducing the number of parameters can improve the stability of the entire neural network, while increasing the network depth can improve the image recognition rate. Practice shows that replacing the basic edge-shrinking module with a bottleneck edge-shrinking module can achieve better recognition results.

[0105] Preferably, in the basic and bottleneck shrinking modules, the convolutional kernels of the convolutional modules are not limited to 3x3 kernels; they can also be other odd-sized kernels, such as 5x5 or 7x7 kernels. The shrinking operation of the shrinking module changes accordingly with the convolutional kernel, adjusting the number of feature element layers removed to ensure an appropriate scale for r(x). In actual shrinking operations, selecting a convolutional module with a 3x3 kernel is most beneficial in avoiding overfitting and also offers relatively high computational efficiency.

[0106] Preferably, the function of r(x) in this invention for removing the feature elements of the edge k layers of the image matrix is:

[0107]

[0108] That is, remove rows 1 to k, columns 1 to k, and rows (n-k+1) to n and columns (n-k+1) to n from matrix x. Here, the image matrix X is A. nn matrix.

[0109] Preferably, the r(x) function after removing the feature elements of the edge layer can be written as: This means removing the first row, first column, last row, and last column of the original matrix x. Similarly, the r(x) function for removing the two layers of feature elements at the edges removes the first and second rows, first and second columns, and last two rows and last two columns of the original matrix x. Preferably, constructing a neural network using basic edge-shrinking modules and bottleneck edge-shrinking modules can reduce the influence weight of protruding edges, thereby achieving better image recognition results.

[0110] Preferably, the neural network includes at least convolutional modules, edge-shrinking modules, residual modules, and fully connected modules. Preferably, the neural network also includes pooling modules, as well as other basic linear / nonlinear transformations.

[0111] Preferably, a convolutional layer is placed at the image input end of the neural network to extract basic geometric features of the image, such as boundaries in various directions. Adjacent to the convolutional layer is a shrinking unit containing several shrinking modules. The number of shrinking modules is determined by the size of the input image and the relative size of the surrounding environment of the object to be identified in the image. The shrinking unit is used to truncate edge layer feature elements in the image matrix. Adjacent to the shrinking unit is a non-shrinking residual unit. The residual unit includes several residual modules. The residual modules include basic residual modules and bottleneck residual modules. Adjacent to the residual modules is a fully connected layer module. The residual modules are placed after the shrinking modules and before the fully connected layer to extract relatively abstract features of the non-edge regions of the image to be identified. The fully connected layer module is used as the last layer of the neural network to extract feature information for the final image classification.

[0112] Compared with existing neural networks, the neural network of this invention achieves better image recognition performance due to the inclusion of an edge-shrinking module. First, the edge-shrinking module extracts feature information from pixels representing the surrounding environment at the image edges, helping to improve the recognition quality of objects in the image. Second, the edge-shrinking module reduces the number of parameters corresponding to these image edge pixel feature information to fewer parameters for the pixel feature information of the object to be recognized in the image center. This reduces the total number of parameters in the neural network without decreasing the number of parameters for the pixel feature information of the object to be recognized in the image center, thus improving the network's stability and recognition quality. Third, the edge-shrinking module reduces the weight of these image edge pixel feature information to less than the weight of the pixel feature information of the object to be recognized in the image center, thereby improving the recognition rate of objects in the image center and reducing misidentification of irrelevant objects at the image edges.

[0113] Preferably, in the neural network, a pooling layer can be set between the edge-shrinking module and the non-edge-shrinking residual module to shrink the image, or a 1x1 convolutional layer can be set to change the number of channels.

[0114] Preferably, placing the edge-shrinking module before the non-edge-shrinking residual module has a relative advantage. Pixels corresponding to the surrounding environment are located at the edges of the image matrix, while the inner feature elements of the image matrix correspond to the pixels of the object to be identified. The feature elements at the very edge of the input image matrix have a low weighted impact on the final output; removing them not only reduces computation but also reduces the influence of image edges. Feature elements near the center and exactly in the center of the input image are equally important for object recognition. Using the non-edge-shrinking residual module preserves these feature elements unaffected. Therefore, placing the edge-shrinking module before the non-edge-shrinking residual module increases the contrast between information in the center and edge regions of the image, highlighting the information in the center of the image.

[0115] Figure 3 This is a specific embodiment of the neural network of the present invention. Figure 3 The example shown is for recognizing digits (0-10) in a single-channel image (commonly known as a black and white image) of a 32x32 image matrix. Figure 3 In this context, M(32x32) and CH(1) represent an image matrix of 32x32 with 1 channel. And so on.

[0116] A 5x5 convolutional module 10 with a 5x5 kernel is connected to a bottleneck shrinking unit 20. A first pooling module 31 and a first 1x1 convolutional module 41 are disposed between the bottleneck shrinking unit 20 and the bottleneck residual unit 50. The first pooling module 31 is positioned upstream of the data of the first 1x1 convolutional module 41. A second pooling module 32 and a second 1x1 convolutional module 42 can be disposed between any two non-shrinking residual modules in the bottleneck residual unit 50. The second pooling module 32 is positioned upstream of the data of the second 1x1 convolutional module 42. The bottleneck residual unit 50 is connected to a fully connected layer module 60. The fully connected layer module 60 outputs the final image data.

[0117] Preferably, the bottleneck shrinking unit 20 includes a first bottleneck shrinking module 21, a second bottleneck shrinking module 22, a third bottleneck shrinking module 23, and a fourth bottleneck shrinking module 24 connected in sequence. Each bottleneck shrinking module removes one layer of edge feature elements, and the four bottleneck shrinking modules can remove four layers of feature elements from the edge of the image matrix, which reduces the number of rows and columns of the image matrix by 8 each. The size of the image matrix changes from 28×28 at the input of the bottleneck shrinking unit 20 to 20×20 at the output of the unit.

[0118] Preferably, the residual modules within the non-shrinking bottleneck residual unit 50 are all non-shrinking residual modules. Preferably, the non-shrinking bottleneck residual unit 50 includes a first bottleneck residual module 51, a second bottleneck residual module 52, a third bottleneck residual module 53, a fourth bottleneck residual module 54, and a fifth bottleneck residual module connected in sequence. A second pooling module 32 and a second 1x1 convolution module 42 are disposed between the third bottleneck residual module 53 and the fourth bottleneck residual module 54. Preferably, the pooling module between the bottleneck residual modules serves to change the size of the image matrix. For example, Figure 3 The pooling module halves the number of rows and columns of the image matrix and adds a non-linear layer to the network. The 1×1 convolution module changes the number of image channels. For example, Figure 3 The number of image channels has been doubled.

[0119] like Figure 3 As shown, the neural network method for image recognition including the surrounding environment of the present invention includes the following steps:

[0120] S1: Extracting the image matrix based on convolutional layers;

[0121] S2: Perform at least one edge shrinking process on the image matrix;

[0122] S3: Perform at least one non-shrink residual processing on the image matrix after edge shrinking;

[0123] S4: Image data recognized based on the output of the fully connected layer.

[0124] This invention provides a detailed description of image recognition methods, including those related to the surrounding environment.

[0125] A 5x5 convolution module 10 with a 5x5 kernel extracts the image matrix, i.e., the geometric features of the image, from the input image data. The output matrix of the 5x5 convolution module 10 changes to M(28x28), CH(16). The first bottleneck shrinking module 21, the second bottleneck shrinking module 22, the third bottleneck shrinking module 23, and the fourth bottleneck shrinking module 24 sequentially shrink the edge feature element layers of the image matrix, removing feature elements from one edge layer at a time, for a total of four edge feature elements. Accordingly, the image matrix changes from M(28x28), CH(16) to M(20x20), CH(16). The first pooling module 31 shrinks the image through pooling, changing the image matrix from M(20x20), CH(16) to M(10x10), CH(16). The first 1x1 convolution module 41 performs a convolution transformation on the image matrix to change the number of channels, changing the number of channels from 16 to 32. The first bottleneck residual module 51, the second bottleneck residual module 52, and the third bottleneck residual module 53 sequentially extract features from the image matrix. The size of the image matrix remains unchanged during this process. The second pooling module 32 shrinks the image matrix through pooling, changing the image matrix from M(10x10), CH(32) to M(5x5), CH(16). The second 1x1 convolution module 42 performs convolution transformation on the image matrix to change the number of channels, from 32 to 64. At this point, the image matrix is ​​M(5x5), CH(64). The fourth bottleneck residual module 54 and the fifth bottleneck residual module 55 perform bottleneck residual processing on the image matrix, and the processed image matrix is ​​input into the fully connected layer module 60. The fully connected layer module 60 fully connects each neuron in the image matrix to all neurons in its previous layer, extracting the feature information for final image classification. The fully connected layer module 60 outputs the integrated image recognition information.

[0126] Preferably, after constructing the neural network, a number of training samples are input into the neural network for deep learning training. After extensive training, the neural network can accurately identify images, including the environment. Furthermore, this significantly reduces the computational load of image recognition.

[0127] Example 2

[0128] This embodiment is a further supplement and explanation of the foregoing embodiment, and repeated content will not be repeated.

[0129] This embodiment provides an image recognition device including the surrounding environment, characterized in that the device includes at least:

[0130] The convolution module is used to extract the image matrix based on the convolutional layer;

[0131] Shrinking unit, used to perform shrinking processing on the image matrix at least once;

[0132] The residual unit is used to perform at least one non-shrink residual process on the edge-shrinked image matrix.

[0133] A fully connected layer module is used to output image data for recognition based on the fully connected layer output. The edge-shrinking unit is located between the convolutional module and the residual unit. That is, the convolutional module is located at the data input end and connected to the edge-shrinking unit to input the image matrix extracted based on the convolutional layer into the edge-shrinking unit. The edge-shrinking unit is connected to the residual unit to input the image matrix with edge layer elements removed into the residual unit. The residual unit is connected to the fully connected layer module and is used to extract relatively abstract features of the non-edge regions of the image to be recognized, and input the extracted relatively abstract features of the non-edge region image matrix into the fully connected layer module. The fully connected layer module is used as the last layer of the neural network to extract and output the feature information for the final image classification.

[0134] Compared with existing neural networks, the neural network of this invention achieves better image recognition performance due to the inclusion of an edge-shrinking module. First, the edge-shrinking module extracts feature information from pixels representing the surrounding environment at the image edges, helping to improve the recognition quality of objects in the image. Second, the edge-shrinking module reduces the number of parameters corresponding to these image edge pixel feature information to fewer parameters for the pixel feature information of the object to be recognized in the image center. This reduces the total number of parameters in the neural network without decreasing the number of parameters for the pixel feature information of the object to be recognized in the image center, thus improving the network's stability and recognition quality. Third, the edge-shrinking module reduces the weight of these image edge pixel feature information to less than the weight of the pixel feature information of the object to be recognized in the image center, thereby improving the recognition rate of objects in the image center and reducing misidentification of irrelevant objects at the image edges.

[0135] Preferably, the various modules of the present invention can be hardware such as servers, processors and / or integrated chips, or servers carrying corresponding computing and processing methods.

[0136] For example, a convolution module can be a server, processor, or integrated chip with convolution transformation functionality. A shrinking unit includes several servers, processors, or integrated chips with shrinking processing functionality. A shrinking module is a server, processor, or integrated chip carrying a shrinking calculation method. A residual unit includes several servers, processors, or integrated chips with residual processing functionality. A residual module is a server, processor, or integrated chip carrying a residual processing algorithm. A fully connected layer module is a server, processor, or integrated chip carrying a fully connected layer calculation method.

[0137] The edge shrinking unit includes several edge shrinking modules. The edge shrinking method of the edge shrinking module includes: extracting the middle and / or edge feature elements of the image matrix based on the convolutional layer, removing the feature elements of the edge k layers of the image matrix, and after edge shrinking, the size of the convolutional kernel of the convolutional layer is: m = k*2 + 1.

[0138] Preferably, the edge shrinking module includes a basic edge shrinking module and a bottleneck edge shrinking module. The basic edge shrinking module includes at least two m*m convolutional layers. The edge shrinking method of the basic edge shrinking module includes: G(x) = conv_2(σ(conv_1(x))) + r(x). Wherein, conv_1(x) and conv_2(x) represent padding-free m*m convolution transformations on the image matrix x, m>1; σ(x) represents nonlinear transformations on the image matrix x; and r(x) represents edge shrinking of the image matrix x.

[0139] The bottleneck edge reduction module includes at least two 1*1 convolutional layers and one m*m convolutional layer. The m*m convolutional layer is positioned between the two 1*1 convolutional layers. The edge reduction method of the bottleneck edge reduction module includes: G(x) = c_2(σ(conv(σ(c_1(x)))))+r(x). Here, conv(x) represents a padding-free m*m convolution transformation on the image matrix x, where m ≥ 1; c_1(x) and c_2(x) represent 1*1 product transformations on the image matrix x; σ(x) represents a non-linear transformation on the image matrix x; and r(x) represents edge reduction on the image matrix x.

[0140] The function r(x) for removing the feature elements of the edge k layers of the image matrix is:

[0141] That is, remove rows 1 to k, columns 1 to k, rows (n-k+1) to n, and columns (n-k+1) to n from the original matrix x.

[0142] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; terms such as "preferredly," "according to a preferred embodiment," or "optionally" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. An image edge reduction method based on an image edge reduction model using matrix edge feature elements, characterized in that, The image shrinking model includes a basic shrinking module. The basic edge-shrinking module includes at least two layers. Convolutional layer The shrinking method of the basic shrinking module includes: , in, This indicates that the image matrix x is applied without padding. Convolution transformation, σ(x) represents a nonlinear transformation of the image matrix x; r(x) represents the shrinking of the image matrix x. The r(x) removes the edges of the image matrix The function of the feature elements of the layer is: ， That is, remove the first element of the original matrix x. Okay, number Column, number row and number List.

2. The image edge reduction method based on the image edge reduction model of matrix edge feature elements according to claim 1, characterized in that, The image shrinking model also includes a bottleneck shrinking module, which comprises at least two layers. Convolutional layer and one layer Convolutional layers, in which Convolutional layers are set in two Between convolutional layers, The edge reduction method of the bottleneck edge reduction module includes: Where conv(x) represents applying padding-free processing to the image matrix x. Convolution transformation c_1(x) and c_2(x) represent the operations performed on the image matrix x. Convolution transformation, σ(x) represents a nonlinear transformation of the image matrix x; r(x) represents the shrinking of the image matrix x.

3. An image edge reduction method based on matrix edge feature elements, characterized in that, The method includes at least: extracting middle and / or edge feature elements of the image matrix based on convolutional layers, removing the feature elements of the edge k layers of the image matrix, and after edge shrinking, the size of the convolutional kernel of the convolutional layer is: ,in, It is a positive integer; Remove the edges of the image matrix The function r(x) of the feature elements of the layer is: ， That is, remove the first element of the original matrix x. Okay, number Column, number row and number List; The basic edge-shrinking module is used for edge shrinking. The basic edge-shrinking module includes at least two layers. Convolutional layer The shrinking method of the basic shrinking module includes: , in, This indicates that the image matrix x is applied without padding. Convolution transformation, σ(x) represents a nonlinear transformation of the image matrix x; r(x) represents the shrinking of the image matrix x.

4. The image edge reduction method based on matrix edge feature elements according to claim 3, characterized in that, The method further includes: The bottleneck shrinking module is used for edge shrinking. The bottleneck shrinking module includes at least two layers. Convolutional layer and one layer Convolutional layers, in which Convolutional layers are set in two Between convolutional layers, The edge reduction method of the bottleneck edge reduction module includes: Where conv(x) represents applying padding-free processing to the image matrix x. Convolution transformation c_1(x) and c_2(x) represent the operations performed on the image matrix x. Convolution transformation, σ(x) represents a nonlinear transformation of the image matrix x; r(x) represents the shrinking of the image matrix x.

5. The image edge reduction method based on matrix edge feature elements according to claim 3 or 4, characterized in that, The method further includes: After performing at least one edge-shrinking process on the image matrix extracted based on the convolutional layer, the edge-shrinking image matrix is ​​subjected to at least one non-edge-shrinking residual process. Image data is recognized based on the output of the fully connected layer.

6. An image shrinking device based on matrix edge feature elements, characterized in that, The device includes at least: The convolution module is used to extract the image matrix based on the convolutional layer; Shrinking unit, used to perform shrinking processing on the image matrix at least once; The residual unit is used to perform at least one non-shrink residual process on the edge-shrinked image matrix. The fully connected layer module is used to output recognized image data based on the fully connected layer. The edge-shrinking unit includes several edge-shrinking modules. The edge-shrinking module is a bottleneck edge-shrinking module, which includes at least two layers. Convolutional layer and one layer Convolutional layers, in which Convolutional layers are set in two Between convolutional layers, The edge reduction method of the bottleneck edge reduction module includes: Where conv(x) represents applying padding-free processing to the image matrix x. Convolution transformation c_1(x) and c_2(x) represent the operations performed on the image matrix x. Convolution transformation, σ(x) represents a nonlinear transformation of the image matrix x; r(x) represents the shrinking of the image matrix x; The r(x) removes the edges of the image matrix The function of the feature elements of the layer is: ， That is, remove the first element of the original matrix x. Okay, number Column, number row and number List.

Images