An image recognition method, device and storage medium for environmental governance supervision

Abstract

The application discloses an image recognition method and device for environmental governance supervision and a storage medium, wherein an improved pyramid fusion method is used to preprocess two input images by using local feature information and global feature information of the images, then a first light-weight attention expansion network model and a second light-weight attention expansion network model are used to output feature vectors D1 and D2, the distance between the two images is calculated, normalization operation is performed according to the distance between the two images, the similarity of the two images is obtained, and whether the two images are similar is judged. In the case of improving the image similarity detection accuracy, the network calculation amount and parameter amount are greatly reduced. For the environmental governance supervision system, the model calculation amount is large, the model is not convenient to deploy on the mobile terminal, the image has the problems of uneven illumination, distortion and blur, and different angles, and the application has good robustness and practicability.

Description

TECHNICAL FIELD

[0001] The application relates to an image recognition method, device and storage medium for environmental governance supervision, and belongs to the technical field of image processing. BACKGROUND

[0002] The image processing technology is used for similarity detection of two images and evaluation of the work of supervisors, so that the environmental governance supervision and work efficiency are improved. However, in actual application, the similarity detection of similar scene images faces many challenges, and how to quickly and accurately detect whether the pictures of the environmental governance sites are similar is crucial to the environmental governance supervision.

[0003] Therefore, how to improve the precision of image similarity detection and shorten the detection time on the basis of ensuring the precision of similarity, reduce the cloud computing power to improve the image detection efficiency, and ensure the accuracy of image scene matching is a technical problem that needs to be solved by the technical personnel in the field. SUMMARY

[0004] Objective: In order to overcome the shortcomings in the prior art, the application provides an image recognition method, device and storage medium for environmental governance supervision, which can improve the accuracy of image matching and is more efficient, achieves excellent effect on rural environment data set, has good practicality on rural environment supervision platform, and has very important significance for improving rural governance supervision.

[0005] Technical scheme: To solve the above technical problems, the technical scheme adopted by the application is:

[0006] In a first aspect, an image recognition method for environmental governance supervision comprises the following steps:

[0007] Step 1: Perform median filtering on the input two images A1 and A2 to obtain two processed images B1 and B2, and simultaneously perform mean filtering on the input two images A1 and A2 to obtain two processed images , .

[0008] Step 2: Perform image fusion processing on images B1 and , B2 and through a pyramid fusion method to output images C1 and C2.

[0009] Step 3: Input images C1 and C2 into an improved first lightweight attention dilated network model and a second lightweight attention dilated network model, respectively, to output two one-dimensional feature vectors D1 and D2.

[0010] Step 4: Calculate the Euclidean distance of the two one-dimensional feature vectors D1 and D2 to obtain the distance between the two images, and then perform normalization operation according to the distance between the two images to obtain the similarity of the two images. When the similarity is greater than a threshold, it is judged that the two images are similar.

[0011] As a preferred solution, it further comprises:

[0012] Step 5: Perform cross-entropy operation according to the output result of the normalization operation and the real label to obtain a loss function, and fit the first lightweight attention dilated network model and the second lightweight attention dilated network model according to the loss function to obtain the updated first lightweight attention dilated network model and the second lightweight attention dilated network model.

[0013] As a preferred solution, the calculation formula of the median filter in step 1 is as follows:

[0014] .

[0015] Wherein, x , y are the pixel point positions in the original image matrix, , are the original image and the processed image, indicates the median value in a given pixel set.

[0016] indicates the pixel centered on . indicates the neighborhood coordinate pixel position, indicates the two-dimensional image template.

[0017] As a preferred solution, the calculation formula of the mean filter in step 1 is as follows:

[0018] .

[0019] Wherein, M indicates the mean filter, I indicates the input image, indicates the pixel coordinates in the original image matrix, u , v indicate the offset of the mean filter in x , y axis direction, k＇ indicates the value range of the offset.

[0020] As a preferred solution, the step 2 specifically comprises:

[0021] Step 2-1: respectively convert the image B1 and Place image B1 and Resized to the size corresponding to each level of the pyramid.

[0022] Step 2-2: Local feature extraction is performed on the image B1 reduced in size at each level of the pyramid using the Laplace algorithm, and the image B1 reduced in size at each level of the pyramid Global feature extraction is performed using the Mobilenetv2 backbone extraction network.

[0023] Step 2-3: The local features of the image B1 at each level of the pyramid and the global features of the image are fused according to each level of the pyramid to obtain the feature vector of each level of the pyramid.

[0024] Step 2-4: The feature vectors of each level of the pyramid are fused layer by layer to obtain the image C1 of the original image size.

[0025] Step 2-5: Place image B2 and Resized to the size corresponding to each level of the pyramid.

[0026] Step 2-6: Local feature extraction is performed on the image B2 reduced in size at each level of the pyramid using the Laplace algorithm, and the image B2 reduced in size at each level of the pyramid Global feature extraction is performed using the Mobilenetv2 backbone extraction network.

[0027] Step 2-7: The local features of the image B2 at each level of the pyramid and the global features of the image are fused according to each level of the pyramid to obtain the feature vector of each level of the pyramid.

[0028] Step 2-8: The feature vectors of each level of the pyramid are fused layer by layer to obtain the image C2 of the original image size.

[0029] As a preferred solution, the first lightweight attention expansion network model and the second lightweight attention expansion network model both include a first convolution unit, a first reverse residual unit, a second reverse residual unit, an average pooling unit and a second convolution unit connected in sequence.

[0030] The first convolution unit includes a convolution layer, a batch normalization layer and a function activation layer connected in sequence, and the convolution layer is set to have a convolution kernel of 3x3x3x32, a step of 2 and an input size of 224x224x3.

[0031] ​The first inverse residual unit comprises a first inverse residual block, two second inverse residual blocks, three third inverse residual blocks and a fourth inverse residual block, and the first inverse residual block is connected in series with the two second inverse residual blocks, the three third inverse residual blocks and the fourth inverse residual block in sequence.

[0032] The second inverse residual unit comprises three fifth inverse residual blocks, three sixth inverse residual blocks, two seventh inverse residual blocks and two eighth inverse residual blocks, and the three fifth inverse residual blocks are connected in series with the three sixth inverse residual blocks, the two seventh inverse residual blocks and the two eighth inverse residual blocks in sequence.

[0033] The average pooling unit comprises an average pooling layer, and the average pooling layer is configured to have a pooling kernel of 7*7 and an input size of 7*7*320.

[0034] The second convolution unit comprises convolution layers, batch normalization layers and function activation layers connected in sequence, and the convolution layers are configured to have a convolution kernel of 1*1*1280*1000, a step length of 1 and an input size of 1*1*1280.

[0035] As a preferred solution, the first inverse residual block, the second inverse residual block, the third inverse residual block and the fourth inverse residual block each comprise pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers, function activation layers, depth convolution layers with a convolution kernel of 3*3 and a step length of 2, batch normalization layers, function activation layers, pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers and linear activation function layers connected in sequence.

[0036] The fifth inverse residual block and the sixth inverse residual block each comprise pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers, function activation layers, depth convolution layers with a convolution kernel of 3*3, a step length of 2 and an expansion system d, batch normalization layers, function activation layers, pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers and linear activation function layers connected in sequence.

[0037] The seventh inverse residual block and the eighth inverse residual block each comprise pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers, function activation layers, depth convolution layers with a convolution kernel of 3*3 and an expansion system d, batch normalization layers, function activation layers, an improved channel attention module, pointwise convolution layers with a convolution kernel of 1*1, batch normalization layers and linear activation function layers connected in sequence.

[0038] As a preferred scheme, the improved channel attention module comprises: the output end of a first pointwise convolution layer with a convolution kernel of 1*1 is connected with the input end of a depth convolution layer, the output end of the depth convolution layer is connected with the input end of a Scale layer, an average pooling layer and a maximum pooling layer respectively, the output end of the average pooling layer is sequentially connected with a first full connection layer, a function activation layer and a second full connection layer, the output end of the maximum pooling layer is sequentially connected with a third full connection layer, a function activation layer and a fourth full connection layer, the output end of the second full connection layer and the fourth full connection layer is sequentially connected with a summation layer and an activation layer and then connected with the input end of the Scale layer, the output end of the Scale layer is connected with the input end of a second pointwise convolution layer with a convolution kernel of 1*1, the output end of the second pointwise convolution layer with a convolution kernel of 1*1 is connected with the input end of the improved channel attention module and the input end of the summation layer respectively; the Scale layer is used for weighting the normalized weight to the feature of each channel.

[0039] As a preferred scheme, the function activation layer in the first reverse residual unit and the second reverse residual unit adopts ReLU6.

[0040] In a second aspect, a computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the image recognition method for environmental governance supervision according to any one of the first aspect.

[0041] In a third aspect, a computer device comprises:

[0042] A memory is configured to store instructions.

[0043] A processor is configured to execute the instructions, so that the computer device performs the operations of the image recognition method for environmental governance supervision according to any one of the first aspect.

[0044] Beneficial effects: the image recognition method for environmental governance supervision, the device and the storage medium provided by the application have good robustness and practicability in the case of improving the image similarity detection accuracy, greatly reducing the network calculation amount and parameter amount, and solving the problems of large model calculation amount, inconvenient model deployment on mobile terminals, uneven illumination, distortion and blur and different angles of images in the environmental governance supervision system. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 FIG. 1 is a flowchart of the image recognition method for environmental governance supervision.

[0046] Figure 2 FIG. 2 is a system diagram of the image recognition method for environmental governance supervision.

[0047] Figure 3A structural schematic diagram of a lightweight attention expansion network model proposed in the present application.

[0048] Figure 4 A structural schematic diagram of a first reverse residual block unit.

[0049] Figure 5 A structural schematic diagram of a second reverse residual block unit.

[0050] Figure 6 A structural schematic diagram of a first reverse residual block, a second reverse residual block, a third reverse residual block, and a fourth reverse residual block.

[0051] Figure 7 A structural schematic diagram of a fifth reverse residual block and a sixth reverse residual block.

[0052] Figure 8 A structural schematic diagram of a seventh reverse residual block and an eighth reverse residual block.

[0053] Figure 9 A structural schematic diagram of an improved channel attention module.

[0054] Figure 10 A schematic diagram of input data of a spiral line composed of m points in a 2-dimensional space.

[0055] Figure 11 A schematic diagram of output data with an output dimension of 2 in a 2-dimensional space after input data is transformed by ReLU6.

[0056] Figure 12 A schematic diagram of output data with an output dimension of 3 in a 2-dimensional space after input data is transformed by ReLU6.

[0057] Figure 13 A schematic diagram of output data with an output dimension of 5 in a 2-dimensional space after input data is transformed by ReLU6.

[0058] Figure 14 A schematic diagram of output data with an output dimension of 15 in a 2-dimensional space after input data is transformed by ReLU6.

[0059] Figure 15 A schematic diagram of output data with an output dimension of 30 in a 2-dimensional space after input data is transformed by ReLU6. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0061] The application will be further described in connection with specific embodiments.

[0062] Embodiment 1:

[0063] As shown in the figure, the embodiment introduces an image recognition method for environmental governance supervision, including the following steps: Figures 1-2

[0064] Step 1: The input two images A1, A2 are respectively subjected to median filtering to obtain two processed images B1, B2, and the input two images A1, A2 are respectively subjected to mean filtering to obtain two processed images , .

[0065] Step 2: images B1 and , B2 and are respectively subjected to image fusion processing through a pyramid fusion method, and output images C1, C2.

[0066] Step 3: images C1, C2 are respectively input into an improved first lightweight attention dilated network model and a second lightweight attention dilated network model, and two one-dimensional feature vectors D1 and D2 are respectively output.

[0067] Step 4: the Euclidean distance of the two one-dimensional feature vectors D1, D2 is calculated to obtain the distance between the two images, and then a normalization operation is performed according to the distance between the two images to obtain the similarity of the two images. When the similarity is greater than a threshold value, it is judged that the two images are similar.

[0068] Step 5: cross-entropy operation is performed according to the output result of the normalization operation and the real label tag to obtain a loss function, and the first lightweight attention dilated network model and the second lightweight attention dilated network model are respectively fitted according to the loss function to obtain updated first lightweight attention dilated network model and second lightweight attention dilated network model.

[0069] Further, the calculation formula of the median filtering in step 1 is as follows:

[0070] (1).

[0071] Wherein, x , y are the pixel point positions in the original image matrix, , are the original image and the processed image, respectively, and the median is calculated in a given pixel set.

[0072] ​represents a pixel centered at ; represents a neighborhood coordinate pixel position, represents a two-dimensional image template.

[0073] Further, the calculation formula of the mean filtering in step 1 is as follows:

[0074] (2).

[0075] wherein, M represents a mean filter, I represents an input image, represents a pixel coordinate in the original image matrix, u , v respectively represent the offset of the mean filter in the x , y axis direction, k＇ represents the value range of the offset.

[0076] Further, the step 2 specifically comprises:

[0077] Step 2-1: respectively place the images B1 and on each level of the pyramid, and reduce the images B1 and to the size corresponding to each level of the pyramid.

[0078] Step 2-2: perform local feature extraction on the reduced images B1 of each level of the pyramid using the Laplace algorithm, and perform global feature extraction on the reduced images using the Mobilenetv2 backbone extraction network.

[0079] Step 2-3: fuse the local features of the images B1 and the global features of the images of each level of the pyramid to obtain the feature vectors of each level of the pyramid.

[0080] Step 2-4: fuse the feature vectors of each level of the pyramid layer by layer to obtain the image C1 of the original image size.

[0081] Step 2-5: respectively place the images B2 and on each level of the pyramid, and reduce the images B2 and to the size corresponding to each level of the pyramid.

[0082] Step 2-6: perform local feature extraction on the reduced images B2 of each level of the pyramid using the Laplace algorithm, and perform global feature extraction on the reduced images The Mobilenetv2 backbone extraction network is used for global feature extraction.

[0083] Step 2-7: local features of the image B2 at each level of the pyramid, image The global features are fused at each level of the pyramid to obtain the feature vector of each level of the pyramid.

[0084] Step 2-8: Fuse the feature vectors of each level of the pyramid layer by layer to obtain image C2 of the original image size.

[0085] During the image preprocessing stage, the present invention uses an improved pyramid fusion method to reduce the two input images to different scales, fuse them using local and global feature information, and finally enlarge the fused image back to its original scale. The improved pyramid fusion method can effectively fuse detailed information at different scales, resulting in a clearer input image, enhancing the recognition capability of the lightweight spatial transformation network model, and improving the accuracy and robustness of image matching.

[0086] Furthermore, the independently designed environmental governance supervision image similarity detection model used in step 3 includes a first lightweight attention expansion network model and a second lightweight attention expansion network model.

[0087] like Figure 3 As shown, the first lightweight attention expansion network model and the second lightweight attention expansion network model structurally include a first convolutional unit, a first inverted residual unit, a second inverted residual unit, an average pooling unit, and a second convolutional unit connected in sequence. The first lightweight attention network model and the second lightweight attention network model have an equivalent mapping relationship.

[0088] The first convolutional unit includes a sequentially connected convolutional layer, a batch normalization layer, and a function activation layer. The convolutional layer is configured with a convolution kernel of 3×3×3×32, a stride of 2, and an input size of 224×224×3. This convolutional layer, by using a convolution kernel with 32 channels and a stride of 2, increases the number of features in the environmental governance and supervision image while also downsampling it. This ensures that the model learns sufficient features while compressing the input governance and supervision image, thereby simplifying the complexity of the network calculation and maintaining feature invariance.

[0089] like Figure 4 As shown, the first inverted residual unit includes: a first inverted residual block, two second inverted residual blocks, three third inverted residual blocks and a fourth inverted residual block, and the first inverted residual block is sequentially connected in series with two series-connected second inverted residual blocks, three series-connected third inverted residual blocks and the fourth inverted residual block.

[0090] like Figure 5As shown, the second reverse residual unit comprises three fifth reverse residual blocks, three sixth reverse residual blocks, two seventh reverse residual blocks and two eighth reverse residual blocks, and the three fifth reverse residual blocks are connected in series with the three sixth reverse residual blocks, the two seventh reverse residual blocks and the two eighth reverse residual blocks in series.

[0091] The average pooling unit comprises an average pooling layer, and the average pooling layer is set to have a pooling kernel of 7x7 and an input size of 7x7x320.

[0092] The second convolution unit comprises convolution layers, batch normalization layers and function activation layers connected in series, the convolution layers are set to have a convolution kernel of 1x1x1280x1000, a step length of 1 and an input size of 1x1x1280.

[0093] As shown in the figure, Figure 6 The first reverse residual block, the second reverse residual block, the third reverse residual block and the fourth reverse residual block each comprise point-by-point convolution layers with a convolution kernel of 1x1, batch normalization layers, function activation layers, depth convolution layers with a convolution kernel of 3x3 and a step length of 2, batch normalization layers, function activation layers, point-by-point convolution layers with a convolution kernel of 1x1, batch normalization layers and linear activation function layers connected in series.

[0094] The first reverse residual block has an input size of 112x112x32, an output channel of 16 and an ascending channel number of 1.

[0095] The second reverse residual block has an input size of 112x112x16, an output channel of 24 and an ascending channel number of 6.

[0096] The third reverse residual block has an input size of 56x56x24, an output channel of 32 and an ascending channel number of 6.

[0097] The fourth reverse residual block has an input size of 28x28x24, an output channel of 64 and an ascending channel number of 6.

[0098] As shown in the figure, Figure 7As shown, the fifth inverse residual block and the sixth inverse residual block each include: sequentially connected pointwise convolution layers with a convolution kernel of 1x1, batch normalization layers, function activation layers, deep convolution layers with a convolution kernel of 3x3, a step of 2, and an expansion system of d, batch normalization layers, function activation layers, pointwise convolution layers with a convolution kernel of 1x1, batch normalization layers, and linear activation function layers.

[0099] As shown in Figure 8 As shown, the seventh inverse residual block and the eighth inverse residual block each include: sequentially connected pointwise convolution layers with a convolution kernel of 1x1, batch normalization layers, function activation layers, deep convolution layers with a convolution kernel of 3x3 and an expansion system of d, batch normalization layers, function activation layers, improved channel attention modules, pointwise convolution layers with a convolution kernel of 1x1, batch normalization layers, linear activation function layers, and summation layers.

[0100] As shown in Figure 9 As shown, the improved channel attention module includes: an output end of a first pointwise convolution layer with a convolution kernel of 1x1 connected to an input end of a deep convolution layer, output ends of the deep convolution layer respectively connected to input ends of Scale layers, average pooling layers, and maximum pooling layers, an average pooling layer output end sequentially connected to a first full connection layer, a function activation layer, and a second full connection layer, a maximum pooling layer output end sequentially connected to a third full connection layer, a function activation layer, and a fourth full connection layer, output ends of the second full connection layer and the fourth full connection layer sequentially connected to a summation layer, an activation layer, and then connected to an input end of a Scale layer, an output end of the Scale layer connected to an input end of a second pointwise convolution layer with a convolution kernel of 1x1, and output ends of the second pointwise convolution layer with a convolution kernel of 1x1 connected to input ends of the improved channel attention module and input ends of the summation layer. The Scale layer is used to weight the normalized weights to the features of each channel.

[0101] The fifth inverse residual block has an input size of 14x14x64, an output channel of 96, and an ascending channel number of 6.

[0102] The sixth inverse residual block has an input size of 14x14x64, an output channel of 160, and an ascending channel number of 6.

[0103] The seventh inverse residual block has an input size of 7x7x160, an output channel of 320, and an ascending channel number of 6.

[0104] The eighth inverse residual block has an input size of 7x7x320, an output channel of 1280, and an ascending channel number of 6.

[0105] Further, in one embodiment, the first inverse residual block unit includes a first inverse residual block, a second inverse residual block, a third inverse residual block, and a fourth inverse residual block, each of which uses a pointwise convolution layer with a convolution kernel size of 1x1. In order to obtain more features, low-dimensional features are mapped to high-dimensional space, so as to solve the problem that the depth convolution itself does not change the channel and does not associate each channel, and some features cannot be obtained.

[0106] Then, a batch normalization layer and a function activation layer of ReLU6 are used to accelerate the training convergence speed and improve the accuracy of the model.

[0107] Then, a depth convolution layer with a size of 3x3 is used, and each channel is independently convolved in the case of a step of 2.

[0108] In order to effectively use the feature information of different channels at the same spatial position, such as the color and texture of the same spatial position in the environmental supervision image being different, a pointwise convolution layer with a size of 1x1 is then used to combine the feature information of different channels at the same spatial position to generate a new feature map by weighting and combining the input image in the depth direction.

[0109] Finally, according to the inverse residual mechanism, the feature vector of the input layer is added to the feature vector after pointwise convolution, and a linear activation function is used for processing. Thus, the problems of uneven illumination, distortion and blur, and different angles of the image are solved.

[0110] The first inverse residual block unit is mainly used to extract the shallow features of the environmental governance supervision image, and contains more pixel point information. Some fine-grained information is the color, texture, edge, and corner information of the image, such as the color of the leaves in the image background, the color of the road, and some texture information. This is because the shallow network has a small receptive field and a small overlapping area of the receptive field, so that the network can capture more details, has a higher resolution, and contains more position and detail information.

[0111] Further, one embodiment, because the first inverse residual block unit has less convolution, lower semantics, and more noise, a second inverse residual block unit is used to extract deep features of the image. The fifth, sixth, seventh, and eighth inverse residual blocks in the second inverse residual block unit use dilated convolution with an expansion coefficient d when performing depth separable convolution, so that the lightweight attention expansion network model obtains a larger receptive field without increasing the convolution kernel. In order to achieve the purpose of similarity detection task by matching the background of the real environment governance supervision image, the background often occupies the main information part of the whole image, such as rivers, highways, and trees in the image often occupy a large part of the area of an image, which have more prominent features and do not need a too deep and large network. Therefore, it is necessary to obtain a larger receptive field without increasing the parameters.

[0112] Because there are too many extractable features in the image, and the model learns too many features, resulting in low efficiency, an improved channel attention module is used to weight the feature maps on all channels, and then the final output is affected according to the size of the weight. First, the input feature map [C, H, W] is subjected to maximum pooling and average pooling respectively to form two [C, 1, 1] weight vectors. [C, 1, 1] can be understood as for each channel C, there is a number corresponding to it, where C, H, and W are the number of channels, height, and width respectively. Then the two weight vectors [C, 1, 1] are respectively subjected to a fully connected layer, a RELU6 layer, and then a fully connected layer to map the weight of each channel. The mapped weights are then added and output through a Sigmoid activation function. Finally, the obtained channel weight [C, 1, 1] is multiplied by the original feature map [C, H, W] by channel to obtain a feature map with weight. Finally, it can be known that when the weight is large, the value of the channel feature map will increase accordingly, and the influence on the final output will also increase. When the weight is small, the value of the channel feature map will be smaller, and the influence on the final output will also be smaller. Here, because maximum pooling can highlight important areas in the image and average pooling can smooth noise, combining maximum pooling and average pooling can better obtain features. In this way, the input and output of each layer of the network structure are maintained, and only processing is done in the middle, similar to the hook function in software development. Finally, two one-dimensional vectors D1 and D2 with a size of (1, 1, 1280) are obtained.

[0113] Further, one embodiment, this is an example of embedding the ReLU6 transformation of the low-dimensional manifold into a high-dimensional space, and the function activation layer in the first inverse residual unit and the second inverse residual unit uses ReLU6. As shown in Figure 10 Assume that there is a spiral line X mInput data (input), 2-dimensional space is the horizontal coordinate X, the vertical coordinate Y, is mapped to n-dimensional space using a random matrix G and ReLU6 operation, the formula is:

[0114] (3).

[0115] Where G is a random matrix, X m is a set of spiral line input data composed of m points, y is the result of ReLU6 operation. Then use the inverse matrix G -1 of random matrix G to map back to 2-dimensional space, the formula is:

[0116] (4).

[0117] Where G -1 is the inverse matrix of G, y is the result of ReLU6 operation, is the result of y mapping back to 2-dimensional space. Formulas (3), (4) are to do ReLU6 operation on a "thing" in n-dimensional space, and then (restore by using the inverse matrix T-1 of T) compare the difference between the result after ReLU6 and the result of X m .

[0118] The specific process is first to use a random matrix G and ReLU6 to input the initial input into n-dimensional space, and then project the inverse matrix G -1 of G to two-dimensional space. As shown in Figure 11 , 12 , 13, when the output dimension (Output / dim) = 2 or 3 or 5, it will cause information loss and some points of the manifold will collapse together, as shown in Figure 14 , 15 , and when the output dimension = 15 or 30, the transformation is non-convex. ReLU6 can save complete information about the input manifold, but it must be satisfied that the input manifold is placed in a low-dimensional subspace of the input space. However, using ReLU6 activation function for high-dimensional space input may cause some information loss or even damage, so the last 1x1 pointwise convolution layer uses a linear activation function.

[0119] Further, step 4 is followed by calculating one-dimensional vectors D1, D2, and calculating according to the Euclidean distance, the formula is:

[0120] (5).

[0121] Where, represents the distance between the two images, , are two one-dimensional vectors after full connection layer flattening. The final similarity of the image is obtained after normalization processing, and the normalization calculation formula is:

[0122] (6).

[0123] wherein, is the Euclidean distance, is the similarity, the similarity is between 0 and 1, the greater the Euclidean distance, the smaller the similarity, and vice versa. The threshold is set to 0.5, greater than 0.5, then the two images are similar, less than or equal to 0.5, then the two images are not similar.

[0124] Further, in step 5, when the network input is two similar images, the label is 1 at this time; when the input of the two images is not similar, and the label at this time is 0. The output result after normalization operation and the real label are used as cross entropy operation to calculate the loss function. As shown in formulas (5), (6):

[0125] (7).

[0126] (8).

[0127] wherein, N represents the number of training samples; represents the probability that the nth sample is a positive sample; y n represents the label of the predicted nth sample, l n represents the probability of the nth sample, represents the calculated loss function value, and Mean represents the average value of the probabilities of n samples.

[0128] Embodiment 2:

[0129] This embodiment introduces a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, a kind of image recognition method for environmental governance supervision as any one of the embodiments 1 is realized.

[0130] Embodiment 3:

[0131] The third embodiment is a kind of computer equipment, comprising:

[0132] Memory for storing instructions.

[0133] Processor, for executing the instruction, so that the computer equipment executes the operation of the image recognition method for environmental governance supervision as any one of the embodiments 1.

[0134] Embodiment 4:

[0135] The embodiment introduces an image recognition method implementation and verification process for environmental governance supervision, including data set enhancement, training method, combined model features and preprocessing method, so that the method proposed in the application can obtain the final prediction result, that is, the similarity of the two images, only by uploading two images to the supervision system by the supervisor.

[0136] The environmental governance supervision image similarity detection model proposed in the application does not need experts to manually design features, and when compared with multiple network models, the network calculation amount and parameter amount are greatly reduced in the case of improving the image similarity detection accuracy, which has good robustness and practicality for the problems of large model calculation amount, inconvenient deployment of the model in the mobile terminal, uneven illumination, distortion blur and different angles of the image in the environmental governance supervision system.

[0137] In order to solve the problems of strong robustness of uneven illumination, distortion blur and projection transformation of the image in the environmental governance supervision system, the application provides the network with training samples with diversity.

[0138] In the aspect of data set construction, the application selects the Jiangsu green rural environment data set, which contains environmental image data of rural areas in counties and cities in Jiangsu Province, including water surface garbage cleaning, road surface garbage cleaning, agricultural production garbage cleaning and other garbage cleaning images. By screening the image data related to environmental governance and manually labeling it, the accuracy and reliability of the data set are ensured. Finally, a data set containing about 3000 environmental governance images is constructed, wherein the images are divided into training set, validation set and test set according to 7:2:1, then the parameters in the network are trained on the training set, the hyperparameters of the model are adjusted on the validation set, and finally the test set is tested.

[0139] In order to ensure the diversity and generalization ability of the data set, the data set is also subjected to data enhancement, such as random rotation, random cropping, random brightness and contrast, random flipping and random noise. Through the above data enhancement methods, more training data is obtained, so that the model can have good recognition performance for images under different angles and illumination conditions. Through these data enhancement techniques, the application finally obtains a diversified and generalization ability strong environmental governance image data set, which provides a basis for subsequent environmental governance image similarity detection research.

[0140] And through the above data set in the construction of data positive and negative samples, specifically, the two images constituting the positive sample come from the same background image, one of which is taken by the supervisor before the garbage cleaning at the site, and the other is taken by the supervisor after the garbage cleaning at the site. The two images have obvious differences in illumination, viewing angle, brightness and resolution; the two images constituting the negative sample come from different scenes. And the parameters trained on the VOC2012 dataset are used as the pre-training weights.

[0141] In the experiment, the initial learning rate is set to 7e-3, the minimum learning rate is set to 0.01 of the initial learning rate, the learning rate momentum update parameter is 0.9, the weight decay coefficient is 1e-4, the Adam optimizer is used, and the learning rate scheduler of YOLOX is used to dynamically adjust the learning rate.

[0142] In the embodiment, a lightweight attention expansion neural network architecture is adopted, and a lightweight attention expansion network is adopted to extract features based on the lightweight attention expansion neural network architecture, and an environment governance supervision image similarity detection model (Enviroment Gover Similarity-Model, referred to as EGSD-M) is proposed. When training this model, a large amount of image data is used for training, so as to obtain a network model with high generalization performance.

[0143] As can be seen from the experiment, the EGSD-M model has smaller space complexity and time complexity. In terms of parameter size and multiply-add operation number, the EGSD-M model performs as shown in Table 1, wherein VGG16-SN is a twin neural network with VGG16 as the backbone network, MobileNetv2-SN is a twin neural network with MobileNetv2 as the backbone network, and Inceptionv3-SN is a twin neural network with Inceptionv3 as the backbone network.

[0144] As can be seen from Table 1, the parameter size of EGSD-M is about one eighteenth of that of VGG16-SN, and the multiply-add operation calculation amount is about one forty-fourth of that of VGG16-SN; the parameter size of EGSD-M is about one third of that of Inceptionv3-SN, and the multiply-add operation calculation amount is about one fifteenth of that of Inceptionv3-SN; the parameter size of EGSD-M is about twice that of MobileNetv2-SN, and the multiply-add operation calculation amount is about 1.04 times that of MobileNetv2-SN; as can be seen from the above, the parameter reduction of EGSD-M is mainly reflected in the lightweight attention expansion network. The lightweight attention expansion uses a residual structure to control the channel number, the maximum channel number is 1280, and the deep convolution, pointwise convolution and dilated convolution are used instead of the standard convolution, so that the calculation amount is reduced by several times.

[0145] Table 1 Comparison of parameter size and number of multiply-add operations

[0146]

[0147] The evaluation index of the experiment on the similarity detection of environmental governance supervision images is quantitative analysis and selection of accuracy, precision, recall, and F1 measurement.

[0148] The test set is prepared with 200 images, including 3 types of images, 100 A-class scenes, 50 B-class scenes, and 25 C-class scenes.

[0149] Taking the matching of A-class scenes from the test set as an example, the input is the matching target A-class image, and the similarity greater than 0.5 is judged as A-class from the same scene; if not, it is determined to be other scenes. The experimental results are shown in Table 2, in which EGSD-M-A is compared with only the improved SeNet module, EGSD-M-B is compared with EGSD-M-A by improving some convolution modules to hollow dilated convolution, and EGSD-M-SN improves both.

[0150] Table 2 Similarity matching accuracy evaluation index of environmental governance supervision images

[0151]

[0152] As shown in Table 2, the experimental results show that the EGSD-M model is superior to the twin neural network with other network models as the backbone network in the similarity detection performance of environmental governance supervision images. While simplifying the model structure and reducing the model parameters, the accuracy of the model can be improved through reasonable network design and training method. This also shows that the use of a more complex and deeper structure of the backbone network model is not necessarily suitable for all application scenarios, but through specific problem-specific analysis. In addition, the background of the environmental governance supervision image often occupies the main information part of the whole image and only needs to match the background to achieve the purpose of similarity detection. Therefore, it is necessary to obtain a larger receptive field without increasing the parameters. In the channel attention mechanism, in order to better capture the features.

[0153] Since the maximum pooling can highlight the important areas in the image, and the average pooling can smooth the noise, two weight vectors are formed by the original average pooling and maximum pooling of the input feature map, and then the features are better extracted. Therefore, from the experimental results of EGSD-M-A and EGSD-M-B, both can achieve good results, and the EGSD-M-SN formed by combining the two can better integrate their respective characteristics and achieve better results.

[0154] The above merely describes the preferred embodiments of the present application, and it should be pointed out that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. An image recognition method for environmental governance supervision, characterized in that: The method comprises the following steps: Step 1: the input two environmental governance images A1, A2 are respectively subjected to median filtering to obtain two processed environmental governance images B1, B2, and the input two environmental governance images A1, A2 are respectively subjected to mean filtering to obtain two processed environmental governance images , ; Step 2: respectively perform image fusion processing on the environmental governance images B1 and B2 and output the environmental governance images C1 and C2 through a pyramid fusion method. Step 3: input the environmental governance images C1 and C2 into the improved first lightweight attention expansion network model and the second lightweight attention expansion network model respectively, and output two one-dimensional feature vectors D1 and D2 respectively; Step 4: calculate the Euclidean distance of the two one-dimensional feature vectors D1 and D2 to obtain the distance between the two images, and then perform normalization operation according to the distance between the two images to obtain the similarity of the two images, and when the similarity is greater than a threshold value, it is judged that the two images are similar; Step 5: perform cross-entropy operation according to the output result of the normalization operation and the real label tag to obtain a loss function, and fit the first lightweight attention expansion network model and the second lightweight attention expansion network model according to the loss function to obtain updated first lightweight attention expansion network model and second lightweight attention expansion network model; The step 2 specifically comprises: Step 2-1: The environmental governance image B1 and are placed in each level of the pyramid, and the environmental governance image B1 and are reduced to the size corresponding to each level of the pyramid. Step 2-2: the Laplace algorithm is used for local feature extraction on the environment governance image B1 after each level of the pyramid is reduced, and the environment governance image B1 after each level of the pyramid is reduced Mobilenetv2 backbone extraction network is used for global feature extraction; Step 2-3: The local features of the environment management image B1 of each level of the pyramid are fused with the global features of the environment management image B1 to obtain the feature vector of each level of the pyramid. Step 2-3: The local features of the environment management image B1 of each level of the pyramid are fused with the global features of the environment management image B1 to obtain the feature vector of each level of the pyramid. Step 2-4: perform fusion processing on the feature vectors of each level of the pyramid layer by layer to obtain the environmental governance image C1 of the original image size; Step 2-5: The environmental governance image B2 and are placed in each level of the pyramid, and the environmental governance image B2 and are reduced to the size corresponding to each level of the pyramid. Step 2-6: the Laplace algorithm is used for local feature extraction on the environment governance image B2 after each level of the pyramid is reduced, and the environment governance image B2 after each level of the pyramid is reduced Mobilenetv2 backbone extraction network is used for global feature extraction; Step 2-7: The local features of the pyramid environment management image B2 of each level are fused with the global features of the environment management image to obtain the feature vector of each level of the pyramid. Step 2-7: The local features of the pyramid environment management image B2 of each level are fused with the global features of the environment management image to obtain the feature vector of each level of the pyramid. Step 2-8: perform fusion processing on the feature vectors of each level of the pyramid layer by layer to obtain the environmental governance image C2 of the original image size. 2.The image recognition method for environmental governance supervision according to claim 1, characterized in that: The calculation formula of the median filter in the step 1 is as follows: ； where x, y are the pixel position in the original image matrix respectively, , are the original image and the processed image respectively, represents the median value in the given pixel set; represents the pixel centered at , , represents the neighborhood coordinate pixel position, represents the image two-dimensional template; The calculation formula of the mean filter in the step 1 is as follows: ； where M represents a mean filter, I represents an input image, represents a pixel coordinate in the original image matrix, u, v represent the offset of the mean filter in the x, y axis direction respectively, and k' represents the value range of the offset.

3. The image recognition method for environmental governance supervision according to claim 1, characterized in that: The first lightweight attention expansion network model and the second lightweight attention expansion network model both include a first convolution unit, a first reverse residual unit, a second reverse residual unit, an average pooling unit and a second convolution unit connected in sequence; The first convolution unit comprises a convolution layer, a batch normalization layer and a function activation layer connected in sequence, and the convolution layer is set to have a convolution kernel of 3x3x3x32, a step of 2 and an input size of 224x224x3; The first reverse residual unit comprises a first reverse residual block, two second reverse residual blocks, three third reverse residual blocks and a fourth reverse residual block, and the first reverse residual block is connected in series with two series-connected second reverse residual blocks, three series-connected third reverse residual blocks and a fourth reverse residual block; The second reverse residual unit comprises three fifth reverse residual blocks, three sixth reverse residual blocks, two seventh reverse residual blocks and two eighth reverse residual blocks, and the three series-connected fifth reverse residual blocks are connected in series with three series-connected sixth reverse residual blocks, two series-connected seventh reverse residual blocks and two series-connected eighth reverse residual blocks; The average pooling unit comprises an average pooling layer, and the average pooling layer is set to have a pooling kernel of 7x7 and an input size of 7x7x320; The second convolution unit comprises a convolution layer, a batch normalization layer and a function activation layer connected in sequence, and the convolution layer is set to have a convolution kernel of 1x1x1280x1000, a step of 1 and an input size of 1x1x1280.

4. The image recognition method for environmental governance supervision according to claim 3, characterized in that: The first inverse residual block, the second inverse residual block, the third inverse residual block and the fourth inverse residual block each comprise: a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer, a function activation layer, a depth convolution layer with a convolution kernel of 3*3 and a step of 2, a batch normalization layer, a function activation layer, a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer and a linear activation function layer; The fifth inverse residual block and the sixth inverse residual block each comprise: a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer, a function activation layer, a depth convolution layer with a convolution kernel of 3*3, a step of 2 and an expansion system of d, a batch normalization layer, a function activation layer, a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer and a linear activation function layer; The seventh inverse residual block and the eighth inverse residual block each comprise: a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer, a function activation layer, a depth convolution layer with a convolution kernel of 3*3 and an expansion system of d, a batch normalization layer, a function activation layer, an improved channel attention module, a point-by-point convolution layer with a convolution kernel of 1*1, a batch normalization layer, a linear activation function layer and a summation layer.

5. The image recognition method for environmental governance supervision according to claim 4, characterized in that: The improved channel attention module comprises: an output end of a first point-by-point convolution layer with a convolution kernel of 1*1 is connected to an input end of a depth convolution layer, an output end of the depth convolution layer is connected to input ends of a Scale layer, an average pooling layer and a maximum pooling layer, the average pooling layer is sequentially connected to a first full connection layer, a function activation layer and a second full connection layer, an output end of the maximum pooling layer is sequentially connected to a third full connection layer, a function activation layer and a fourth full connection layer, output ends of the second full connection layer and the fourth full connection layer are sequentially connected to a summation layer and an activation layer and then connected to an input end of the Scale layer, an output end of the Scale layer is connected to an input end of a second point-by-point convolution layer with a convolution kernel of 1*1, and output ends of the second point-by-point convolution layer with a convolution kernel of 1*1 are connected to input ends of the improved channel attention module and an input end of the summation layer; the Scale layer is used to weight the normalized weight to the features of each channel.

6. The image recognition method for environmental governance supervision according to claim 3, characterized in that: The function activation layers in the first inverse residual unit and the second inverse residual unit each adopt ReLU6.

7. A computer-readable storage medium, characterized in that: The computer program is stored on the computer program product and is executed by the processor to implement the image recognition method for environmental governance supervision according to any one of claims 1-6.

8. A computer device, comprising: The computer program product comprises: a memory for storing instructions; a processor for executing the instructions to enable the computer device to perform the operations of the image recognition method for environmental governance supervision according to any one of claims 1-6.

Images