Green land damage monitoring method and device and readable medium

Abstract

The invention belongs to the technical field of image processing, and relates to a green land damage monitoring method and device and a readable medium, and the method comprises the steps: obtaining a video image of a green land, and marking the video image; performing green land target segmentation and bare land target segmentation on the video image of the green land through a deep learning model to obtain a green land target segmentation map and a bare land target segmentation map; obtaining a green land area and a bare land area through the green land target segmentation map and the bare land target segmentation map; whether the greenbelt is damaged or not is obtained through the greenbelt area and the bare land area, and if the greenbelt is damaged, an alarm is given to the lower computer. DeepLabV < 3 + > is improved into a lightweight model, so that the lightweight, the balance speed and the precision of a network model are realized.

Description

Technical Field

[0001] The invention relates to a green space damage monitoring method, device and readable medium, and belongs to the technical field of image processing. Background Art

[0002] Green space is an important part of the earth's ecology. With the increase of environmental awareness, there is a strong call to protect the natural world on which human beings depend for survival. Regular inspection and timely detection of damage to green space such as lawns and flower beds have become an important part of urban management and ecological environment maintenance. It is of great significance to be able to accurately and quickly monitor whether green space is damaged and protect it, which not only helps to prevent further damage, but also effectively reduces maintenance costs.

[0003] DeepLabV3+ is a model for semantic segmentation that proposes a new encoder-decoder structure, using DeepLabv3 as the encoder module and a simple but effective decoder module. The model can control the resolution of the extracted encoder features through Atrous convolution (hole convolution), thereby balancing accuracy and running time. In addition, the network also uses the Xception model for segmentation tasks and applies Depthwise Separable Convolution to the Spatial Pyramid Pooling (ASPP) module and the decoder module, resulting in a faster and stronger encoder-decoder network.

[0004] like Figure 1 As shown in the figure, the DeepLabV3P+ model effectively captures multi-scale contextual information through dilated convolution and ASPP. This enables the model to perform accurate segmentation in complex scenes. Although DeepLabV3+ can capture multi-scale information, it may still lose some detail information when dealing with very subtle scene structures. Especially in the decoder stage, the gradual upsampling of features may lead to a reduction in resolution, resulting in insufficient capture of details. Although data augmentation alleviates the overfitting problem to a certain extent, for small-scale data sets, the model may still be at risk of overfitting due to overly complex architecture. In addition, the DeepLabV3+ model has a complex structure and contains a large number of parameters, and the training process requires a lot of computing resources and time. Therefore, it is crucial to study a lighter DeepLabV3+ model in these scenarios. However, the traditional DeepLabV3+ model has a complex structure, large training resource requirements, overfitting risks, and limited applications. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to propose a green space damage monitoring method, device and readable medium, by improving DeepLabV3+ into a lightweight model to achieve lightweight network model and balance speed and accuracy.

[0006] To achieve the above-mentioned objectives, the present invention proposes the following technical solutions: a green space damage monitoring method, comprising the following steps: obtaining a video image of a green space, and annotating the video image; performing green space target segmentation and bare land target segmentation on the video image of the green space through a deep learning model, and obtaining a green space target segmentation map and a bare land target segmentation map; obtaining the green space area and the bare land area through the green space target segmentation map and the bare land target segmentation map; obtaining whether the green space is damaged through the green space area and the bare land area, and if the green space is damaged, an alarm is sent to the lower computer.

[0007] Furthermore, the deep learning model is a DeepLabV3+ model or a MobileNetV3 model.

[0008] Furthermore, the DeepLabV3+ model includes an encoder and a decoder, the encoder includes a backbone network and an ASPP module, the input video image generates low-level features through the backbone network, part of the low-level features enter the decoder, and the other part is subjected to hole convolution through the ASPP module to generate high-level features, in the decoder, the low-level features are fused with the high-level features after convolution, and the fused feature vector is input into the classification layer for classification, thereby generating a segmentation map.

[0009] Furthermore, the backbone network adopts the MobeilNetV4 network structure.

[0010] Furthermore, the attention block in the MobeilNetV4 network structure is a multi-query attention block.

[0011] Furthermore, all pooling layers in the DeepLabV3+ model are replaced by a convolutional network including several convolutional layers.

[0012] Furthermore, the convolutional layer is a depth-wise convolutional layer DSC.

[0013] Furthermore, the method for obtaining whether the green space is damaged by using the green space area and the bare land area is: judging whether the green space area and the bare land area have changed compared with the previous frame image; if the green space area decreases and the bare land area increases, it is judged that the green space is damaged.

[0014] The present invention also discloses a green space damage monitoring method, comprising: an image acquisition module, used to obtain a video image of a green space and mark the video image; an image segmentation module, used to perform green space target segmentation and bare land target segmentation on the video image of the green space through a deep learning model, and obtain a green space target segmentation map and a bare land target segmentation map; an area calculation module, used to obtain the green space area and the bare land area through the green space target segmentation map and the bare land target segmentation map; a green space damage detection module, used to obtain whether the green space is damaged through the green space area and the bare land area, and if the green space is damaged, an alarm is sent to a lower computer.

[0015] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the green space damage monitoring methods described above.

[0016] The technical solution of the present invention has at least the following technical effects or advantages: the present invention attempts to improve DeepLabV3+ into a lightweight model to achieve lightweight network models and balance speed and accuracy. Compared with the traditional DeepLabV3+ model, the green space damage monitoring method of the present invention uses an improved lightweight algorithm to replace ResNet50 in the backbone network with MobeilNetV4. MobeilNetV4 uses the Mobile MQA attention module to significantly improve the reasoning speed and computational efficiency without sacrificing accuracy. By replacing all pooling layers of DeepLabV3+ with DSC, features of arbitrary resolution can be extracted, significantly improving accuracy. MobeilNetV4 improves performance without introducing any reasoning cost, improves the performance of the backbone model, and does not require additional reasoning cost or change the model structure. DSC makes full use of features of different scales, improves the scale invariance of features, and reduces reasoning overhead. It can not only improve accuracy, but also reduce the amount of computation, making the model more lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the structure of the DeepLabV3+ algorithm in the prior art; Figure 2 is a flow chart of a green space damage monitoring method in one embodiment of the present invention; Figure 3 is a schematic diagram of a deep convolutional layer DSC in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.

[0019] In order to solve the problems existing in the prior art such as the complex structure of the DeepLabV3+ model, large training resource requirements, overfitting risk, and limited application; the present invention proposes a green space damage monitoring method, device, and readable medium, which replaces the ResNet50 in the backbone network with MobeilNetV4 on the basis of the original DeepLabV3+, and replaces all pooling layers of DeepLabV3+ with depthwise separable convolution (DSC). MobeilNetV4 uses the Mobile MQA attention module to significantly improve the inference speed and computing efficiency without sacrificing accuracy; DSC extracts features of arbitrary resolution and significantly improves accuracy. The scheme of the present invention is described in detail below through examples.

[0020] Embodiment 1 This embodiment discloses a method for monitoring green space damage. Figure 2 As shown, the following steps are included: S1 obtains a video image of a green area and annotates the video image.

[0021] Video images are annotated using dataset annotation software.

[0022] S2 uses a deep learning model to perform green space target segmentation and bare land target segmentation on the video image of the green space to obtain the green space target segmentation map and the bare land target segmentation map.

[0023] In this embodiment, the deep learning model is a DeepLabV3+ model or a MobileNetV3 model. The DeepLabV3+ model includes an encoder and a decoder, and the encoder includes a backbone network and an ASPP module. The input video image generates low-level features through the backbone network, and a part of the low-level features enters the decoder, and the other part is subjected to hollow convolution through the ASPP module to generate high-level features. In the decoder, the low-level features are fused with the high-level features after convolution, and the fused feature vectors are input into the classification layer for classification, thereby generating a segmentation map. The backbone network adopts the MobeilNetV4 network structure, that is, the DNCC module in the backbone network of the DeepLabV3+ model is replaced by the MobeilNetV4 network structure. The attention block in the MobeilNetV4 network structure is a multi-query attention block, i.e., an MQA block, which combines the spatial reduction attention into the optimized MQA block to reduce the resolution of keys and values.

[0024] All pooling layers in the DeepLabV3+ model are replaced by a convolutional network consisting of several convolutional layers. The convolutional layer is a deep convolutional layer DSC. The schematic diagram of DSC is as follows: Figure 3 As shown. The deep convolutional layer DSC includes deep convolution and point-by-point convolution. The deep convolution is to perform convolution operation independently on each input channel to extract the features of each channel, and perform 1x1 convolution operation on the output of the deep convolution to merge the feature channels.

[0025] S3 obtains the green space area and the bare land area through the green space target segmentation map and the bare land target segmentation map.

[0026] S4 determines whether the green land is damaged based on the green land area and bare land area. If the green land is damaged, an alarm is sent to the lower computer.

[0027] The method of obtaining whether the green land is damaged through the green land area and the bare land area is: determine whether the green land area and the bare land area have changed compared with the previous frame image. If the green land area decreases and the bare land area increases, it is determined that the green land is damaged.

[0028] Embodiment 2 Based on the same inventive concept, this embodiment discloses a green space damage monitoring method, including: An image acquisition module is used to obtain video images of green areas and annotate the video images; An image segmentation module is used to perform green space target segmentation and bare land target segmentation on the video image of the green space through a deep learning model to obtain a green space target segmentation map and a bare land target segmentation map; An area calculation module is used to obtain the green area and the bare land area through the green area target segmentation map and the bare land target segmentation map; The green space damage detection module is used to determine whether the green space is damaged based on the green space area and bare land area. If the green space is damaged, an alarm is sent to the lower computer.

[0029] Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned green space damage monitoring methods.

[0030] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0031] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0032] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0033] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for monitoring green space damage, characterized in that: The following steps are involved: Obtaining a video image of a green space, and annotating the video image; By using a deep learning model, the video image of the green space is segmented into green space targets and the bare land targets are segmented into bare land targets to obtain a green space target segmentation map and a bare land target segmentation map; The green area and the bare land area are obtained through the green area target segmentation map and the bare land target segmentation map; Through the green area and bare land area, it is determined whether the green area is damaged. If the green area is damaged, an alarm is sent to the lower computer.

2. The green space damage monitoring method according to claim 1, characterized in that: The deep learning model is a DeepLabV3+ model or a MobileNetV3 model.

3. The green space damage monitoring method according to claim 2, characterized in that: The DeepLabV3+ model includes an encoder and a decoder. The encoder includes a backbone network and an ASPP module. The input video image generates low-level features through the backbone network. Part of the low-level features enter the decoder, and the other part is subjected to hole convolution through the ASPP module to generate high-level features. In the decoder, the low-level features are fused with the high-level features after convolution, and the fused feature vector is input into the classification layer for classification, thereby generating a segmentation map.

4. The green space damage monitoring method according to claim 3, characterized in that: The backbone network adopts the MobeilNetV4 network structure.

5. The green space damage monitoring method according to claim 4, characterized in that: The attention block in the MobeilNetV4 network structure is a multi-query attention block.

6. The green space damage monitoring method according to claim 2, characterized in that: All pooling layers in the DeepLabV3+ model are replaced by a convolutional network including several convolutional layers.

7. The green space damage monitoring method according to claim 2, characterized in that: The convolutional layer is a depthwise convolutional layer DSC.

8. The green space damage monitoring method according to any one of claims 1 to 7, characterized in that: The method for obtaining whether the green space is damaged by using the green space area and the bare land area is as follows: determining whether the green space area and the bare land area have changed compared with the previous frame image; if the green space area decreases and the bare land area increases, determining that the green space is damaged.

9. A method for monitoring green space damage, characterized in that: include: An image acquisition module, used to obtain a video image of a green space and annotate the video image; An image segmentation module is used to perform green space target segmentation and bare land target segmentation on the video image of the green space through a deep learning model to obtain a green space target segmentation map and a bare land target segmentation map; An area calculation module is used to obtain the green area and the bare land area through the green area target segmentation map and the bare land target segmentation map; The green space damage detection module is used to determine whether the green space is damaged based on the green space area and bare land area. If the green space is damaged, an alarm is sent to the lower computer.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the green space damage monitoring method as described in any one of claims 1-8.

Images