A real-time monitoring method and system for illegal rural buildings

Abstract

The invention relates to the technical field of image recognition, and specifically discloses a real-time monitoring method and system for illegal buildings in rural areas. The method uses a high-altitude camera for real-time monitoring to solve the problem that remote sensing image technology and unmanned aerial vehicle aerial photography technology cannot monitor illegal buildings in rural areas in real time; an actual illegal building data set is used to train a defogging, raining and haze removal model to solve the influence of foggy days, rainy days, foggy and rainy days and foggy and hazy days on the detection accuracy of illegal buildings in rural areas; a clear illegal building data set that is processed by the defogging, raining and haze removal model to remove rain, fog and haze is used to train an illegal building recognition model, and the model is constructed based on a SwinTransformer model. The SwinTransformer model has higher detection accuracy than other algorithms in normal detection tasks and performs excellently in small target detection tasks. The invention adds a local perception module on the basis of the SwinTransformer model to expand the local perception ability of the network, further improve the detection accuracy of small targets, and reduce the missed detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a real-time monitoring method and system for illegal buildings in rural areas. Background Art

[0002] How to quickly and accurately detect illegal buildings in rural areas and protect arable land resources has become an urgent problem to be solved.

[0003] The two most mainstream methods for illegal building detection are using remote sensing image technology and drone aerial photography technology combined with deep learning methods. However, both methods have limitations, as follows:

[0004] Remote sensing image technology has a long monitoring cycle, poor timeliness, is restricted by weather factors, and has poor quality of remote sensing image data, resulting in low detection accuracy; drone aerial photography technology is more suitable for small target detection in urban buildings, and the drone aerial photography method requires a lot of manpower and material resources and cannot monitor specific areas in real time.

[0005] Due to the shooting distance and shooting angle, some detection targets in the aerial video are too small. Currently, most deep learning networks such as SSD and Retinanet are still not very friendly to small target detection and have a high missed detection rate. Summary of the invention

[0006] The invention provides a real-time monitoring method and system for illegal rural buildings, and solves the technical problem of how to monitor illegal rural buildings in real time with high accuracy.

[0007] In order to solve the above technical problems, the present invention provides a real-time monitoring method for illegal buildings in rural areas, comprising the steps of:

[0008] S1. Use high ground cameras to shoot video data of illegal buildings in foggy, rainy, foggy and rainy weather, and haze weather to build an actual illegal building dataset;

[0009] S2, constructing a network architecture for defogging, rain removal and haze removal, and using the actual illegal building data set to train the network architecture for defogging, rain removal and haze removal to obtain a defogging, rain removal and haze removal model;

[0010] S3, using the defogging, rain removal and haze removal model to perform defogging, rain removal and haze removal processing on the actual illegal building data set, as well as image cutting, screening and data expansion to obtain a clear illegal building data set;

[0011] S4, constructing an illegal building recognition network architecture, using the clear illegal building data set to train the illegal building recognition network architecture, and obtaining an illegal building recognition model;

[0012] S5. Install a high-altitude camera at a place where illegal buildings need to be monitored to obtain video data to be identified in real time, convert the video data to be identified into image data and input the defogging, rain removal and haze removal model to obtain a defogging and rain removal image;

[0013] S6. Cutting the rain- and fog-free image according to the input image size requirement of the illegal building recognition model to obtain a plurality of cut images;

[0014] S7, inputting the plurality of cut images into the illegal building recognition model for recognition and labeling.

[0015] Specifically, the defogging, deraining and dehaze network architecture includes an atmospheric light scattering model reconstruction module and a clear image estimation module;

[0016] The atmospheric light scattering model reconstruction module is used to reconstruct the atmospheric light scattering model. The output K(x) of the atmospheric light scattering model reconstruction module is expressed as:

[0017]

[0018] Where I(x) is the input image, t(x) is the atmospheric projection rate, A is the atmospheric light value, and b is the deviation value;

[0019] The clear image estimation module is used to restore I(x) to a clear image J(x) according to K(x), which is expressed as:

[0020] J(x)=K(x)I(x)-K(x)+b.

[0021] Specifically, the atmospheric light scattering model reconstruction module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a multi-scale feature fusion layer and a maximum pooling layer; the input image I(x) is input into the first convolution layer, and the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer are convolved layer by layer, and the multi-scale feature fusion layer fuses the outputs of the first convolution layer to the fourth convolution layer to obtain a fused feature, which is input into the fifth convolution layer for convolution and then input into the maximum pooling layer, and finally the feature image K(x) is output;

[0022] The convolution kernel of the first convolution layer is 1, the convolution step is 1, and the padding value is 0, which are used to deepen the network depth; the convolution kernel of the second convolution layer is 3, the convolution step is 1, and the padding value is 1, which are used to change the number of feature channels; the convolution kernel of the third convolution layer is 5, the convolution step is 1, and the padding value is 2, which are used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fourth convolution layer is 7, the convolution step is 1, and the padding value is 3, which are used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fifth convolution layer is 3, the convolution step is 1, and the padding value is 1, which are used to restore the number of channels; the convolution kernel of the maximum pooling layer is 1, the convolution step is 1, and the padding value is 0, which are used to extract the main texture features and reduce the number of model parameters.

[0023] Specifically, the illegal building recognition network architecture includes an improved SwinTransformer module, a feature fusion module, and an RPN network module; the improved SwinTransformer module is used to extract features of different scales on the input cut image, and obtain feature images of different scales and input them into the feature fusion module; the feature fusion module is used to perform multi-scale fusion on feature images of different scales, and obtain a multi-scale fusion feature map and input it into the RPN network module; the RPN network module is used to generate a prediction box of the multi-scale fusion feature map.

[0024] Specifically, the improved SwinTransformer module adopts the architecture of the SwinTrainformer module, but a local perception module is inserted before the SwinT module in each stage; the local perception module includes an image restoration unit, a 1*1 convolution unit, a hole convolution unit, a splicing unit, a residual addition unit, an activation unit and an image restoration unit;

[0025] The image restoration unit inputs The feature map is restored to the form of B×H×W×C, 4 represents the side length of the block when the SwinTrainformer module divides the input H×W×3 feature image into blocks, and C represents the dimension of the feature map output by the previous module of the local perception module;

[0026] The 1*1 convolution unit is used to transform the input Perform convolution operation on the feature map;

[0027] The atrous convolution unit is used to perform atrous convolution on the feature map output by the first image restoration unit using convolution kernels of three different sizes to obtain feature maps of different scales;

[0028] The splicing unit is used to splice the feature maps of different scales obtained by the dilated convolution unit;

[0029] The residual addition unit is used to perform residual addition on the feature map output by the 1*1 convolution unit and the feature map output by the splicing unit;

[0030] The activation unit is used to process the feature map output by the residual addition unit using a Gelu activation function;

[0031] The image restoration unit is used to restore the feature map output by the activation unit to The size of and input to the SwinT module.

[0032] Specifically, the convolution kernel K of the dilated convolution unit is calculated by the following formula:

[0033] K=(r-1)(k-1)+k

[0034] Among them, the expansion rate r is set to 1, 2, 3, and the ordinary convolution kernel size k is set to 3, thereby obtaining receptive fields of 3*3, 5*5, and 7*7 sizes.

[0035] Specifically, the feature fusion module adopts the Bifpn network.

[0036] Furthermore, the step S1 specifically includes the steps of:

[0037] S11. Install high-altitude cameras to target illegal buildings in rural areas;

[0038] S12, using the high ground camera to shoot videos in foggy days, rainy days, foggy rain and haze weather to obtain video data;

[0039] S13, converting the video data into image data, unifying the image size, filtering out clear images, and using the remaining images to construct the actual illegal building data set;

[0040] In step S3 and step S6, each image is equally divided into 8 pictures;

[0041] In the step S3, the cut images are screened by: removing the images that do not contain illegal buildings;

[0042] In step S3, data expansion of the filtered image is specifically performed by performing various transformations on the filtered image in the manner of randomly adding points, adding noise, randomly rotating, mirroring, and changing brightness.

[0043] The present invention also provides a rural illegal building real-time monitoring system, which mainly comprises a video acquisition module, a video processing module, an image defogging, rain removal and haze removal module, an image cutting module and an illegal building identification module;

[0044] The video acquisition module is used to use a high-altitude camera to obtain video data to be identified in real time at a place where illegal buildings need to be monitored;

[0045] The video processing module is used to convert the video data to be identified into image data and input the image defogging, deraining and dehaze removal module;

[0046] The image defogging, rain removal and haze removal module is used to process the input image data using the defogging, rain removal and haze removal model according to any one of claims 1 to 8 to obtain a de-rained and de-hazed image and input it into the image cutting module;

[0047] The image cutting module is used to equally cut the derained and defogged image to obtain a plurality of cut images which are input into the illegal building identification module;

[0048] The illegal building recognition module is used to recognize the input image using the illegal building recognition model described in any one of claims 1 to 8, and output the illegal building recognition result.

[0049] Specifically, the system also includes an early warning module; when the illegal building identification result is that there is an illegal building, the early warning module will issue an early warning prompt, and calibrate and save the corresponding illegal building identification picture and upload it to the cloud.

[0050] The present invention provides a method and system for real-time monitoring of illegal buildings in rural areas, which has the following beneficial effects:

[0051] The method of using high ground cameras for real-time monitoring can solve the problem that remote sensing image technology and drone aerial photography technology cannot monitor illegal rural buildings in real time;

[0052] Use the actual illegal building data set to train a defogging, deraining and dehazing model suitable for rural illegal buildings, and solve the impact of foggy, rainy, foggy and rainy, and foggy and hazy days on the detection accuracy of rural illegal buildings;

[0053] An illegal building recognition model suitable for rural illegal buildings is trained using a clear illegal building data set that has been processed with a defogging, deraining and dehaze model. The model is constructed based on a SwinTransformer model. The SwinTransformer model has higher detection accuracy than other algorithms in normal detection tasks and performs well in small target detection tasks. The present invention adds a local perception module on the basis of the SwinTransformer model to expand the local perception ability of the network, and combines the feature fusion network Bifpn to perform multi-scale fusion of feature information, which can further improve the detection accuracy of small targets and reduce the missed detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a method for real-time monitoring of illegal buildings in rural areas provided by an embodiment of the present invention;

[0055] Figure 2 This is a structural diagram of a network architecture for removing fog, rain and haze provided by an embodiment of the present invention;

[0056] Figure 3 It is a feature map of the input image after passing through each convolution layer during the defogging area model training process provided by the embodiment of the present invention;

[0057] Figure 4 is a comparison diagram before and after defogging provided by an embodiment of the present invention;

[0058] Figure 5 This is a comparison diagram before and after rain removal provided by an embodiment of the present invention;

[0059] Figure 6 This is a comparison diagram before and after haze removal provided by an embodiment of the present invention;

[0060] Figure 7 is a network architecture diagram of an improved SwinTransformer module provided by an embodiment of the present invention;

[0061] Figure 8 is a comparison diagram of the calculation principles of W-MSA and SW-MSA provided by an embodiment of the present invention;

[0062] Fig. 9 This is an example diagram of the recognition effect of the illegal building recognition model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0064] The embodiment of the present invention first provides a real-time monitoring method for illegal buildings in rural areas. Figure 1 As shown in the flowchart, the specific steps include:

[0065] S1. Use high ground cameras to shoot video data of illegal buildings in foggy, rainy, foggy and rainy weather, and haze weather to build an actual illegal building dataset;

[0066] S2. Build a network architecture for defogging, rain removal and haze removal, and use the actual illegal building data set to train the network architecture for defogging, rain removal and haze removal to obtain a defogging, rain removal and haze removal model;

[0067] S3, using the defogging, deraining and dehazing model to perform defogging, deraining and dehazing processing on the actual illegal building dataset, as well as image cutting, screening and data expansion, to obtain a clear illegal building dataset;

[0068] S4. Construct an illegal building recognition network architecture, use a clear illegal building data set to train the illegal building recognition network architecture, and obtain an illegal building recognition model;

[0069] S5. Install a high ground camera at the place where illegal buildings need to be monitored to obtain the video data to be identified in real time, convert the video data to be identified into image data and input it into the defogging, rain removal and haze removal model to obtain a defogging and rain removal image;

[0070] S6. According to the input image size requirement of the illegal building recognition model, the rain-free and fog-free image is cut to obtain a plurality of cut images;

[0071] S7, inputting the multiple cut images into the illegal building recognition model for recognition and labeling.

[0072] The above steps are described in more detail below.

[0073] Step S1 specifically includes the following steps:

[0074] S11. Install high-altitude cameras to target illegal buildings in rural areas;

[0075] S12, respectively shooting videos with a high ground camera in foggy days, rainy days, foggy rainy days, and foggy haze weather to obtain video data;

[0076] S13, converting the video data into image data (saving the video data as an image every 60 frames), unifying the image size to 1920*1080, filtering out clear images, and using the remaining images to construct an actual illegal building data set.

[0077] In step S2, in order to make the recognition accuracy as high as possible, it is necessary to pre-process the video images under special weather conditions such as foggy days, rainy days, haze days, foggy and rainy days, etc., and transform them into corresponding clear pictures. Figure 2 As shown, the network architecture for removing fog, rain and haze includes: an atmospheric light scattering model reconstruction module (ReK module) and a clear image estimation module (Clear module) for estimating a clear image.

[0078] The atmospheric light scattering model reconstruction module is used to reconstruct the atmospheric light scattering model. The output K(x) of the atmospheric light scattering model reconstruction module is expressed as:

[0079]

[0080] Where I(x) is the input image, t(x) is the atmospheric projection rate, A is the atmospheric light value, and b is the deviation value;

[0081] The clear image estimation module is used to restore I(x) to the clear image J(x) according to K(x), which can be expressed as:

[0082] J(x)=K(x)I(x)-K(x)+b.

[0083] exist Figure 2 As can be seen in the figure, the atmospheric light scattering model reconstruction module includes the first convolution layer (Conv1), the second convolution layer (Conv2), the third convolution layer (Conv3), the fourth convolution layer (Conv4), the fifth convolution layer (Conv5), the multi-scale feature fusion layer (Concat3) and the maximum pooling layer (MaxPooling). The input image I(x) is input into the first convolution layer, which is convolved layer by layer by the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer. The multi-scale feature fusion layer fuses the outputs of the first to fourth convolution layers, obtains the fused features, which are input into the fifth convolution layer for convolution and then input into the maximum pooling layer, and finally outputs the feature image K(x).

[0084] Among them, the convolution kernel of the first convolution layer is 1 (K=1), the convolution step is 1 (S=1), and the padding value is 0 (P=0), which is used to deepen the network depth; the convolution kernel of the second convolution layer is 3 (K=3), the convolution step is 1 (S=1), and the padding value is 1 (P=1), which is used to change the number of feature channels; the convolution kernel of the third convolution layer is 5 (K=5), the convolution step is 1 (S=1), and the padding value is 2 (P=2), which is used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fourth convolution layer is 5 (K=5), the convolution step is 1 (S=1), and the padding value is 2 (P=2), which is used to increase the number of feature channels and increase the receptive field; The convolution kernel of the convolution layer is 7 (K=7), the convolution step is 1 (S=1), and the padding value is 3 (P=3), which is used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fifth convolution layer is 3 (K=3), the convolution step is 1 (S=1), and the padding value is 1 (P=1), which is used to restore the number of channels; the convolution kernel of the maximum pooling layer is 1 (K=1), the convolution step is 1 (S=1), and the padding value is 0 (P=1), which is used to extract the main texture features and reduce the number of model parameters. The exemplary input image (original) is input into the atmospheric light scattering model reconstruction module, and the output results of each layer (Conv1, Conv2, Conv3, Conv4, Conv5, maxpooling) are as follows Figure 3 As shown, the final result output by the clear image estimation module is as follows Figure 3 As shown in the last map.

[0085] The network architecture for removing fog, rain and haze in this embodiment adopts Figure 2 The advantages of the architecture shown are: the number of parameters of the constructed network model is small, which is convenient for migration to other deep learning algorithms; the model is an end-to-end detection method with fast processing speed. Experimental verification shows that the model has obvious effects in removing fog, rain and haze.

[0086] During the training process, the loss function used is the mean square error loss function (MSE), the learning rate is set to 0.001, the gradient descent algorithm is used to optimize the loss function, and the training rounds are 10.

[0087] The specific training process is:

[0088] PA1 builds a network architecture for removing fog, rain and haze. Figure 2 As shown;

[0089] PA2 sets relevant model parameters, such as setting the learning rate to 0.001, setting the number of training times to 10, etc., and then starts training the model;

[0090] PA3 verifies the model effect, adjusts relevant parameters to optimize the model, and retains the model with the best defogging index as the defogging, rain removal and haze removal model.

[0091] Figure 4This is a demonstration of the effect of an exemplary foggy image after being processed by the defogging, rain removal and haze removal model. The left side is before defogging, and the right side is after defogging. Figure 4 It can be seen that the outline of the building in the processed image is clearer.

[0092] Figure 5 This is a demonstration of the effect of an exemplary rainy image after being processed by the defogging, rain removal and haze removal model. The left side is before the rain removal, and the right side is after the rain removal. Figure 5 It can be seen that the processed image removes the occlusion of rain and the image contour is clearer.

[0093] Figure 6 This is a demonstration of the effect of an exemplary haze image after being processed by a mature model for dehazing, rain removal and haze removal. The left side is before dehazing, and the right side is after dehazing. Figure 6 It can be seen that the quality of the processed image is significantly improved.

[0094] For step S3, after training the defogging, deraining and dehazing models, it is necessary to train the recognition model for illegal buildings, and first a data set needs to be constructed. This step uses the defogging, deraining and dehazing models to perform defogging, deraining and dehazing processing on the actual illegal building data set, as well as image cutting, screening and data expansion to obtain a clear illegal building data set. The present invention also counts the relevant features of illegal buildings in the clear illegal building data set and performs correlation analysis. The features of rural illegal buildings in the clear illegal building data set include farms, new house frames, abandoned houses, etc., and the Spearman correlation coefficient is used to analyze the data. A heat map of the importance of each feature is made according to the Spearman correlation coefficient, and several groups of highly correlated features are screened out as the targets of subsequent feature extraction. After selecting the appropriate illegal building features through the Spearman correlation coefficient calculation, the image is cut.

[0095] The segmentation here means dividing each image into 8 pictures equally, that is, dividing the 1920*1080 image into 8 640*540 pictures according to fixed points.

[0096] The screening here refers to removing images that do not contain illegal buildings. The data expansion here refers to performing various transformations on the filtered images by randomly adding points, adding noise, randomly rotating, mirroring, and changing brightness. The final clear illegal building data set contains 3,000 images of 9 types of features, including people, bricks, steel bars, new house frames, adobe houses, and abandoned houses.

[0097] The images in the clear illegal building dataset are randomly divided into training set, validation set and test set in a ratio of 6:2:2 and stored in the train, val and test folders respectively. The labelme annotation tool is used to label the images in the dataset and store the images and label files in the same folder, for example, the test set and the label files of the test set are stored in the test folder.

[0098] For step S4, the illegal building recognition network architecture constructed by the embodiment of the present invention includes an improved SwinTransformer module, a feature fusion module, and an RPN network module; the improved SwinTransformer module is used to extract features of different scales on the input cut image, and obtain feature images of different scales and input them into the feature fusion module; the feature fusion module is used to perform multi-scale fusion on feature images of different scales, and obtain a multi-scale fusion feature map and input it into the RPN network module; the RPN network module is used to generate a prediction box of the multi-scale fusion feature map.

[0099] Among them, Figure 7 As shown, the improved SwinTransformer module adopts the architecture of the SwinTrainformer module, and its input image is a 640*540 (H×W) three-channel RGB image (the number of channels C is equal to 3). The present invention inserts a local perception module (DCB) before the SwinT module in each stage, aiming to expand the local perception ability of the network, while the remaining structures are the structures of the existing SwinTransformer module. The input image is first processed by the patch partition module to divide the image into blocks, and the image is divided into 160*135 non-overlapping windows with 4*4 as a patch, and the 640*540*3 image is converted to 160*135*48, so that the length and width of the image are reduced to 4 times the original, and the number of channels is increased. If the length and width of the input image are not an integer multiple of 4, a padding operation is performed, and the default padding value is 0. The number of image channels is changed from 160*135*48 to 160*135*96 through the patch Embedding module. After the dimensionality transformation, the length and width in the feature map are multiplied to become Transfer the feature map to the DCB module.

[0100] It should be noted that the local perception module includes an image restoration unit, a 1*1 convolution unit, a hole convolution unit, a splicing unit, a residual addition unit, an activation unit and an image restoration unit;

[0101] The image restoration unit takes the input The feature map is restored to the form of B×H×W×C, 4 represents the side length of the block when the SwinTrainformer module divides the input H×W×3 feature image into blocks, and C represents the dimension of the feature map output by the previous module of the local perception module;

[0102] 1*1 convolution unit is used to process the input Perform convolution operation on the feature map;

[0103] The dilated convolution unit is used to perform dilated convolution on the feature map output by the first image restoration unit using convolution kernels of three different sizes to obtain feature maps of different scales;

[0104] The splicing unit is used to splice feature maps of different scales obtained by the dilated convolution unit;

[0105] The residual addition unit is used to perform residual addition on the feature map output by the 1*1 convolution unit and the feature map output by the concatenation unit;

[0106] The activation unit is used to process the feature map output by the residual addition unit using the Gelu activation function;

[0107] The image restoration unit is used to restore the feature map output by the activation unit to The size of and input to the SwinT module.

[0108] The convolution kernel K of the dilated convolution unit is calculated as follows:

[0109] K=(r-1)(k-1)+k

[0110] Among them, the expansion rate r is set to 1, 2, 3, and the ordinary convolution kernel size k is set to 3, thereby obtaining receptive fields of 3*3, 5*5, and 7*7 sizes.

[0111] The SwinT module mainly performs multi-head self-attention calculations on feature maps. The SwinT module contains two window self-attention modules, W-MSA and SW-MSA. Figure 8As shown in the figure, the W-MSA module on the left divides the input feature map into non-overlapping windows of size 7*7. This operation is similar to the patch division operation mentioned above, and then multi-head self-attention calculation is performed in each window. The W-MSA module is introduced to reduce the amount of calculation. The W-MSA module on the right only performs self-attention calculation in the divided windows, and information cannot be transferred between windows. Therefore, the SW-MSA module is introduced to realize information interaction between windows and achieve the purpose of global receptive field. As can be seen from the above, the W-MSA module divides the feature map into windows of size 7*7, and then calculates self-attention in the window. SW-MSA needs to shift each 7*7 window to realize information interaction between windows. After the shift is completed, the number of windows increases, and the window needs to be encoded using relative position encoding, and the shifted pixels are re-divided into 7*7 window styles for window self-attention calculation. In order to prevent information confusion during self-attention calculation, a mask operation is used, and non-corresponding positions are set to -100.

[0112] After the W-MSA and SW-MSA modules perform self-attention calculations, they will enter the MLP module. The MLP module used in the present invention includes one hidden layer and uses GELU as the activation function. Figure 6 It can be seen that patch merging operations are required after the SwinT modules in stage1-stage3. Patch merging is similar to downsampling the feature map, which halves the width and height of the feature map and doubles the depth. The specific steps are as follows:

[0113] PM1 takes the input feature map as a token with 2*2 pixels;

[0114] PM2 stitches together the pixels of the same color at the same position in each token to get 4

[0115] New feature maps;

[0116] PM3 cat-concatenates the newly generated feature maps through the channel dimension;

[0117] PM4 uses the layer norm layer to normalize the feature map;

[0118] PM5 uses linear mapping to change the output dimension of the feature map.

[0119] The output feature map of stage1-stage4 after SwinT module is used as the input feature map of Bifpn network, and multi-scale fusion of features is realized through multi-layer Bifpn network. The fused multi-scale feature map is input into RPN network to generate the prediction box of the detected object. The iou (intersection-over-union ratio) is calculated for each predicted target box and the target box annotated in the data set. The samples with iou greater than 0.7 are selected as positive samples, and those with iou less than 0.3 are selected as negative samples. The classification loss is calculated using cross entropy loss function, and the regression loss is calculated using L1 Loss loss function. NMS non-maximum suppression is used to retain some prediction boxes with better effects, and ROI Align is used to classify and regress the prediction boxes. Complete the construction of the improved Swin Transformer model, start training the model, adjust the model parameters, and save the trained weight parameters.

[0120] In the process of training the illegal building recognition network architecture, two loss functions are mainly used, the cross entropy loss function for calculating the classification loss and the L1 Loss loss function for calculating the regression loss.

[0121] The specific training process is:

[0122] PB1 builds an improved Swin Transformer model, such as Figure 7 As shown;

[0123] PB2 modifies the training environment file and correctly calls the improved network model; modifies the parameter configuration, such as setting the learning rate to 0.01, setting the training rounds to 50, and selecting the AdamW (adaptive gradient) optimizer to adaptively adjust the learning rate during model training. Adjust the learning rate to verify the impact of the initial learning rate on the model convergence speed and determine the optimal learning rate. Select the optimal learning rate and change the training rounds

[0124] is 300, training model;

[0125] PB3 verifies the recognition effect of the model, fine-tunes relevant parameters, and retains the best weight file. This weight file is used to implement subsequent monitoring and early warning functions.

[0126] Corresponding to the above method, an embodiment of the present invention also provides a rural illegal building real-time monitoring system, which includes a video acquisition module, a video processing module, an image defogging, rain removal and haze removal module, an image cutting module, an illegal building identification module and an early warning module. Among them, the video acquisition module is used to use a high-altitude camera to obtain the video data to be identified in real time at the place where the illegal building needs to be monitored;

[0127] The video processing module is used to convert the video data to be identified into image data and input it into the image defogging, deraining and dehaze removal module;

[0128] The image defogging, deraining and dehazing module is used to process the input image data using the defogging, deraining and dehazing models to obtain the derained and dehazing images and input them into the image cutting module;

[0129] The image cutting module is used to cut the rain and fog removal images equally, and obtain multiple cut images to input into the illegal building recognition module;

[0130] The illegal building recognition module is used to recognize the input image using the illegal building recognition model and output the illegal building recognition result;

[0131] When the illegal building identification result is that there is an illegal building, such as Fig. 9 For the part framed by the framed lines shown, the early warning module will issue an early warning prompt (such as a voice prompt), and will calibrate and save the corresponding illegal building identification picture and then upload it to the cloud. Through the cloud, the recognition results of multiple high-altitude cameras set up in multiple locations can be intuitively viewed in real time, which is convenient for staff to handle.

[0132] In summary, the embodiments of the present invention provide a method and system for real-time monitoring of illegal buildings in rural areas, which:

[0133] The method of using high ground cameras for real-time monitoring can solve the problem that remote sensing image technology and drone aerial photography technology cannot monitor illegal rural buildings in real time;

[0134] Use the actual illegal building data set to train a defogging, deraining and dehazing model suitable for rural illegal buildings, and solve the impact of foggy, rainy, foggy and rainy, and foggy and hazy days on the detection accuracy of rural illegal buildings;

[0135] An illegal building recognition model suitable for rural illegal buildings is trained using a clear illegal building data set that has been processed with a defogging, deraining and dehaze model. The model is constructed based on a SwinTransformer model. The SwinTransformer model has higher detection accuracy than other algorithms in normal detection tasks and performs well in small target detection tasks. The present invention adds a local perception module on the basis of the SwinTransformer model to expand the local perception ability of the network, and combines the feature fusion network Bifpn to perform multi-scale fusion of feature information, which can further improve the detection accuracy of small targets and reduce the missed detection rate.

[0136] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A real-time monitoring method for illegal buildings in rural areas, characterized in that: Includes steps: S1. Use high-altitude cameras to shoot video data of illegal buildings in foggy, rainy, foggy and rainy weather, and haze weather to build an actual illegal building dataset; S2. Constructing a network architecture for defogging, rain removal and haze removal, and using the actual illegal building data set to train the network architecture for defogging, rain removal and haze removal to obtain a defogging, rain removal and haze removal model; S3, using the defogging, rain removal and haze removal model to perform defogging, rain removal and haze removal processing on the actual illegal building data set, as well as image cutting, screening and data expansion to obtain a clear illegal building data set; S4, constructing an illegal building recognition network architecture, using the clear illegal building data set to train the illegal building recognition network architecture, and obtaining an illegal building recognition model; The illegal building recognition network architecture includes an improved SwinTransformer module, a feature fusion module, and an RPN network module; the improved SwinTransformer module is used to extract features of different scales on the input cut image, obtain feature images of different scales and input them into the feature fusion module; the feature fusion module is used to perform multi-scale fusion on feature images of different scales, obtain a multi-scale fusion feature map and input it into the RPN network module; the RPN network module is used to generate a prediction frame of the multi-scale fusion feature map; The improved SwinTransformer module adopts the architecture of the SwinTrainformer module, but a local perception module is inserted before the SwinT module in each stage; the local perception module includes an image restoration unit, a 1*1 convolution unit, a hole convolution unit, a splicing unit, a residual addition unit, an activation unit and an image restoration unit; The image restoration unit inputs The feature map is restored to the form of B×H×W×C, 4 represents the side length of the block when the SwinTrainformer module divides the input H×W×3 feature image into blocks, and C represents the dimension of the feature map output by the previous module of the local perception module; The 1*1 convolution unit is used to transform the input Perform convolution operation on the feature map; The atrous convolution unit is used to perform atrous convolution on the feature map output by the image restoration unit using convolution kernels of three different sizes to obtain feature maps of different scales; The splicing unit is used to splice the feature maps of different scales obtained by the dilated convolution unit; The residual addition unit is used to perform residual addition on the feature map output by the 1*1 convolution unit and the feature map output by the splicing unit; The activation unit is used to process the feature map output by the residual addition unit using a Gelu activation function; The image restoration unit is used to restore the feature map output by the activation unit to The size is input into the SwinT module; S5. Install a high-altitude camera at a place where illegal buildings need to be monitored to obtain video data to be identified in real time, convert the video data to be identified into image data and input the defogging, rain removal and haze removal model to obtain a defogging and rain removal image; S6. Cutting the rain- and fog-free image according to the input image size requirement of the illegal building recognition model to obtain a plurality of cut images; S7, inputting the plurality of cut images into the illegal building recognition model for recognition and labeling.

2. A method for real-time monitoring of illegal rural buildings according to claim 1, characterized in that: The defogging, rain removal and haze removal network architecture includes an atmospheric light scattering model reconstruction module and a clear image estimation module; The atmospheric light scattering model reconstruction module is used to reconstruct the atmospheric light scattering model. The output K(x) of the atmospheric light scattering model reconstruction module is expressed as: Where I(x) is the input image, t(x) is the atmospheric projection rate, A is the atmospheric light value, b is the deviation value, and x refers to the pixel in the image; The clear image estimation module is used to restore I(x) to a clear image J(x) according to K(x), which is expressed as: J(x)=K(x)I(x)-K(x)+b.

3. A real-time monitoring method for illegal rural buildings according to claim 2, characterized in that: The atmospheric light scattering model reconstruction module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a multi-scale feature fusion layer and a maximum pooling layer; the input image I(x) is input into the first convolution layer, and the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer are convolved layer by layer, and the multi-scale feature fusion layer fuses the outputs of the first convolution layer to the fourth convolution layer to obtain a fused feature, which is input into the fifth convolution layer for convolution and then input into the maximum pooling layer, and finally a feature image K(x) is output; The convolution kernel of the first convolution layer is 1, the convolution step is 1, and the padding value is 0, which are used to deepen the network depth; the convolution kernel of the second convolution layer is 3, the convolution step is 1, and the padding value is 1, which are used to change the number of feature channels; the convolution kernel of the third convolution layer is 5, the convolution step is 1, and the padding value is 2, which are used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fourth convolution layer is 7, the convolution step is 1, and the padding value is 3, which are used to increase the number of feature channels and increase the receptive field; the convolution kernel of the fifth convolution layer is 3, the convolution step is 1, and the padding value is 1, which are used to restore the number of channels; the convolution kernel of the maximum pooling layer is 1, the convolution step is 1, and the padding value is 0, which are used to extract the main texture features and reduce the number of model parameters.

4. A real-time monitoring method for illegal rural buildings according to claim 1, characterized in that: The convolution kernel K of the dilated convolution unit is calculated by the following formula: K=(r-1)(k-1)+k Among them, the expansion rate r is set to 1, 2, 3, and the ordinary convolution kernel size k is set to 3, thereby obtaining receptive fields of 3*3, 5*5, and 7*7 sizes.

5. A method for real-time monitoring of illegal rural buildings according to claim 1, characterized in that: The feature fusion module adopts the Bifpn network.

6. A real-time monitoring method for illegal rural buildings according to claim 1, characterized in that: The step S1 specifically includes the following steps: S11. Install high-altitude cameras to target illegal buildings in rural areas; S12, using the high ground camera to shoot videos in foggy days, rainy days, foggy rain and haze weather to obtain video data; S13, converting the video data into image data, unifying the image size, filtering out clear images, and using the remaining images to construct the actual illegal building data set; In step S3 and step S6, each image is equally divided into 8 pictures; In the step S3, the cut images are screened by: removing the images that do not contain illegal buildings; In step S3, data expansion of the filtered image is specifically performed by performing various transformations on the filtered image in the manner of randomly adding points, adding noise, randomly rotating, mirroring, and changing brightness.

7. A real-time monitoring system for illegal buildings in rural areas, characterized by: It includes video acquisition module, video processing module, image defogging, rain removal and haze removal module, image cutting module and illegal building recognition module; The video acquisition module is used to use a high-altitude camera to obtain video data to be identified in real time at a place where illegal buildings need to be monitored; The video processing module is used to convert the video data to be identified into image data and input the image defogging, deraining and dehaze removal module; The image defogging, rain removal and haze removal module is used to process the input image data using the defogging, rain removal and haze removal model of the rural illegal building real-time monitoring method according to any one of claims 1 to 6, obtain the defogging and rain removal image and input it into the image cutting module; The image cutting module is used to equally cut the derained and defogged image to obtain a plurality of cut images which are input into the illegal building identification module; The illegal building identification module is used to identify the input image using the illegal building identification model of the rural illegal building real-time monitoring method described in any one of claims 1 to 6, and output the illegal building identification result.

8. A rural illegal building real-time monitoring system according to claim 7, characterized in that: The system also includes an early warning module; when the illegal building identification result is that there is an illegal building, the early warning module will issue an early warning prompt, and calibrate and save the corresponding illegal building identification picture and upload it to the cloud.

Images