Visible light image-based cultivated land detection method and system, and readable medium

Abstract

The invention belongs to the technical field of video image processing, and relates to a cultivated land detection method and system based on a visible light image, and a readable medium, and the method comprises the steps: reading high point camera image data and images of the same region in different periods in a historical image library; inputting the image into a defogging, rain-removing and snow-removing network model to obtain an image without the influence of fog, rain and snow; inputting the image without the influence of fog, rain and snow into a moving target detection algorithm model to obtain a high-point camera image and a moving target in a historical image library image, and removing the moving target; performing image segmentation on the image without the moving target to obtain a high-point camera image and a cultivated land range in a historical image library image; and comparing the cultivated land range of the image of the high-point camera with the cultivated land range of the image of the historical image library to obtain a cultivated land detection result. The problems that cultivated land detection is poor in timeliness and prone to being affected by weather and moving targets are solved, and the timeliness and accuracy of cultivated land detection are improved.

Description

Technical Field

[0001] The invention relates to a method, system and readable medium for detecting cultivated land based on visible light images, and belongs to the technical field of video image processing. Background Art

[0002] In order to promote the development of agriculture and rural economy and safeguard the interests of farmers, the dynamic monitoring of farmland protection has become more sophisticated in recent years. The use of artificial intelligence technology to automatically and accurately monitor farmland changes plays an important role in the monitoring and protection of farmland resources.

[0003] At present, cultivated land monitoring technology is based on remote sensing images, such as the patent with publication number CN116994025A: A method and device for improving cultivated land identification accuracy using image super-resolution technology and the patent with publication number CN115661655A: A method for extracting cultivated land in mountainous areas in southwest China by fusion of deep features of hyperspectral and high-resolution images. Both are based on remote sensing images for cultivated land detection. Affected by the satellite revisit cycle, the monitoring cycle of remote sensing image technology is long and the timeliness is poor. In addition, remote sensing images are greatly restricted by weather factors. For example, clouds will block the remote sensing sensor's view of the ground, and raindrops or snowflakes will scatter and absorb light, reducing the contrast and clarity of the image. Therefore, the quality of remote sensing image data is poor, resulting in low detection accuracy.

[0004] With the development of artificial intelligence in recent years, change detection based on deep learning has developed rapidly. Using deep learning methods to extract objects from images and then perform change detection has also become a common solution for cultivated land detection. For example, the patent with publication number CN111986099A: Cultivated land monitoring method and system based on convolutional neural network with fusion residual correction discloses a cultivated land monitoring scheme that extracts cultivated land based on convolutional neural network with fusion residual correction and then compares the cultivated land extraction results in remote sensing images of different phases. However, the existing cultivated land change detection technology is greatly affected by weather changes and moving targets such as people, cars, animals and crop phenological changes, resulting in insufficient robustness of the model. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a method, system and readable medium for farmland detection based on visible light images, so as to solve the problems of poor timeliness of farmland detection and susceptibility to weather and moving targets, thereby improving the timeliness and accuracy of farmland detection.

[0006] To achieve the above-mentioned purpose, the present invention proposes the following technical solutions: a method for detecting cultivated land based on visible light images, comprising the following steps: reading high-point camera image data and images of the same area in different periods in a historical image library; inputting the high-point camera images and the images in the historical image library into a defogging, deraining and desnowing network model to obtain the high-point camera images and the images in the historical image library from which the effects of fog, rain and snow are removed; inputting the high-point camera images and the images in the historical image library from which the effects of fog, rain and snow are removed into a moving target detection algorithm model to obtain moving targets in the high-point camera images and the images in the historical image library, and remove the moving targets; performing image segmentation on the high-point camera images and the images in the historical image library from which the moving targets are removed to obtain the cultivated land range in the high-point camera images and the images in the historical image library; comparing the cultivated land range of the high-point camera images with the cultivated land range of the historical image library images to obtain the cultivated land detection results.

[0007] Furthermore, the defogging, deraining and desnowing network model includes a feature extraction module, an encoding-decoding module and an output module. The feature extraction module is used to extract low-dimensional features from high-point camera image data and historical image library images; the encoding-decoding module is used to learn the low-dimensional features to obtain a mapping relationship between high-dimensional abstract features and image content details; and the output module is used to output the image content details obtained by the decoding layer.

[0008] Furthermore, the feature extraction module adopts dilated convolution as the basic structure of the feature block network. In the feature block network, the following dilated convolutions of different scales are adopted; DF=1 means that there is no gap between adjacent pixels during convolution, DF=2 means that the kernel interval between adjacent pixels is one pixel during convolution, DF=3 means that the kernel interval between adjacent pixels is two pixels during convolution, DF=4 means that the kernel interval between adjacent pixels is three pixels during convolution, and the value of the convolution kernel at the interval pixel point is filled with 0.

[0009] Furthermore, the encoding-decoding module includes an encoding feature aggregation dense convolution block and a decoding feature aggregation dense convolution block, wherein the encoding part and the decoding part appear in pairs, the encoding part is used to encode the feature information of the image content, and includes several pairs of sequentially connected encoding feature aggregation dense convolution blocks, and the decoding part is used to decode high-dimensional abstract image features to restore the image content details, and includes several pairs of sequentially connected decoding feature aggregation dense convolution blocks, and each pair of the downsampling maximum pooling layer of the encoding feature aggregation dense convolution block and the upsampling maximum pooling layer of the decoding feature aggregation dense convolution block are short-connected through Pooling Indices.

[0010] Furthermore, in the coding feature aggregation dense convolution block, the three features DF=1, DF=2 and DF=3 extracted in the feature extraction module are input, the three features are aggregated, and the aggregated features are input into the densely connected convolution block, the densely connected convolution block includes two or more convolution layers, the aggregated features and the output features of each layer are input into each subsequent convolution layer through short connections, all features are connected and aggregated through the connection layer, the three input features are added to the output features through the convolution layer, and then input into the downsampling maximum pooling layer maxpooling to generate the final output result.

[0011] Furthermore, in the decoding feature aggregation dense convolution block, the three features DF=1, DF=2 and DF=3 extracted in the feature extraction module are input, the three features are aggregated, and the aggregated features are input into a densely connected convolution block, the densely connected convolution block includes two or more convolution layers, the aggregated features and the output features of each layer are input into each subsequent convolution layer through short connections, all features are connected and aggregated through the connection layer, the three input features are added to the output features through the convolution layer, and then input into the upsampling maximum pooling layer upmaxpooling to generate the final output result.

[0012] Furthermore, in the moving target detection algorithm model, the moving area where the moving target may exist is first obtained by using the inter-frame difference method, and then the background modeling of the moving area is performed by using the VIBE background modeling algorithm, so as to extract the target from the image.

[0013] Furthermore, image segmentation is performed based on a U-Net semantic segmentation model with variable kernel convolution. In the U-Net semantic segmentation model with variable kernel convolution, information for generating a dynamic convolution kernel is extracted by inputting a feature map; based on the input features, sampling coordinates are generated for each point in the convolution kernel by a coordinate generation algorithm, and the offset of each sampling point is calculated in combination with target characteristics; a convolution kernel with an adaptive size and shape is generated by the sampling coordinates and the offset, and a convolution operation is performed on the input feature map using the generated dynamic convolution kernel, thereby generating an output feature map and obtaining the cultivated land area in the image.

[0014] Furthermore, the cultivated land range of the high point camera image is compared with the cultivated land range of the historical image library image, and the method for obtaining the cultivated land detection result is: identifying the cultivated land area in the cultivated land range in the high point camera image and the historical image library image by masking, and removing the moving target area in the cultivated land area; optimizing the mask by morphological operation, further optimizing the cultivated land area, and eliminating noise; finding the connected area in the mask, and calculating the area and position of each connected area; matching the connected area in the high point camera image and the historical image library image according to the IOU of the connected area according to the Hungarian algorithm, and obtaining a paired connected area pair; taking the difference between the areas of two connected areas in the connected area pair, and when the difference is greater than a threshold, it is considered that the cultivated land has changed.

[0015] The present invention also discloses a cultivated land detection system based on visible light images, comprising: an image acquisition module, used for reading high-point camera image data and images of the same area in different periods in a historical image library; a defogging, deraining and desnowing module, used for inputting the high-point camera images and the historical image library images into a defogging, deraining and desnowing network model to obtain the high-point camera images and the historical image library images with the effects of fog, rain and snow removed; a moving target removal module, used for inputting the high-point camera images and the historical image library images with the effects of fog, rain and snow removed into a moving target detection algorithm model to obtain moving targets in the high-point camera images and the historical image library images, and remove the moving targets; an image segmentation module, used for performing image segmentation on the high-point camera images and the historical image library images with the moving targets removed to obtain the cultivated land range in the high-point camera images and the historical image library images; an image comparison module, used for comparing the cultivated land range of the high-point camera images with the cultivated land range of the historical image library images to obtain the cultivated land detection result.

[0016] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned methods for detecting cultivated land based on visible light images.

[0017] The technical solution of the present invention has at least the following technical effects or advantages: 1. The present invention performs cultivated land detection based on visible light images. Compared with the traditional method of cultivated land detection based on remote sensing images, the method of the present invention has better real-time performance.

[0018] 2. The present invention uses a trained defogging, deraining and desnowing network model to perform defogging, deraining and declouding processing on the RGB images collected by the high-point camera, which can greatly reduce false detections caused by weather and improve the accuracy of cultivated land detection.

[0019] 3. The present invention adopts a moving target detection network architecture to remove moving targets in the image, which can greatly reduce false detections caused by moving targets in the cultivated land area and improve the accuracy of cultivated land detection.

[0020] 4. The present invention uses variable kernel convolution to improve the segmentation model, thereby increasing the robustness of the segmentation model, reducing false detections caused by crop phenological changes, and improving the accuracy of cultivated land detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a method for detecting cultivated land using visible light images in one embodiment of the present invention; Figure 2 2 is a schematic diagram of the structure of a network model for removing fog, rain and snow in one embodiment of the present invention; Figure 3 is a schematic diagram of the structure of a densely connected block for encoding feature aggregation in one embodiment of the present invention; Figure 4 is a schematic diagram of the structure of a densely connected block for decoding feature aggregation in one embodiment of the present invention; Figure 5 It is a structural diagram of a U-Net semantic segmentation model with variable kernel convolution in one embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.

[0023] In order to solve the problems of poor timeliness of cultivated land detection in the prior art and susceptibility to the influence of weather and moving targets, the present invention proposes a cultivated land detection method, system and readable medium based on visible light images, using a defogging, rain and snow removal model and a moving target removal model to remove fog, rain, snow and moving targets from the image, and then using an improved U-Net semantic segmentation model based on an alterable kernel convolution (AKConv) to segment the image of the image without fog, rain, snow and moving targets, and finally comparing the cultivated land segmentation results of two frames of images in the same area at different times to obtain the cultivated land detection result. The present invention adopts a trained defogging, rain and snow removal network architecture to reduce the influence of weather on the cultivated land detection method. A moving target detection network architecture is used to remove moving targets in the image to reduce false detection caused by the activities of moving targets in the cultivated land area. The segmentation model is improved by using a variable kernel convolution to improve the robustness of the segmentation model, reduce false detection caused by crop phenological changes, and improve the accuracy of cultivated land detection. The scheme of the present invention is described in detail below through embodiments in conjunction with the accompanying drawings.

[0024] Embodiment 1 This embodiment discloses a method for detecting cultivated land based on visible light images. Figure 1 As shown, the following steps are included: S1 reads the image data from the high-point camera and the images of the same area at different times in the historical image library.

[0025] Acquire an RGB image data from the high-point camera in real time, and obtain an RGB image data of the same area at different times from the historical image library.

[0026] S2 inputs the high-point camera images and historical image library images into the defogging, raining and snowing network model to obtain the high-point camera images and historical image library images without the influence of fog, rain and snow.

[0027] like Figure 2 As shown in FIG. 1 , the network model for removing fog, rain and snow includes a feature extraction module, an encoder-decoder module and an output module.

[0028] The feature extraction module Feature Block is set at the beginning of the entire defogging, deraining and desnowing network model, and is used to extract low-dimensional features from high-point camera image data and historical image library images; the feature extraction module uses dilated convolution as the basic structure of the feature block network Feature Block. In order to be able to identify rain, snow and fog of various sizes and directions, and to aggregate these feature extractions at the same time, the following dilated convolutions of different scales are used in the feature block network; DF=1 means that there is no gap between adjacent pixels during convolution, DF=2 means that the kernel interval between adjacent pixels during convolution is one pixel, DF=3 means that the kernel interval between adjacent pixels during convolution is two pixels, and DF=4 means that the kernel interval between adjacent pixels during convolution is three pixels, and the value of the convolution kernel at the interval pixel point is filled with 0. This can better extract the features of rain, snow and fog of different scales.

[0029] The encoding-decoding module is used to learn low-dimensional features and obtain the mapping relationship between high-dimensional abstract features and image content details. It can learn complex feature mappings from a large number of input and output image processors. The encoding-decoding module includes an encoding feature aggregation dense convolution block and a decoding feature aggregation dense convolution block, wherein the encoding part and the decoding part appear in pairs. The encoding part is used to encode the feature information of the image content, including several pairs of sequentially connected decoding feature aggregation dense convolution blocks. The decoding part is used to decode high-dimensional abstract image features to restore image content details, including several pairs of sequentially connected decoding feature aggregation dense convolution blocks. Each pair of downsampling maximum pooling layers of the encoding feature aggregation dense convolution blocks and the upsampling maximum pooling layers of the decoding feature aggregation dense convolution blocks are short-connected through pooling indices.

[0030] In the encoding part, the Encoder contains three dense convolution blocks for encoding feature aggregation. The last layer of the dense convolution block for encoding feature aggregation is the maximum pooling layer Maxpooling, which is used as the downsampler in the encoding stage. Symmetrically, the Decoder stage also contains three dense convolution blocks for decoding feature aggregation. The last layer of the dense convolution block for decoding feature aggregation is the maximum unpooling layer as the decoding upsampler MaxUnPooling. Between the symmetrical Encoder and Decoder parts, a short connection is made through the Pooling Indices between the downsampling and upsampling layers, and the correct recovery of the upsampling stage is guided by recording the position information of the maximum value of MaxPooling during downsampling.

[0031] The MaxUnPooling layer performs nonlinear upsampling using the pooling indices calculated in the MaxPooling layer of the corresponding encoding part. According to the recorded pooling index matrix, the output feature map of MaxUnPooling is calculated as follows: first, the pooling kernel size is initialized before the maximum pooling operation, and then the feature columns recorded by the index matrix are assigned to the corresponding positions of the input feature map and feature map, and then the other positions are set to zero. The upsampled feature activations are sparse and convolved with the subsequent convolutional layers to generate dense feature maps. Experiments show that this pooling index-guided decoding scheme is more suitable for rain removal, fog removal, and snow removal than the traditional bilinear upsampling method.

[0032] like Figure 3 As shown in Figure 4, in the encoding feature aggregation dense convolution block and the decoding feature aggregation dense convolution block, the three features DF=1, DF=2 and DF=3 extracted from the input feature extraction module are input. The convolution kernel size of the three different dilated convolutions is 3*3. Experiments have shown that the convolution kernel size of 3*3 can well extract the features of rain, snow and fog, while retaining fewer parameters and being easier to train. Aggregate the three features, and transform the channels of the feature map into the initial number of input channels through 1*1 convolution. The number of channels must be kept consistent in each stage of the densely connected convolution block to ensure that the features can flow between layers and the feature transfer is meaningful. Keep the number of channels consistent in the Encoder and Decoder stages to ensure that the features extracted in the encoding stage flow normally in the decoding stage, and complete the decoding recovery in the Decoder stage. Input the aggregated features into the decoding feature aggregation dense convolution block Dense Block. The densely connected convolution block includes two or more convolution layers. The aggregated features and the output features of each layer are input to each subsequent convolution layer through short connections, as shown in Figure 4. Figure 3 , as shown by the arrows in 4, each arrow represents a different feature transfer method. All features are connected and aggregated through the connection layer Concat, and the channels are restored to the number of channels of the input through the 1*1 convolution kernel. The input and output channels are consistent to ensure the normal calculation of the forward propagation and back propagation of the entire network. At the same time, in order to avoid the gradient explosion and gradient disappearance that hinder network training, the three input features are added to the output features through the convolution layer to generate the final output result.

[0033] The output module is used to output the image content details obtained by the decoding layer.

[0034] In the process of training the network model for defogging, deraining and desnowing, a training data set is first obtained. The data set includes videos shot by a high-altitude camera in foggy, rainy, foggy and rainy, haze and snowy weather conditions, and the video data is converted into image data as a training data set. The data set also includes foggy, rainy, foggy and rainy, haze and snowy weather images downloaded through the Internet as training data sets.

[0035] The Adam optimizer was used to iteratively update the parameters of the network, the total number of training rounds was set to 100, the initial learning rate was set to 0.0001, and the learning rate was multiplied by 0.8 at 30, 50, and 80 rounds. The mean absolute error (MAE) function, which is sensitive to sparse features, was selected as the loss function for training.

[0036] The network model for removing fog, rain and snow can be replaced by an adversarial neural network method.

[0037] S3 inputs the high-point camera images and historical image library images without the influence of fog, rain and snow into the moving target detection algorithm model, obtains the moving targets in the high-point camera images and historical image library images, and removes the moving targets.

[0038] The moving target detection algorithm model uses a three-frame three-channel difference method with an adaptive threshold combined with a background difference method to detect moving targets. First, the moving area where the moving target may exist is obtained by the inter-frame difference method, and then the VIBE (Visual Background Extraction) background modeling algorithm is used to model the background of the moving area, so that the target can be accurately extracted from the image.

[0039] The motion region extraction adopts a three-frame three-channel differential fusion algorithm. First, the three R, G, and B color channels of the image are processed using the three-frame difference method. The basic implementation steps of the so-called three-frame difference method are: first, the grayscale difference of the images of the first two frames is made, and then the grayscale difference of the current frame image and the previous frame image is made. Since there are three color channels, a total of six frame difference images are obtained.

[0040] The frame difference result image is segmented using an adaptive threshold. The method is to obtain the statistical histogram of the two-frame difference image and analyze the data of the statistical histogram to obtain a point that perfectly divides the histogram data into two parts. The left side of the point is the background pixel histogram, and the right side is the foreground pixel histogram. The horizontal coordinate of the point is used as the segmentation threshold to segment the frame difference image and distinguish the foreground and background.

[0041] After processing the R, G, and B color channel frame difference images of three adjacent frames of images respectively using the adaptive threshold method, a total of six frame difference images are obtained, of which the R, G, and B color channels of the first two frames of images have a total of three frame difference images, and the R, G, and B color channels of the last two frames of images have a total of three frame difference images. These six frame difference images are fused using the AND operation to obtain the final frame difference image. Then, the opening and closing operations in the morphological processing are performed to filter out the points to obtain the target complete frame difference image. Using the target complete frame difference image, a connected domain search is performed, a motion region is constructed for a larger connected domain, small connected domains are eliminated, and then the overlapping motion regions are spliced ​​to obtain the final motion region.

[0042] After the motion region is acquired, the ViBe (Visual Background Extraction) background modeling algorithm is used to model the moving target. The ViBe algorithm uses a pixel-level adaptive modeling method. Its core idea is to identify the foreground target by modeling and matching the pixel sequence of each pixel. The following are the main principles of the ViBe algorithm: Pixel modeling: The ViBe algorithm builds a background model by sampling pixel values ​​of adjacent frames. The background model consists of the pixel values ​​around the current pixel, such as the grayscale mean, median, or random sampling value of adjacent pixels as the background model.

[0043] Pixel matching: For each pixel, the ViBe algorithm calculates the difference between the current pixel and the background model and compares it with a preset threshold. If the difference is less than the threshold, the pixel is marked as background; if the difference is greater than the threshold, the pixel is marked as foreground.

[0044] Pixel update: For pixels marked as foreground, the ViBe algorithm updates the background model of its surrounding pixels. The values ​​of adjacent pixels can be updated by replacement or fusion to adapt to changes in the target and evolution of the background.

[0045] The ViBe algorithm includes the following key steps: Initialization: At the beginning, the background model of each pixel needs to be initialized. Usually, the first frame image or random sampling values ​​can be used to initialize the background model.

[0046] Modeling and matching: For each new input image, the ViBe algorithm models and matches each pixel according to the background model. By comparing the difference between the pixel value and the background model, the foreground and background segmentation results can be obtained.

[0047] Background update: Based on the segmentation results of the foreground and background, the ViBe algorithm updates the background model of each pixel. Generally speaking, samples of foreground pixels are discarded, while samples of background pixels are retained and used for modeling and matching of the next frame. In this embodiment, the ViBe algorithm can be replaced by other background subtraction algorithms for motion detection, such as kernel density estimation methods, Gaussian mixture model algorithms, etc., as long as the algorithm can extract moving targets.

[0048] The foreground detection image obtained by the motion detection algorithm often cannot directly extract the moving target, and morphological processing is required. Due to the factors such as leaf shaking, illumination change, dynamic background, etc. in the background, it is difficult to completely remove them by segmentation threshold alone. Therefore, there is a lot of noise in the foreground detection image, such as useless noise, isolated areas, holes, etc. In order to accurately extract the moving target, this embodiment uses morphological operation to process the foreground detection image to obtain accurate detection results. Morphological operation is one of the most widely used technologies in image processing. Its principle is to use a certain morphological structure to perform a series of operations on the corresponding structure in the image to eliminate or increase it. Common morphological operations include expansion, corrosion, opening operation and closing operation, among which opening operation and closing operation are mixed operations of expansion and corrosion. This embodiment uses morphological processing combined with connected domain detection to process the detection image. A 3*3 rectangular core is used to perform a closed operation on the foreground detection image to connect the target detection points as much as possible, and to connect the interference points as much as possible into a small connected domain. Then, the connected domain detection is used to eliminate the isolated area and fill the small hole area.

[0049] S4 performs image segmentation on the high-point camera image and the historical image library image with moving objects removed, and obtains the cultivated land range in the high-point camera image and the historical image library image.

[0050] like Figure 5 As shown in the figure, image segmentation is performed based on the U-Net semantic segmentation model with variable kernel convolution. In the U-Net semantic segmentation model with variable kernel convolution, the information used to generate the dynamic convolution kernel is extracted by inputting the feature map; based on the input features, the sampling coordinates are generated for each point in the convolution kernel by the coordinate generation algorithm, and the offset of each sampling point is calculated in combination with the target characteristics; the convolution kernel with adaptive size and shape is generated by sampling the coordinates and the offset, and the generated dynamic convolution kernel is used to perform a convolution operation on the input feature map to generate an output feature map, thereby obtaining the cultivated land area in the image.

[0051] Unet is an excellent semantic segmentation model. The Unet network structure is U-shaped, which allows it to use fewer training pictures while the accuracy of segmentation will not be bad. UNet uses convolutional neural networks. Neural networks based on convolution operations have achieved remarkable results in the field of deep learning, but standard convolution operations have two inherent defects. On the one hand, the convolution operation is limited to a local window, cannot capture information from other locations, and its sampling shape is fixed. On the other hand, the size of the convolution kernel is fixed to k×k, which is a fixed square shape, and the number of parameters tends to grow at right angles to the size. Obviously, the shape and size of the target are different in different data sets and different locations. Convolution kernels with fixed sample shapes and squares cannot adapt well to changing targets. In response to the above problems, this embodiment uses variable kernel convolution (AKConv) to improve the Unet network. AKConv provides an arbitrary number of parameters and arbitrary sampling shapes for the convolution kernel, providing a richer choice for the trade-off between network overhead and performance. AKConv provides an arbitrary number of parameters and arbitrary sampling shapes for the convolution kernel, providing a richer choice for the trade-off between network overhead and performance. AKConv defines the initial position of convolution kernels of any size through a new coordinate generation algorithm. In order to adapt to the changes in the target, an offset is introduced to adjust the sample shape at each position. AKConv completes the process of efficient feature extraction through irregular convolution operations, improves the robustness of the segmentation model, reduces false detections caused by changes in crop phenology, and improves the accuracy of cultivated land detection.

[0052] S5 compares the cultivated land range of the high-point camera image with the cultivated land range of the historical image library image to obtain the cultivated land detection result.

[0053] The method of comparing the cultivated land range of the high-point camera image with the cultivated land range of the historical image library image to obtain the cultivated land detection result is as follows: identify the cultivated land area in the cultivated land range in the high-point camera image and the historical image library image through the mask, and remove the moving target area in the cultivated land area; optimize the mask through morphological operations (such as dilation and erosion), further optimize the cultivated land area, and eliminate noise; find the connected area in the mask, and calculate the area and position of the cultivated land area for each connected area; match the high-point camera image and the connected area in the historical image library image according to the IOU (Intersection over Union) of the connected area according to the Hungarian algorithm, and obtain the paired connected area pairs; make a difference between the areas of the two connected areas in the connected area pair, and when the difference is greater than the threshold, it is considered that the cultivated land has changed. The threshold is obtained based on empirical values ​​and a large number of implementation optimizations.

[0054] Embodiment 2 Based on the same inventive concept, this embodiment provides a farmland detection system based on visible light images, including: Image acquisition module, used to read high-point camera image data and images of the same area at different times in the historical image library; The defogging, raining and snowing removal module is used to input the high-point camera images and the historical image library images into the defogging, raining and snowing removal network model to obtain the high-point camera images and the historical image library images without the influence of fog, rain and snow; A moving target removal module is used to input the high-point camera images and historical image library images without the effects of fog, rain and snow into the moving target detection algorithm model, obtain the moving targets in the high-point camera images and historical image library images, and remove the moving targets; An image segmentation module is used to perform image segmentation on the high-point camera image and the historical image library image from which moving objects have been removed, so as to obtain the cultivated land range in the high-point camera image and the historical image library image; The image comparison module is used to compare the cultivated land range of the high-point camera image with the cultivated land range of the historical image library image to obtain the cultivated land detection result.

[0055] Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned methods for detecting cultivated land based on visible light images.

[0056] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting cultivated land based on visible light images, characterized in that: The following steps are involved: Read high-point camera image data and images of the same area at different times in the historical image library; Inputting the high point camera image and the historical image library image into a defogging, deraining and desnowing network model to obtain the high point camera image and the historical image library image with the effects of fog, rain and snow removed; Inputting the high-point camera images and historical image library images without the effects of fog, rain and snow into the moving target detection algorithm model, obtaining the moving targets in the high-point camera images and historical image library images, and removing the moving targets; Performing image segmentation on the high-point camera image and the historical image library image from which moving objects are removed, to obtain the cultivated land range in the high-point camera image and the historical image library image; The cultivated land range of the high-point camera image is compared with the cultivated land range of the historical image library image to obtain the cultivated land detection result.

2. The method for detecting cultivated land based on visible light images according to claim 1, characterized in that: The defogging, deraining and desnowing network model includes a feature extraction module, an encoding-decoding module and an output module, wherein the feature extraction module is used to extract low-dimensional features from high-point camera image data and historical image library images; The encoding-decoding module is used to learn the low-dimensional features to obtain the mapping relationship between high-dimensional abstract features and image content details; the output module is used to output the image content details obtained by the decoding layer.

3. The method for detecting cultivated land based on visible light images according to claim 2, characterized in that: The feature extraction module adopts dilated convolution as the basic structure of the feature block network. In the feature block network, the following dilated convolutions of different scales are adopted; DF=1 means that there is no gap between adjacent pixels during convolution, DF=2 means that the kernel interval between adjacent pixels during convolution is one pixel, DF=3 means that the kernel interval between adjacent pixels during convolution is two pixels, DF=4 means that the kernel interval between adjacent pixels during convolution is three pixels, and the value of the convolution kernel at the interval pixel point is filled with 0.

4. The method for detecting cultivated land based on visible light images according to claim 3, characterized in that: The encoding-decoding module includes an encoding feature aggregation dense convolution block and a decoding feature aggregation dense convolution block, wherein the encoding part and the decoding part appear in pairs, the encoding part is used to encode the feature information of the image content, and includes several pairs of sequentially connected decoding feature aggregation dense convolution blocks, and the decoding part is used to decode high-dimensional abstract image features to restore the image content details, and includes several pairs of sequentially connected decoding feature aggregation dense convolution blocks, and each pair of downsampling maximum pooling layers of the encoding feature aggregation dense convolution blocks and the upsampling maximum pooling layers of the decoding feature aggregation dense convolution blocks are short-connected through pooling indexes PoolingIndices.

5. The method for detecting cultivated land based on visible light images according to claim 4, characterized in that: In the encoding feature aggregation dense convolution block and the decoding feature aggregation dense convolution block, the three features DF=1, DF=2 and DF=3 extracted in the feature extraction module are input, the three features are aggregated, and the aggregated features are input into the densely connected convolution block. The densely connected convolution block includes two or more convolution layers. The aggregated features and the output features of each layer are input into each subsequent convolution layer through short connections. All features are connected and aggregated through the connection layer. Through the convolution layer, the three input features and the output features are added to generate the final output result.

6. The method for detecting cultivated land based on visible light images according to any one of claims 1 to 5, characterized in that: In the moving target detection algorithm model, the moving area where the moving target may exist is first obtained by the inter-frame difference method, and then the background modeling of the moving area is performed by the VIBE background modeling algorithm, so as to extract the target from the image.

7. The method for detecting cultivated land based on visible light images according to any one of claims 1 to 5, characterized in that: Image segmentation is performed based on the U-Net semantic segmentation model with variable kernel convolution. In the U-Net semantic segmentation model with variable kernel convolution, the information used to generate the dynamic convolution kernel is extracted by inputting the feature map; Based on the input features, sampling coordinates are generated for each point in the convolution kernel through a coordinate generation algorithm, and the offset of each sampling point is calculated in combination with the target characteristics; a convolution kernel with an adaptive size and shape is generated through the sampling coordinates and the offset, and a convolution operation is performed on the input feature map using the generated dynamic convolution kernel to generate an output feature map and obtain the cultivated land area in the image.

8. The method for detecting cultivated land based on visible light images according to any one of claims 1 to 5, characterized in that: The cultivated land range of the high point camera image is compared with the cultivated land range of the historical image library image to obtain the method of cultivated land detection result: identify the cultivated land area in the cultivated land range in the high point camera image and the historical image library image through a mask, and remove the moving target area in the cultivated land area; optimize the mask through morphological operations, further optimize the cultivated land area, and eliminate noise; find the connected area in the mask, and calculate the area and position of each connected area; match the connected area in the high point camera image and the historical image library image according to the IOU of the connected area according to the Hungarian algorithm to obtain a paired connected area pair; make a difference in the areas of two connected areas in the connected area pair, and when the difference is greater than a threshold, it is considered that the cultivated land has changed.

9. A farmland detection system based on visible light images, characterized in that: include: Image acquisition module, used to read high-point camera image data and images of the same area at different times in the historical image library; A defogging, deraining and desnowing module is used to input the high-point camera image and the historical image library image into the defogging, deraining and desnowing network model to obtain the high-point camera image and the historical image library image with the effects of fog, rain and snow removed; A moving target removal module is used to input the high-point camera images and historical image library images without the effects of fog, rain and snow into the moving target detection algorithm model, obtain the moving targets in the high-point camera images and historical image library images, and remove the moving targets; An image segmentation module is used to perform image segmentation on the high-point camera image and the historical image library image from which moving objects have been removed, so as to obtain the cultivated land range in the high-point camera image and the historical image library image; The image comparison module is used to compare the cultivated land range of the high-point camera image with the cultivated land range of the historical image library image to obtain the cultivated land detection result.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for detecting cultivated land based on visible light images as described in any one of claims 1 to 8.

Images