Natural resource change early warning method and system based on machine vision
By using machine vision technology, ground image data captured by drones is distributed and preprocessed. Combined with feature enhancement learning networks and U-Net neural networks, the problems of low efficiency and poor accuracy in traditional natural resource monitoring are solved, and efficient resource change early warning and management support are achieved.
Patent Information
- Application Number
- CN202511450219.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-11
- Publication Date
- 2026-02-27
AI Technical Summary
Traditional natural resource monitoring methods rely on manual inspections, which are inefficient and easily affected by human factors, resulting in inaccurate and untimely monitoring results.
A machine vision-based early warning method for natural resource changes is adopted. Ground image data is captured by UAVs, stored in a distributed manner, and preprocessed. Features are extracted using global and local feature enhancement learning networks, and detection is performed using U-Net neural networks. Early warning information is generated and the changed areas are marked on the map.
It has enabled real-time monitoring and early warning of changes in natural resources, improved monitoring efficiency and accuracy, and provided technical support for resource protection and management.
Smart Images
Figure CN121582811A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, in particular to a natural resource change early warning method and system based on machine vision. BACKGROUND
[0002] Natural resources are an important support basis for green and harmonious development, but at present natural resources are facing increasingly serious destruction and changes, timely monitoring and early warning of natural resource changes are of great significance for the protection and management of natural resources, the traditional natural resource monitoring method mainly relies on manual patrol and regular investigation, this way is not only inefficient, but also easily affected by human factors, leading to inaccurate and untimely monitoring results, therefore, studying a natural resource change early warning method and system based on machine vision has important significance for improving the efficiency and accuracy of monitoring. SUMMARY
[0003] The present application aims to solve the above problems, and designs a natural resource change early warning method and system based on machine vision.
[0004] The first aspect of the present application provides a natural resource change early warning method based on machine vision, which comprises the following steps:
[0005] The ground image data photographed by the unmanned aerial vehicle is stored in a distributed manner, and the ground image data is evenly distributed to multiple database nodes to obtain the ground image data in the multiple database nodes;
[0006] The image gray scale of the ground image data is obtained by normalization processing, the image gray scale of the ground image data is preprocessed, and the preprocessed ground image data is obtained;
[0007] The preprocessed ground image data is subjected to feature extraction by a global feature enhancement learning network and a local feature enhancement learning network to generate feature image data;
[0008] The feature image data is input into a U-Net neural network model, the change of natural resources is detected based on the feature image data, and a detection result is output;
[0009] Based on the detection result, it is judged whether the natural resource change exceeds a preset threshold, if yes, an early warning information is generated, wherein the natural resource change at least includes the change of vegetation coverage, the increase and decrease change of water area, and the change of land area;
[0010] The early warning information is output to the terminal of the relevant personnel, and the natural resource change area is marked on the map.
[0011] Optionally, in the first implementation manner of the first aspect, the obtaining the image gray scale graph of the ground image data, and adaptively generating a threshold by OTSU threshold segmentation, to obtain a binary graph corresponding to the image gray scale graph, comprises:
[0012] obtaining the image gray scale graph of the ground image data, and adaptively generating a threshold by OTSU threshold segmentation, to obtain a binary graph corresponding to the image gray scale graph;
[0013] performing smoothing processing on the binary graph corresponding to the image gray scale graph by using a Gaussian filter, and performing down-sampling to obtain images of different scales;
[0014] performing image registration on the images of different scales by using an ORB algorithm to obtain a target registration image, and performing noise reduction on the target registration image by using a morphological filling processing to obtain the pre-processed ground image data.
[0015] Optionally, in the second implementation manner of the first aspect, the performing image registration on the images of different scales by using an ORB algorithm to obtain a target registration image, comprises:
[0016] performing initial positioning on the overlapping regions of the images of different scales by using a phase correlation method, performing feature point detection on the images of different scales after initial positioning by using a FAST algorithm, and screening the feature points by using a non-maximum suppression algorithm to obtain image coarse matching points;
[0017] performing sorting on the image coarse matching points according to Hamming distances from small to large by using a hash algorithm, selecting K pairs of matching points, and constructing an initial data set for a RANSAC algorithm,
[0018] randomly extracting N+1 groups of matching point pairs from the initial data set to obtain an inlier candidate set, and calculating a candidate homography matrix by using N groups of matching point pairs in the inlier candidate set;
[0019] judging whether the N+1 group of matching point pairs in the inlier candidate set satisfies the homography matrix, and if not, removing the N+1 group of matching points from the homography matrix;
[0020] if yes, verifying the homography matrix on the remaining matching point pairs in the initial data set until there is no new matching point pair in the inlier candidate set;
[0021] if the number of inliers in the inlier candidate set is greater than the number of inliers corresponding to the homography matrix, updating the optimal homography matrix, and otherwise, not updating, completing image registration by using the optimal homography matrix to obtain a target registration image.
[0022] Optionally, in a third implementation form of the first aspect of the present application, the feature extraction on the preprocessed ground image data by the global feature enhancement learning network and the local feature enhancement learning network to generate the feature image data comprises:
[0023] The preprocessed ground image data is acquired, and an LBP feature extraction algorithm is used to enhance the image texture features, and the LBP texture features are stacked with three channels of the RGB image of the preprocessed ground image data to form four-channel image information;
[0024] The four-channel image information is input into a model using an SE-Net structure as a backbone network, the output of a layer4 layer in the SE-Net structure is taken as the input of the global feature enhancement learning network, and a first feature map is extracted, and the first feature map is flattened into a plurality of feature vectors through a linear projection module;
[0025] Each feature vector is input into a Transformer encoding block, and global average pooling processing is performed on each vector to obtain a global feature vector;
[0026] The outputs of layer1, layer2 and layer3 in the SE-Net structure are taken as the input of the local feature enhancement learning network, a detail attention module is used to enhance the local features, and the local features are connected to the network through a residual connection and subjected to global average pooling to obtain a local feature vector;
[0027] The global feature vector obtained by the global feature enhancement learning network and the local feature vector obtained by the local feature enhancement learning network are spliced to obtain a fused feature vector, and the feature image data is generated according to the fused feature vector.
[0028] Optionally, in a fourth implementation form of the first aspect of the present application, the taking the outputs of layer1, layer2 and layer3 in the SE-Net structure as the input of the local feature enhancement learning network, using the detail attention module to enhance the local features, connecting the local features to the network through the residual connection, and performing the global average pooling to obtain the local feature vector comprises:
[0029] The second feature map output by the SE-Net structure is acquired, and the second feature map is compressed into a single channel through a 1x1 convolution kernel to fuse the information between different channels;
[0030] The spatial attention weight is obtained by using a ReLU function and a 3x3 convolution kernel to learn the weight of different spatial positions and using a sigmoid activation function for normalization;
[0031] The second feature map is subjected to global average pooling processing, information between different channels is fused through a 1*1 convolution kernel, the number of channels is compressed to 1 / 16, then the number of channels is expanded 16 times through a 1*1 convolution kernel, a sigmoid activation function is used for normalization, and a channel attention weight is obtained;
[0032] The spatial attention weight and the channel attention weight are outer multiplied to obtain a detail attention weight, multiplied with a corresponding position of input image information, subjected to residual connection, and a local feature vector is obtained.
[0033] Optionally, in a fifth implementation manner of the first aspect of the present application, the feature image data is input into a U-Net neural network model, a change of a natural resource is detected based on the feature image data, and a detection result is output, including:
[0034] After the feature image data is input into the U-Net neural network model, the feature image data is subjected to maximum pooling down-sampling processing through the first three groups of convolution, and is subjected to overlapping pooling processing through the last two groups of convolution;
[0035] A cascaded dilated convolution module is embedded in the fifth convolution layer, and an intermediate feature map is obtained through the cascaded dilated convolution module, wherein the intermediate feature map is 1 / 8 of the input feature image data;
[0036] The intermediate feature map obtained by each layer of the encoding end of the U-Net neural network model is subjected to a feature fusion module, and is used as input together with a feature vector of a corresponding layer of the decoding end of the U-Net neural network model, wherein the feature fusion module includes an upper branch and a lower branch, the upper branch inputs the intermediate feature map into a 3*3 convolution to extract features and obtain a low-dimensional tensor, the lower branch subjects the intermediate feature map to a 1*1 convolution and a 3*3 convolution to reduce dimensions and extract global information, and obtains a high-dimensional feature tensor, and the high-dimensional feature tensor is subjected to bilinear interpolation up-sampling and pixel-by-pixel multiplication with the low-dimensional tensor;
[0037] Channel weighting processing is performed in the decoding module, spatial information is integrated through a global average pooling module to obtain a decoding end output result, and the decoding end output result is sequentially superimposed and up-sampled to obtain a detection result.
[0038] Optionally, in a sixth implementation manner of the first aspect of the present application, whether the change of the natural resource exceeds a preset threshold is judged based on the detection result, if yes, a warning information is generated, including:
[0039] When the change of the natural resource is a change of vegetation coverage, whether the change of the vegetation coverage exceeds 12% is judged, if yes, a warning information about the vegetation coverage is generated.
[0040] When the natural resource change is the increase or decrease of the water area, it is judged whether the increase or decrease of the water area exceeds 5%, and if so, the early warning information about the water area is generated;
[0041] When the natural resource change is the change of the land area, it is judged whether the change of the land area exceeds 10%, and if so, the early warning information about the land area is generated.
[0042] The second aspect of the present application provides a natural resource change early warning system based on machine vision, which comprises an image acquisition module, an image preprocessing module, a feature extraction module, a detection module, an early warning module and a labeling module, wherein
[0043] The image acquisition module is used to store the ground image data photographed by the unmanned aerial vehicle in a distributed manner, uniformly distribute the ground image data into multiple database nodes, and acquire the ground image data in the multiple database nodes.
[0044] The image preprocessing module is used to acquire the image grayscale of the ground image data by normalization processing, preprocess the image grayscale of the ground image data, and obtain the preprocessed ground image data.
[0045] The feature extraction module is used to extract features from the preprocessed ground image data through a global feature enhancement learning network and a local feature enhancement learning network, and generate feature image data.
[0046] The detection module is used to input the feature image data into a U-Net neural network model, detect the change of the natural resource based on the feature image data, and output a detection result.
[0047] The early warning module is used to judge whether the natural resource change exceeds a preset threshold based on the detection result, and if so, generate early warning information, wherein the natural resource change at least includes the change of the vegetation coverage, the increase or decrease of the water area, and the change of the land area.
[0048] The labeling module is used to output the early warning information to a related personnel terminal and label the natural resource change area on a map.
[0049] Optionally, in the first implementation manner of the second aspect of the present application, the image preprocessing module comprises an image segmentation submodule, a smoothing processing submodule and an image registration submodule, wherein
[0050] The image segmentation submodule is used to acquire the image grayscale of the ground image data, generate a threshold value adaptively by OTSU threshold segmentation, and acquire a binary image corresponding to the image grayscale.
[0051] The smoothing processing submodule is configured to perform smoothing processing on the binary image corresponding to the image grayscale image by using Gaussian filtering processing, and to perform down-sampling to obtain images of different scales.
[0052] The image registration submodule is configured to perform image registration on the images of different scales by using an ORB algorithm to obtain a target registration image, and to perform noise reduction on the target registration image by using morphological filling processing to obtain preprocessed ground image data.
[0053] Optionally, in a second implementation manner of the second aspect of the present application, the feature extraction module comprises a stacking submodule, an input submodule, an average pooling submodule, an enhancement submodule and a splicing submodule, wherein,
[0054] The stacking submodule is configured to obtain the preprocessed ground image data, enhance image texture features by using an LBP feature extraction algorithm, stack the LBP texture features as a channel with three channels of the RGB image of the preprocessed ground image data to form four-channel image information.
[0055] The input submodule is configured to input the four-channel image information into a model using an SENet structure as a backbone network, input an output of a layer4 layer in the SENet structure as an input of a global feature enhancement learning network, extract a first feature map, and flatten the first feature map into a plurality of feature vectors by using a linear projection module.
[0056] The average pooling submodule is configured to input each feature vector into a Transformer encoding block, perform global average pooling processing on each vector to obtain a global feature vector.
[0057] The enhancement submodule is configured to input outputs of layer1, layer2 and layer3 layers in the SENet structure as inputs of a local feature enhancement learning network, enhance local features by using a detail attention module, connect the local features to the network by using a residual connection, perform global average pooling, and obtain a local feature vector.
[0058] The splicing submodule is configured to obtain a fused feature vector by splicing the global feature vector obtained by the global feature enhancement learning network and the local feature vector obtained by the local feature enhancement learning network, and generate feature image data according to the fused feature vector.
[0059] The technical scheme provided by the present application, by adopting a distributed manner to store the ground image data photographed by the unmanned aerial vehicle, uniformly distributing the ground image data to multiple database nodes, and obtaining the ground image data in the multiple database nodes; adopting normalization processing to obtain the image grayscale map of the ground image data, preprocessing the image grayscale map of the ground image data to obtain the preprocessed ground image data; performing feature extraction on the preprocessed ground image data through a global feature enhancement learning network and a local feature enhancement learning network to generate feature image data; inputting the feature image data into a U-Net neural network model, detecting the change of the natural resources based on the feature image data, and outputting a detection result; judging whether the change of the natural resources exceeds a preset threshold based on the detection result, and if so, generating an early warning information; outputting the early warning information to a terminal of a relevant person, and marking a natural resource change area on a map; the present application realizes real-time monitoring and change early warning of the natural resources, greatly improves the efficiency and accuracy of the monitoring, and provides strong technical support for natural resource protection and management. BRIEF DESCRIPTION OF DRAWINGS
[0060] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the present application.
[0061] Figure 1 The first embodiment schematic diagram of the natural resource change early warning method based on machine vision provided by the embodiment of the present application;
[0062] Figure 2 The second embodiment schematic diagram of the natural resource change early warning method based on machine vision provided by the embodiment of the present application;
[0063] Figure 3 The third embodiment schematic diagram of the natural resource change early warning method based on machine vision provided by the embodiment of the present application;
[0064] Figure 4 The structure schematic diagram of the natural resource change early warning system based on machine vision provided by the embodiment of the present application. DETAILED DESCRIPTION
[0065] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-described drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that data so described can be interchanged, under appropriate circumstances, to correspond to embodiments devised to perform the same functionality but involving, for example, a different sequence of steps or units. Furthermore, the terms "comprising" or "including" and any of their derivatives, are intended to be construed as encompassing not only the listed items but also others that are similar or equivalent in function, appearance and / or performance.
[0066] For the purpose of facilitating understanding, the specific flow of the embodiments of the present application is described below, please refer to Figure 1 The first embodiment of the natural resource change early warning method based on machine vision provided by the embodiments of the present application is shown in the figure, and the method specifically comprises the following steps:
[0067] Step 101, store the ground image data taken by the unmanned aerial vehicle in a distributed manner, distribute the ground image data uniformly into multiple database nodes, and obtain the ground image data in the multiple database nodes;
[0068] Step 102, obtain the image grayscale image of the ground image data by using normalization processing, pre-process the image grayscale image of the ground image data, and obtain the pre-processed ground image data;
[0069] Step 103, perform feature extraction on the pre-processed ground image data through a global feature enhancement learning network and a local feature enhancement learning network, and generate feature image data;
[0070] Step 104, input the feature image data into a U-Net neural network model, detect the change of the natural resource based on the feature image data, and output a detection result;
[0071] In this embodiment, after the feature image data is input into the U-Net neural network model, the feature image data passes through 5 groups of convolution layers, and the first 3 groups of convolution layers are used for maximum pooling downsampling processing of the feature image data, and the last 2 groups of convolution layers are used for overlapping pooling processing; a cascaded dilated convolution module is embedded in the fifth convolution layer, and an intermediate feature map is obtained through the cascaded dilated convolution module, wherein the intermediate feature map is 1 / 8 of the input feature image data; the intermediate feature map obtained by each layer of the encoding end of the U-Net neural network model is input into a feature fusion module, and is used as an input together with a feature vector of a corresponding layer of the decoding end of the U-Net neural network model, wherein the feature fusion module includes an upper branch and a lower branch, the upper branch inputs the intermediate feature map into a 3x3 convolution for feature extraction to obtain a low-dimensional tensor, and the lower branch inputs the intermediate feature map into a 1x1 convolution and a 3x3 convolution for dimension reduction and extraction of global information processing to obtain a high-dimensional feature tensor, the high-dimensional feature tensor is bilinearly interpolated and upsampled and multiplied with the low-dimensional tensor pixel by pixel; channel weighting processing is performed in the decoding module, spatial information is integrated through a global average pooling module to obtain a decoding end output result, and the decoding end output result is sequentially superimposed and upsampled to obtain a detection result.
[0072] Step 105, judging whether the natural resource change exceeds the preset threshold based on the detection result, if yes, generating an early warning information;
[0073] In this embodiment, the natural resource change at least includes a change of vegetation coverage, an increase and decrease change of water area, and a change of land area.
[0074] In this embodiment, when the natural resource change is the change of vegetation coverage, it is judged whether the change of vegetation coverage exceeds 12%, if yes, an early warning information about the vegetation coverage is generated; when the natural resource change is the increase and decrease change of water area, it is judged whether the increase and decrease change of water area exceeds 5%, if yes, an early warning information about the water area is generated; when the natural resource change is the change of land area, it is judged whether the change of land area exceeds 10%, if yes, an early warning information about the land area is generated.
[0075] Step 106, outputting the early warning information to a terminal of a related person, and marking a natural resource change area on a map.
[0076] In the embodiment of the present application, the ground image data photographed by the unmanned aerial vehicle is stored in a distributed manner, the ground image data is uniformly distributed to multiple database nodes, and the ground image data in the multiple database nodes is obtained; the image gray scale of the ground image data is obtained by normalization processing, the image gray scale of the ground image data is preprocessed, and the preprocessed ground image data is obtained; the feature extraction of the preprocessed ground image data is performed through the global feature enhancement learning network and the local feature enhancement learning network, and the feature image data is generated; the feature image data is input into the U-Net neural network model, the change of the natural resources is detected based on the feature image data, and the detection result is output; whether the change of the natural resources exceeds the preset threshold is judged based on the detection result, if yes, the early warning information is generated; the early warning information is output to the terminal of the relevant personnel, and the natural resource change area is marked on the map; the real-time monitoring and change early warning of the natural resources are realized, the efficiency and accuracy of the monitoring are greatly improved, and strong technical support is provided for the protection and management of the natural resources.
[0077] Please refer to Figure 2 The second embodiment of the natural resource change early warning method based on machine vision provided by the embodiment of the present application is shown in the figure, and the method comprises the following steps:
[0078] Step 201, obtaining the image gray scale of the ground image data, generating a threshold value adaptively by OTSU threshold segmentation, and obtaining the binary image corresponding to the image gray scale;
[0079] Step 202, performing smoothing processing on the binary image corresponding to the image gray scale by using Gaussian filtering processing, and performing down-sampling to obtain images of different scales;
[0080] Step 203, performing image registration on the images of different scales by using the ORB algorithm, obtaining the target registration image, and performing noise reduction on the target registration image by using morphological filling processing, and obtaining the preprocessed ground image data.
[0081] In this embodiment, the phase correlation method is used for initial positioning of the overlapping regions of images of different scales, the FAST algorithm is used for feature point detection of the images of different scales after initial positioning, and the non-maximum suppression algorithm is used for screening the feature points to obtain image coarse matching points; the hash algorithm is used to sort the image coarse matching points according to the Hamming distance from small to large, select K matching points, and construct an initial data set for the RANSAC algorithm, randomly extract N+1 matching point pairs from the initial data set to obtain an inner point candidate set, and use N matching point pairs in the inner point candidate set to calculate a candidate homography matrix; it is judged whether the N+1 matching point pairs in the inner point candidate set satisfy the homography matrix, if not, the N+1 matching point pairs are removed from the homography matrix; if yes, the homography matrix verifies the remaining matching point pairs in the initial data set until there is no new matching point pair in the inner point candidate set; if the number of inner points of the inner point candidate set is greater than the number of inner points corresponding to the homography matrix, the optimal homography matrix is updated, otherwise it is not updated, and the image registration is completed through the optimal homography matrix to obtain a target registration image.
[0082] Referring to Figure 3 The third embodiment of the machine vision-based natural resource change early warning method provided in the embodiment of the present application is schematically shown, and the method comprises the following steps:
[0083] In step 301, preprocessed ground image data is obtained, an LBP feature extraction algorithm is used to enhance image texture features, the LBP texture features are stacked as a channel with three channels of the RGB image of the preprocessed ground image data to form four-channel image information, and the four-channel image information is input into a model using an SENet structure as a backbone network.
[0084] In step 302, the four-channel image information is input into a model using an SENet structure as a backbone network, the output of a layer4 layer in the SENet structure is taken as input of a global feature enhancement learning network, a first feature map is obtained by extraction, and the first feature map is flattened into a plurality of feature vectors through a linear projection module.
[0085] In step 303, each feature vector is input into a Transformer encoding block, global average pooling processing is performed on each vector to obtain a global feature vector.
[0086] In step 304, the outputs of layer1, layer2 and layer3 in the SENet structure are taken as input of a local feature enhancement learning network, a detail attention module is used to enhance local features, and the local features are connected to the network through a residual connection and subjected to global average pooling to obtain a local feature vector.
[0087] In this embodiment, the second feature map output by the SENet structure is obtained, the second feature map is compressed into a single channel through a 1*1 convolution kernel to fuse information between different channels, a weight at different spatial positions is learned through a ReLU function and a 3*3 convolution kernel, and a sigmoid activation function is used for normalization to obtain a spatial attention weight; the second feature map is subjected to global average pooling processing, information between different channels is fused through a 1*1 convolution kernel, the number of channels is compressed to 1 / 16, and then the number of channels is expanded 16 times through a 1*1 convolution kernel, a sigmoid activation function is used for normalization to obtain a channel attention weight; the spatial attention weight and the channel attention weight are outer product to obtain a detail attention weight, the detail attention weight is multiplied with a corresponding position of input image information, residual connection is performed, and a local feature vector is obtained.
[0088] In step 305, the global feature vector obtained by the global feature enhancement learning network and the local feature vector obtained by the local feature enhancement learning network are spliced to obtain a fused feature vector, and a feature image data is generated according to the fused feature vector.
[0089] Referring to Figure 4 The structure diagram of the natural resource change early warning system based on machine vision provided by the embodiment of the present application, the system comprises an image acquisition module, an image preprocessing module, a feature extraction module, a detection module, an early warning module and a labeling module, wherein,
[0090] The image acquisition module 401 is configured to store ground image data photographed by a UAV in a distributed manner, distribute the ground image data uniformly to a plurality of database nodes, and acquire the ground image data in the plurality of database nodes.
[0091] The image preprocessing module 402 is configured to acquire an image grayscale map of the ground image data by normalization processing, preprocess the image grayscale map of the ground image data, and obtain preprocessed ground image data.
[0092] The feature extraction module 403 is configured to extract features of the preprocessed ground image data by a global feature enhancement learning network and a local feature enhancement learning network, and generate feature image data.
[0093] The detection module 404 is configured to input the feature image data into a U-Net neural network model, detect a change of a natural resource based on the feature image data, and output a detection result.
[0094] The early warning module 405 is configured to determine whether the change of the natural resource exceeds a preset threshold based on the detection result, and if yes, generate early warning information, wherein the change of the natural resource at least comprises a change of vegetation coverage, an increase or decrease change of water area, and a change of land area.
[0095] The marking module 406 is configured to output the early warning information to the terminal of the relevant personnel and mark the natural resource change area on the map.
[0096] In this embodiment, the image preprocessing module includes an image segmentation submodule, a smoothing processing submodule and an image registration submodule, wherein,
[0097] The image segmentation submodule is configured to obtain an image grayscale image of the ground image data, adopt OTSU threshold segmentation to adaptively generate a threshold, and obtain a binary image corresponding to the image grayscale image.
[0098] The smoothing processing submodule is configured to adopt Gaussian filtering processing to perform smoothing processing on the binary image corresponding to the image grayscale image, and perform down-sampling to obtain images of different scales.
[0099] The image registration submodule is configured to adopt an ORB algorithm to perform image registration on the images of different scales to obtain a target registration image, and adopt morphological filling processing to perform noise reduction on the target registration image to obtain the preprocessed ground image data.
[0100] In this embodiment, the feature extraction module includes a stacking submodule, an input submodule, an average pooling submodule, an enhancement submodule and a splicing submodule, wherein,
[0101] The stacking submodule is configured to obtain the preprocessed ground image data, adopt an LBP feature extraction algorithm to enhance image texture features, stack the LBP texture features as a channel with three channels of RGB images of the preprocessed ground image data to form four-channel image information.
[0102] The input submodule is configured to input the four-channel image information into a model with an SE-Net structure as a backbone network, take the output of a layer4 layer in the SE-Net structure as an input of a global feature enhancement learning network, extract a first feature map, and flatten the first feature map into a plurality of feature vectors through a linear projection module.
[0103] The average pooling submodule is configured to input each feature vector into a Transformer encoding block, perform global average pooling processing on each vector to obtain a global feature vector.
[0104] The enhancement submodule is configured to take the outputs of layer1, layer2 and layer3 layers in the SE-Net structure as inputs of a local feature enhancement learning network, adopt a detail attention module to enhance local features, connect the local features to the network through a residual connection, perform global average pooling, and obtain a local feature vector.
[0105] The splicing submodule is configured to splice the global feature vector obtained by the global feature enhancement learning network and the local feature vector obtained by the local feature enhancement learning network to obtain a fused feature vector, and generate feature image data according to the fused feature vector.
[0106] Through implementation of the above scheme, the system comprises an image acquisition module, an image preprocessing module, a feature extraction module, a detection module, a prewarning module and a labeling module; the application realizes real-time monitoring and change prewarning of natural resources, greatly improves the monitoring efficiency and accuracy, and provides strong technical support for natural resource protection and management.
[0107] The basic principle, main features and advantages of the application are shown and described above. Those skilled in the art should understand that the application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the application and are not intended to limit the application. Without departing from the spirit and scope of the application, various changes and improvements can be made to the application, and these changes and improvements all fall within the scope of the claimed application. The scope of protection of the application is defined by the appended claims and their equivalents.
Claims
1. A machine vision-based method for early warning of changes in natural resources, characterized in that, The machine vision-based early warning method for natural resource changes includes the following steps: Ground image data captured by drones is stored in a distributed manner, and the ground image data is evenly distributed to multiple database nodes to obtain the ground image data from multiple database nodes; The ground image data is obtained by normalization processing, and the ground image data is preprocessed to obtain preprocessed ground image data. Feature image data is generated by extracting features from preprocessed ground image data through a global feature enhancement learning network and a local feature enhancement learning network. The feature image data is input into the U-Net neural network model, and the changes in natural resources are detected based on the feature image data, and the detection results are output. Based on the detection results, it is determined whether the changes in natural resources exceed a preset threshold. If so, an early warning message is generated. The changes in natural resources include at least changes in vegetation coverage, changes in water area, and changes in land area. The warning information is output to the terminals of relevant personnel, and the areas of natural resource change are marked on the map.
2. The method for early warning of natural resource changes based on machine vision as described in claim 1, characterized in that, The process involves obtaining a grayscale image of the ground image data using normalization processing, and preprocessing the grayscale image of the ground image data to obtain preprocessed ground image data, including: The grayscale image of the ground image data is obtained, and the threshold is adaptively generated using OTSU threshold segmentation to obtain the binary image corresponding to the grayscale image. Gaussian filtering is used to smooth the binary image corresponding to the grayscale image, and downsampling is performed to obtain images of different scales; The ORB algorithm is used to register images at different scales to obtain target registered images, and morphological filling is used to denoise the target registered images to obtain preprocessed ground image data.
3. The method for early warning of natural resource changes based on machine vision as described in claim 2, characterized in that, The process of using the ORB algorithm to register images at different scales to obtain target registered images includes: The phase correlation method is used to initially locate the overlapping regions of the images at different scales. The FAST algorithm is used to detect feature points in the images at different scales after initial localization. The non-maximum suppression algorithm is used to filter the feature points to obtain coarse matching points for the images. Using a hash algorithm, the coarse matching points in the image are sorted in ascending order of Hamming distance, and K pairs of matching points are selected to construct the initial dataset for the RANSAC algorithm. N+1 pairs of matching points are randomly selected from the initial dataset to obtain an inlier candidate set. The candidate homography matrix is calculated using the N pairs of matching points in the inlier candidate set. Determine whether the (N+1)th set of matching points in the candidate inlier satisfies the homography matrix. If not, remove the (N+1)th set of matching points from the homography matrix. If satisfied, the homography matrix is used to verify the remaining matching point pairs in the initial dataset until no new matching point pairs are added to the candidate inlier set. If the number of inliers in the candidate inlier set is greater than the number of inliers corresponding to the homography matrix, then the optimal homography matrix is updated; otherwise, it is not updated. Image registration is completed using the optimal homography matrix to obtain the target registered image.
4. The method for early warning of natural resource changes based on machine vision as described in claim 1, characterized in that, The step of extracting features from the preprocessed ground image data using a global feature enhancement learning network and a local feature enhancement learning network to generate feature image data includes: The preprocessed ground image data is acquired, and the LBP feature extraction algorithm is used to enhance the image texture features. The LBP texture features are used as one channel and stacked with the three channels of the RGB image of the preprocessed ground image data to form four-channel image information. The four-channel image information is input into a model using the SENet structure as the backbone network. The output of layer 4 in the SENet structure is used as the input of the global feature enhancement learning network to extract the first feature map. The first feature map is then flattened into multiple feature vectors through a linear projection module. Each feature vector is input into a Transformer encoding block, and global average pooling is performed on each vector to obtain a global feature vector. The outputs of layers 1, 2, and 3 in the SENet structure are used as inputs to the local feature enhancement learning network. The detail attention module is used to enhance the local features, which are then connected to the network through residuals and subjected to global average pooling to obtain the local feature vector. The global feature vector obtained from the global feature enhancement learning network and the local feature vector obtained from the local feature enhancement learning network are concatenated to obtain a fused feature vector, and feature image data is generated based on the fused feature vector.
5. The machine vision-based early warning method for natural resource changes as described in claim 4, characterized in that, The outputs of layers 1, 2, and 3 in the SENet structure are used as input to a local feature enhancement learning network. A detail attention module is employed to enhance local features, which are then connected to the network via residuals. After global average pooling, a local feature vector is obtained, including: The second feature map output by the SENet structure is obtained, and the second feature map is compressed into a single channel by a 1×1 convolution kernel, and the information between different channels is fused. We learn weights at different spatial locations by using the ReLU function and a 3×3 convolution kernel, and then normalize them using the sigmoid activation function to obtain spatial attention weights. The second feature map is subjected to global average pooling. Information from different channels is fused through a 1×1 convolution kernel, compressing the number of channels to 1 / 16. Then, it is expanded by 16 times through another 1×1 convolution kernel. The channel attention weights are obtained by normalizing the data using the sigmoid activation function. The spatial attention weight and the channel attention weight are multiplied together to obtain the detail attention weight, which is then multiplied by the corresponding position of the input image information and residually connected to obtain the local feature vector.
6. The method for early warning of natural resource changes based on machine vision as described in claim 1, characterized in that, The feature image data is input into the U-Net neural network model, and the changes in natural resources are detected based on the feature image data. The detection results are output, including: After the feature image data is input into the U-Net neural network model, the feature image data goes through 5 sets of convolutional layers. The first 3 sets of convolutions each perform max pooling downsampling on the feature image data, and the last 2 sets of convolutional layers perform overlapping pooling. A cascaded dilated convolution module is embedded in the 5th convolutional layer to obtain an intermediate feature map, wherein the intermediate feature map is 1 / 8 of the input feature image data; The intermediate feature maps obtained from each layer of the U-Net neural network model's encoding end are processed by a feature fusion module and used as input together with the feature vectors of the corresponding layers of the U-Net neural network model's decoding end. The feature fusion module includes an upper branch and a lower branch. The upper branch inputs the intermediate feature maps into a 3×3 convolution to extract features and obtain a low-dimensional tensor. The lower branch processes the intermediate feature maps through a 1×1 convolution and a 3×3 convolution to reduce dimensionality and extract global information, resulting in a high-dimensional feature tensor. The high-dimensional feature tensor is then upsampled using bilinear interpolation and multiplied pixel-by-pixel with the low-dimensional tensor. In the decoding module, channel weighting is performed, and spatial information is integrated through the global average pooling module to obtain the decoding output. The decoding output is then superimposed and upsampled sequentially to obtain the detection result.
7. The method for early warning of natural resource changes based on machine vision as described in claim 1, characterized in that, Based on the detection results, it is determined whether the changes in natural resources exceed a preset threshold. If so, an early warning message is generated, including: When natural resources change in terms of vegetation cover, determine whether the change in vegetation cover exceeds 12%. If so, generate an early warning message about vegetation cover. When changes in natural resources manifest as increases or decreases in water area, determine whether the increase or decrease in water area exceeds 5%. If so, generate an early warning message regarding the water area. When changes in natural resources manifest as changes in land area, it is determined whether the change in land area exceeds 10%. If so, an early warning message regarding the land area is generated.
8. A machine vision-based early warning system for changes in natural resources, characterized in that, The machine vision-based natural resource change early warning system includes an image acquisition module, an image preprocessing module, a feature extraction module, a detection module, an early warning module, and a labeling module. An image acquisition module is used to store ground image data captured by a drone in a distributed manner, evenly distribute the ground image data to multiple database nodes, and acquire the ground image data from multiple database nodes. The image preprocessing module is used to obtain the grayscale image of the ground image data by normalization processing, and to preprocess the grayscale image of the ground image data to obtain the preprocessed ground image data. The feature extraction module is used to extract features from the preprocessed ground image data through a global feature enhancement learning network and a local feature enhancement learning network to generate feature image data. The detection module is used to input the feature image data into the U-Net neural network model, detect the changes in natural resources based on the feature image data, and output the detection results. The early warning module is used to determine whether the changes in natural resources exceed a preset threshold based on the detection results. If so, an early warning message is generated. The changes in natural resources include at least changes in vegetation coverage, changes in water area, and changes in land area. The annotation module is used to output the early warning information to the terminals of relevant personnel and to mark the areas of natural resource change on the map.
9. A natural resource change early warning system based on machine vision as described in claim 8, characterized in that, The image preprocessing module includes an image segmentation submodule, a smoothing submodule, and an image registration submodule, wherein... The image segmentation submodule is used to obtain the grayscale image of the ground image data, and to adaptively generate a threshold using OTSU threshold segmentation to obtain the binary image corresponding to the grayscale image. The smoothing submodule is used to smooth the binary image corresponding to the grayscale image of the image using Gaussian filtering and to downsample it to obtain images of different scales; The image registration submodule is used to perform image registration on images of different scales using the ORB algorithm to obtain target registration images, and to perform noise reduction on the target registration images using morphological filling processing to obtain preprocessed ground image data.
10. A natural resource change early warning system based on machine vision as described in claim 8, characterized in that, The feature extraction module includes a stacking submodule, an input submodule, an average pooling submodule, an enhancement submodule, and a concatenation submodule, wherein... The stacking submodule is used to acquire preprocessed ground image data, and uses the LBP feature extraction algorithm to enhance the image texture features. The LBP texture features are used as one channel and stacked with the three channels of the RGB image of the preprocessed ground image data to form four-channel image information. The input submodule is used to input four-channel image information into a model using the SENet structure as the backbone network, and to use the output of layer 4 in the SENet structure as the input of the global feature enhancement learning network to extract the first feature map. The first feature map is then flattened into multiple feature vectors by the linear projection module. The average pooling submodule is used to input each feature vector into the Transformer encoding block and perform global average pooling on each vector to obtain a global feature vector. The enhancement submodule is used to take the outputs of layers 1, 2, and 3 in the SENet structure as input to the local feature enhancement learning network. It uses the detail attention module to enhance the local features, connects them to the network through residuals, and obtains the local feature vector after global average pooling. The splicing submodule is used to splice the global feature vector obtained by the global feature enhancement learning network and the local feature vector obtained by the local feature enhancement learning network to obtain the fused feature vector, and generate feature image data based on the fused feature vector.