Model feature encryption technology-based water conservation monitoring optimization method and monitoring equipment

Abstract

The invention provides a model feature encryption technology-based water conservation monitoring optimization method and monitoring equipment, and solves the problems of low pixel ratio of photos corresponding to key features of existing construction ground features, low subsequent three-dimensional modeling precision and the like. The method comprises the following steps: carrying out multi-source image data acquisition on the appearance of a to-be-constructed site through aerial photography; the method comprises the following steps: carrying out feature recognition on multi-source image data by a convolutional neural network model, fusing the image data through recognized common feature points and correspondingly carrying out three-dimensional modeling, finally, splicing with the convolutional neural network model through coordinate conversion, classifying each functional region in the three-dimensional model by the convolutional neural network model, and carrying out three-dimensional modeling on each functional region in the three-dimensional model. Meanwhile, introducing a potential water and soil loss coefficient as a judgment basis to judge whether the functional region is a water and soil loss key region of interest; and continuously iteratively optimizing the water and soil conservation measure position in the water and soil loss key interest area according to the classification result, thereby ensuring the accuracy of the obtained regional water and soil loss amount.

Description

Technical Field

[0001] The present invention relates to the technical field of soil and water conservation monitoring, and in particular to a soil and water conservation monitoring optimization method and monitoring equipment based on model feature encryption technology. Background Art

[0002] With the improvement of drone hardware, scholars have paid more and more attention to drone remote sensing, and the research content has become more in-depth and extensive, including theoretical exploration. Micro drones are light in weight, and their flight posture has a greater impact on the image. Some scholars have conducted research on image calibration and image stitching to address this problem. Since then, the wireless transmission system of drone images has gradually improved. With the gradual maturity of drone technology, researchers at home and abroad generally believe that this technology can be applied in many fields, such as emergency rescue, territorial monitoring, water conservation monitoring, forest stand surveys, engineering surveying and mapping, and many other fields.

[0003] Soil and water conservation monitoring is a key means to reflect the status of soil and water loss and the comprehensive management of soil and water loss in the study area. It is of great significance to the scientific implementation of soil and water conservation prevention and control measures and the maintenance of ecological balance. Remote sensing images with good timeliness and high resolution are playing an increasingly important role in soil and water conservation monitoring, and UAV remote sensing technology can easily and quickly obtain high-resolution remote sensing images of the study area. Therefore, it is feasible to apply UAV remote sensing technology in soil and water conservation monitoring.

[0004] The encryption of key features of three-dimensional models refers to the phenomenon that some waypoints are far away from the object being photographed, so the key features are not obvious. Therefore, it is necessary to encrypt the features of the above areas. In this case, because the image has rich color and texture information, scholars at home and abroad have preliminarily realized the top layer encryption of the forest model through the application of interpolation methods, such as weighted contrast average interpolation, local polynomial interpolation, approximate interpolation of nearest neighbor point cloud, and inverse weighted average interpolation. The method points out that although the TIN interpolation method is the optimal interpolation method for feature encryption, this interpolation method still has technical problems such as excessive reference data processing, cumbersome workload, and low efficiency. Therefore, a data distance weighted inverse interpolation method using local terrain search is proposed. This method interpolates the canopy model data. The results show that this interpolation method can not only efficiently remove the canopy pits in the canopy model, but also retain the actual gap between the objects to accurately express the canopy morphology of the forest and has a strong adaptability to the model downsampling density. However, this method relies on the uniform distribution of the downsampled data under model preprocessing, and is suitable for studying the encryption of model data with large height changes.

[0005] The traditional soil and water conservation monitoring method mainly calculates the amount of soil and water loss by setting up relevant soil and water conservation monitoring areas on site and analyzing data, and then estimates the overall soil loss in the project area. However, soil and water conservation measures may be damaged due to the impact of construction progress, thus affecting the soil and water conservation monitoring work in the project area. Therefore, it is necessary to optimize measures in a timely manner according to the project progress. Affected by the environmental conditions of the site, some water conservation measures are blocked by construction equipment, buildings and other objects. In the process of collecting images of the site, the photo pixels of the above-mentioned key features of the objects are relatively low, resulting in poor modeling quality of the objects and the inability to obtain key attribute features such as size and elevation.

[0006] To this end, we propose a water conservation monitoring optimization method and monitoring equipment based on model feature encryption technology to solve the above problems. Summary of the invention

[0007] The technical problem to be solved by the present invention is to overcome the defects of the prior art. The present invention proposes a water conservation monitoring optimization method and monitoring equipment based on model feature encryption technology to solve the above problems. By introducing close photogrammetry and combining the registration and splicing technology of deep image data, the feature encryption of key objects in the model is realized, and the data of interest can be completely extracted from the model after feature encryption. Finally, the water conservation measures of the site are optimized based on the three-dimensional model after feature encryption.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: a water conservation monitoring optimization method based on model feature encryption technology, comprising the following steps:

[0009] S1. Collect multi-source image data of the construction site through aerial photography;

[0010] S2. The convolution neural network model is used to perform feature recognition on the multi-source image data, and the image data is fused through the identified common feature points and three-dimensional modeling is performed accordingly. Finally, the convolution neural network model is used to classify each functional area in the three-dimensional model through coordinate transformation and splicing with the convolution neural network model. At the same time, the potential soil and water loss coefficient is introduced as a judgment basis to determine whether the functional area is a key interest area for soil and water loss;

[0011] S3. Based on the classification results, the locations of soil and water conservation measures in key interest areas of soil and water loss are continuously iteratively optimized to ensure the accuracy of the regional soil and water loss data.

[0012] Furthermore, the multi-source image data in step S1 includes aerial survey photos with POS information, photos taken by a camera, depth photos and point cloud data.

[0013] Furthermore, the steps of constructing the convolutional network model in step S2 are as follows:

[0014] Convolutional neural network layer by layer, and perform two-dimensional convolution dimensionality reduction through convolution kernel;

[0015] Introducing a spatial attention mechanism into the above layered convolutional neural network to perform an inverse residual convolution layer;

[0016] Use the feature pyramid network FPN to fuse different feature layers in multi-source image data through bottom-up, top-down and lateral connections to generate feature maps accordingly;

[0017] Construct subnetworks, including classification subnetwork, regression subnetwork and fully connected subnetwork, where the loss function of the fully connected subnetwork is calculated based on softmax loss.

[0018] Furthermore, the classification step in step S2 is specifically as follows:

[0019] The edge uploads multi-source image data to the cloud, performs synchronous calculations to generate feature point information, and then the convolutional neural network model predicts the feature point information through the classification sub-network, combines the potential soil and water loss coefficient to suppress the extraction of the best feature point information and outputs it, and uploads the best feature point information and corresponding images to the cloud in a separate channel;

[0020] The cloud side rechecks the images uploaded by the edge side, re-extracts all the feature point information corresponding to all the images and outputs them through the regression sub-network in the classification model. The output feature vector is matched with the best feature point information uploaded by the edge side for similarity, and the images corresponding to those above the similarity threshold are determined as key areas of interest for soil and water loss.

[0021] Furthermore, the similarity threshold is 0.6.

[0022] A monitoring device according to the above-mentioned water conservation monitoring optimization method based on model feature encryption technology comprises:

[0023] The acquisition module uses a variety of sensor devices to take multi-angle aerial photos of the construction area;

[0024] The classification module has a built-in convolutional neural network model, which communicates with a variety of sensor devices to perform convolution processing and classification on the uploaded image data. The classification process first predicts the current uploaded image data, then combines the potential soil and water loss coefficient corresponding to the current construction area to suppress and extract the best feature point information and the corresponding image, and then rechecks all image data to match the similarity with the best feature point information, and re-identifies all key areas of interest for soil and water loss;

[0025] The output module is used to fuse the image data uploaded by various sensor devices into a three-dimensional model, and simultaneously connect it with the convolutional neural network model through coordinate transformation to realize the identification and judgment of key areas of interest of soil erosion in the three-dimensional model.

[0026] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the water conservation monitoring optimization method based on model feature encryption technology are implemented.

[0027] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, is used to implement the steps of the water conservation monitoring optimization method based on model feature encryption technology described above.

[0028] Compared with the prior art, the beneficial effects of the present invention include:

[0029] 1. The image data of the key area of ​​the ground object obtained by the sensor is spliced ​​and fused through the relevant registration and splicing algorithm, and finally the fused photo pair is used for 3D modeling to complete the encryption of the key feature area of ​​the ground object;

[0030] 2. Classification of site functional area characteristics: Use the refined model and orthophoto to classify the functional areas of the construction site, and introduce the potential soil and water loss coefficient as a basis for judging whether the functional area is a key area of ​​interest for soil and water loss;

[0031] 3. Optimization of soil and water loss measures based on feature classification: timely optimize and modify the location of soil and water conservation measures based on the classification results to ensure the accuracy of soil and water loss in the project area. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings, the same reference numerals are used to refer to the same components. Among them:

[0033] Figure 1 The schematic diagram shows the overall method flow chart according to one embodiment of the present invention. DETAILED DESCRIPTION

[0034] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0035] According to one embodiment of the present invention, Figure 1 Shown.

[0036] As for the overall method process, in this embodiment, the water conservation monitoring optimization method based on the model feature encryption technology includes the following steps:

[0037] S1. Collect multi-source image data of the construction site through aerial photography;

[0038] S2. The convolution neural network model is used to perform feature recognition on the multi-source image data, and the image data is fused through the identified common feature points and three-dimensional modeling is performed accordingly. Finally, the convolution neural network model is used to classify each functional area in the three-dimensional model through coordinate transformation and splicing with the convolution neural network model. At the same time, the potential soil and water loss coefficient is introduced as a judgment basis to determine whether the functional area is a key interest area for soil and water loss;

[0039] S3. Based on the classification results, the locations of soil and water conservation measures in key interest areas of soil and water loss are continuously iteratively optimized to ensure the accuracy of the regional soil and water loss data.

[0040] For the construction of convolutional neural network model:

[0041] Backbone network

[0042] Construct the backbone network ResNet50 residual network. Based on the residual mapping method of H(x)=F(x)+x, construct the five blocks Res1, Res2, Res3, Res4, and Res5 in turn. Their downsampling rates are 2^1, 2^2, 2^3, 2^4, and 2^5 respectively. Usually RetinaNet will select three modules as the initial detection layer, namely Res3, Res4, and Res5.

[0043] Feature Pyramid Network

[0044] The feature pyramid network FPN is used to fuse the different feature layers of ResNet50 through bottom-up, top-down and lateral connections. The top-down and bottom-up lines generate feature maps such as Res3, Res4, Res5 and P3, P4, P4, P5, P6, and P7 respectively. P3 to P5 are calculated from Res3 to Res5, and P6 to P7 are to enable the model to better detect large objects. Thanks to the larger receptive field, such operations can ensure that each layer has appropriate resolution and strong semantic features, and cooperate with the target detection algorithm and Focal Loss to improve the detection performance of objects.

[0045] The specific steps of the entire model construction structure are as follows:

[0046] The first layer of the neural network is constructed, and the zeroth layer is the convolution layer (conv2d_1): the convolution kernel size is 3×3, the number of kernels is 32, and the stride is 2. The input image with an input size of 416×416×3 is processed by convolution and the output image size is 208×208×32.

[0047] Construct the second layer of the neural network, which is the convolution layer (conv2d_2): the convolution kernel size is 3×3, the number of kernels is 64, and the stride is 2. The input image of size 208×208×32 is processed by convolution and the output image size is 208×208×64.

[0048] Construct the third layer of the neural network, which is the inverse residual convolution layer (bneck_1): The inverse residual convolution includes 2 1×1 convolutions and 1 3×3 convolution, and each convolution layer is followed by a BN layer and a ReLU activation function. The 208×208×64 feature map is output after the inverse residual convolution to a feature map size of 208×208×64, which is output to bneck_2;

[0049] Construct the fourth layer of the neural network, which is the inverse residual convolution layer (bneck_2): The inverse residual convolution includes 2 1×1 convolutions and 1 3×3 convolution, and each convolution layer is followed by a BN layer and a ReLU activation function. The 208×208×64 feature map is output with a feature map size of 104×104×128 after the inverse residual convolution, and is output to bneck_3;

[0050] Construct the fifth layer of the neural network, which is the inverse residual convolution layer (bneck_3): the feature map with a size of 104×104×128 is output after inverse residual convolution with a feature map size of 52×52×256, which is output to samBneck_1.

[0051] Construct the sixth layer of the neural network. The sixth layer is the sam inverse residual convolution (samBneck_1): the feature map with a size of 52×52×256 is output after the sam inverse residual convolution, and the feature map size is 52×52×256, which is output to bneck_4.

[0052] Construct the seventh layer of the neural network, which is the inverse residual convolution layer (bneck_4): the feature map with a size of 52×52×256 is output after inverse residual convolution to a feature map size of 26×26×512, which is output to samBneck_2.

[0053] Construct the eighth layer of the neural network. The eighth layer is sam inverse residual convolution (samBneck_2): the feature map with a size of 26×26×512 is output after sam inverse residual convolution, and the output feature map is 26×26×512, and the output is bneck_5.

[0054] Construct the ninth layer of the neural network, which is the inverse residual convolution layer (bneck_5): the feature map with a size of 26×26×512 is output after the inverse residual convolution to a feature map of 13×13×1024, and the output is samBneck_3.

[0055] Construct the tenth layer of the neural network. The tenth layer is sam inverse residual convolution 1 (samBneck_3): the feature map with a size of 13×13×1024 is output after sam inverse residual convolution, and the output feature map is 13×13×1024, and the output is bneck_6.

[0056] Furthermore, the present application first uploads multi-source image data to the cloud at the edge, performs synchronous calculations to generate feature point information, and then the convolutional neural network model first predicts the feature point information through a classification subnetwork, combines the potential soil and water loss coefficient to suppress the extraction of the best feature point information and outputs it, and uploads the best feature point information and the corresponding image to the cloud in a separate channel;

[0057] The cloud then rechecks the images uploaded by the edge, re-extracts all the feature point information corresponding to all the images and outputs them through the regression subnetwork in the classification model, and matches the output feature vector with the best feature point information uploaded by the edge for similarity. Images corresponding to images above the similarity threshold are determined to be key areas of interest for soil and water loss.

[0058] In some embodiments, the similarity threshold is 0.6.

[0059] Similarly, the monitoring equipment using the above-mentioned water conservation monitoring optimization method based on model feature encryption technology is also within the protection scope of the present invention. Specifically, the system may include:

[0060] The acquisition module uses a variety of sensor devices to take multi-angle aerial photos of the construction area;

[0061] The classification module has a built-in convolutional neural network model, which communicates with a variety of sensor devices to perform convolution processing and classification on the uploaded image data. The classification process first predicts the current uploaded image data, then combines the potential soil and water loss coefficient corresponding to the current construction area to suppress and extract the best feature point information and the corresponding image, and then rechecks all image data to match the similarity with the best feature point information, and re-identifies all key areas of interest for soil and water loss;

[0062] The output module is used to fuse the image data uploaded by various sensor devices into a three-dimensional model, and simultaneously connect it with the convolutional neural network model through coordinate transformation to realize the identification and judgment of key areas of interest of soil erosion in the three-dimensional model.

[0063] In addition, the above-mentioned method steps and system can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0064] The technical scope of the present invention is not limited to the contents in the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. The water conservation monitoring optimization method based on model feature encryption technology is characterized by: The steps include: S1. Collect multi-source image data of the construction site through aerial photography; S2. The convolution neural network model is used to perform feature recognition on the multi-source image data, and the image data is fused through the identified common feature points and three-dimensional modeling is performed accordingly. Finally, the convolution neural network model is used to classify each functional area in the three-dimensional model through coordinate transformation and splicing with the convolution neural network model. At the same time, the potential soil and water loss coefficient is introduced as a judgment basis to determine whether the functional area is a key interest area for soil and water loss; S3. Based on the classification results, the locations of soil and water conservation measures in key interest areas of soil and water loss are continuously iteratively optimized to ensure the accuracy of the regional soil and water loss data.

2. The water conservation monitoring optimization method based on model feature encryption technology according to claim 1 is characterized in that: The multi-source image data in step S1 includes aerial survey photos with POS information, photos taken by a camera, depth photos and point cloud data.

3. The water conservation monitoring optimization method based on model feature encryption technology according to claim 1 is characterized in that: The steps of constructing the convolutional network model in step S2 are as follows: Convolutional neural network layer by layer, and perform two-dimensional convolution dimensionality reduction through convolution kernel; Introducing a spatial attention mechanism into the above layered convolutional neural network to perform an inverse residual convolution layer; Use the feature pyramid network FPN to fuse different feature layers in multi-source image data through bottom-up, top-down and lateral connections to generate feature maps accordingly; Construct subnetworks, including classification subnetwork, regression subnetwork and fully connected subnetwork, where the loss function of the fully connected subnetwork is calculated based on softmax loss.

4. The water conservation monitoring optimization method based on model feature encryption technology according to claim 3 is characterized in that: The classification steps in step S2 are specifically as follows: The edge uploads multi-source image data to the cloud, performs synchronous calculations to generate feature point information, and then the convolutional neural network model predicts the feature point information through the classification sub-network, combines the potential soil and water loss coefficient to suppress the extraction of the best feature point information and outputs it, and uploads the best feature point information and corresponding images to the cloud in a separate channel; The cloud side rechecks the images uploaded by the edge side, re-extracts all the feature point information corresponding to all the images and outputs them through the regression sub-network in the classification model. The output feature vector is matched with the best feature point information uploaded by the edge side for similarity, and the images corresponding to those above the similarity threshold are determined as key areas of interest for soil and water loss.

5. The water conservation monitoring optimization method based on model feature encryption technology according to claim 4 is characterized in that: The similarity threshold is 0.

6.

6. A monitoring device according to the water conservation monitoring optimization method based on model feature encryption technology according to any one of claims 1 to 5, characterized in that: include: The acquisition module uses a variety of sensor devices to take multi-angle aerial photos of the construction area; The classification module has a built-in convolutional neural network model, which communicates with a variety of sensor devices to perform convolution processing and classification on the uploaded image data. The classification process first predicts the current uploaded image data, then combines the potential soil and water loss coefficient corresponding to the current construction area to suppress and extract the best feature point information and the corresponding image, and then rechecks all image data to match the similarity with the best feature point information, and re-identifies all key areas of interest for soil and water loss; The output module is used to fuse the image data uploaded by various sensor devices into a three-dimensional model, and simultaneously connect it with the convolutional neural network model through coordinate transformation to realize the identification and judgment of key areas of interest of soil erosion in the three-dimensional model.

7. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that : When the processor executes the program, the steps of the water conservation monitoring optimization method based on model feature encryption technology as described in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it is used to implement the steps of the water conservation monitoring optimization method based on model feature encryption technology as described in any one of claims 1 to 5.

Images