Surface collapse monitoring method and system based on twin neural network fusion

Abstract

The invention relates to the technical field of surface collapse monitoring, in particular to a surface collapse monitoring method based on twin neural network fusion, and the method comprises the steps: obtaining the monitoring range of surface collapse, and laying control points and target points; acquiring a real-time InSAR scanning image, and performing first preprocessing on the real-time InSAR scanning image to obtain an InSAR collapse field image; obtaining a plurality of real-time 3D local scanning images, performing second preprocessing on the real-time 3D local scanning images, and processing the real-time 3D local scanning images through a twin neural network to obtain a plurality of 3D local collapse field images; formulating a weight rule, marking the weight of the plurality of 3D local collapse field images according to the weight rule, and splicing the 3D local collapse field images to obtain a 3D global collapse field image with weight marks; and splicing the InSAR collapse field image and the 3D global collapse field image with the weight mark through a twin neural network to obtain an InSAR seamless collapse field fusion image, and monitoring the InSAR seamless collapse field fusion image to obtain a monitoring result. According to the method, the defects of a single technical means can be overcome, and high-precision and full-coverage monitoring of the surface collapse is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of ground subsidence monitoring, and more specifically, to a ground subsidence monitoring method and system based on twin neural network fusion. Background Art

[0002] Ground collapse is a common geological disaster, usually caused by natural factors such as excessive groundwater exploitation, seismic activity, karstification, etc., and may also be caused by human factors such as mining activities, tunnel construction, underground engineering, etc. This phenomenon can cause the surface to sink, affect the stability of buildings, damage infrastructure, and even endanger human life. Ground collapse is often sudden and hidden, which brings great challenges to monitoring and early warning work.

[0003] Existing ground subsidence monitoring methods mainly rely on a single technical means. For example, the interferometric synthetic aperture radar InSAR technology can provide a large range of surface deformation data, but there may be data missing or errors; 3D laser scanning technology can provide high-precision local surface deformation data, but the monitoring range is limited; the existing technology also combines the above two technologies, but only uses simple splicing technology, which cannot achieve high accuracy and precise monitoring. The above methods have their own advantages and disadvantages, and cannot independently fully cover and monitor ground subsidence with high accuracy. Therefore, it is necessary to make necessary improvements to the ground subsidence method. Summary of the invention

[0004] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a ground subsidence monitoring method and system based on twin neural network fusion to achieve high-precision and full-coverage monitoring of ground subsidence.

[0005] Acquire a monitoring range of ground collapse, and pre-deploy control points and target points in the monitoring range;

[0006] Acquire a real-time InSAR scanning image of the monitoring range, perform a first preprocessing on the InSAR scanning image, and acquire an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR;

[0007] Acquire a plurality of real-time 3D local scanning images of the monitoring range, perform a second preprocessing on the plurality of 3D local scanning images, and process the plurality of 3D local scanning images that have completed the second preprocessing by means of a twin neural network to acquire a plurality of 3D local collapse field images, wherein the 3D local scanning images are acquired by means of 3D laser scanning;

[0008] Formulate a weight rule according to the control point and the target point, the InSAR collapse field image and the plurality of 3D local collapse field images, and perform weight marking on the plurality of 3D local collapse field images according to the weight rule and then splice them to obtain a 3D global collapse field image with weight marking;

[0009] The InSAR collapse field image and the weight-labeled 3D global collapse field image are spliced ​​by the twin neural network to obtain an InSAR seamless collapse field fusion image, and the InSAR seamless collapse field fusion image is monitored to obtain the monitoring result of the monitoring range.

[0010] It is understandable that, because of the current technology, the monitoring of ground collapse mainly uses interferometric synthetic aperture radar InSAR and 3D laser scanning technology for scanning monitoring, but using interferometric synthetic aperture radar InSAR technology alone for monitoring, although it can monitor a large range of terrain, there will be serious data errors in some areas; such as using 3D laser scanning technology alone for monitoring, although it can obtain high-precision scanning data, it can only scan a small range of terrain areas at a time, and subsequent complex and large amounts of splicing work are required, and there will also be large superposition errors at the boundaries of the splicing work. Therefore, this application combines interferometric synthetic aperture radar InSAR and 3D laser scanning technology to monitor the collapsed ground, and uses twin neural networks and weight rules for splicing, which reduces the errors caused by splicing, accurately monitors important areas, and ensures that the important areas of the formed collapse field image have relatively high-precision data, and the coverage is comprehensive.

[0011] Optionally, the InSAR collapse field image is a perforated InSAR collapse field image, wherein the perforated area of ​​the perforated InSAR collapse field image is obtained by removing a number of error areas using a masking method, wherein the attribute data of the error area meets a preset error threshold.

[0012] It is understandable that the mask method is used to remove the error area of ​​the InSAR collapse field image to form holes, which can ensure that the error area will not affect the subsequent stitching work, so that the data used in the stitching work is basically reliable data, thereby improving the accuracy of the entire collapse field image.

[0013] Optionally, the formulating a weight rule according to the control point and the target point, the InSAR collapse field image and a plurality of the 3D local collapse field images specifically includes:

[0014] Taking the control point and the target point as the first selected area;

[0015] taking obvious collapse areas in the InSAR collapse field image and a plurality of the 3D local collapse field images as second selected areas, wherein the attribute data of the obvious collapse areas meet a preset collapse data threshold;

[0016] The area prone to collapse is used as the third selected area, wherein the area prone to collapse meets the preset collapse risk attribute;

[0017] taking a number of high-quality regions in the 3D local collapse field images as fourth selected regions, wherein attribute data of the high-quality regions meets a preset data quality threshold;

[0018] Weights are set for the first selected area, the second selected area, the third selected area and the fourth selected area respectively, and four independent weight rules are formulated for the first selected area, the second selected area, the third selected area and the fourth selected area and their corresponding weights.

[0019] It is understandable that for the monitoring range, some accurate data is needed as a benchmark for other data, so control points need to be set. Because the positions of the control points are selected on objects that are not easy to change, this ensures that the coordinates of the control points have long-term stability and representativeness. In the 3D local collapse field image, the coordinates corresponding to the control points can be found, and their weights can be increased to give priority to splicing these areas, thereby providing a reliable benchmark for subsequent measurement work; corresponding to the monitoring range, target points need to be arranged to facilitate the registration of 3D point cloud data. In the 3D local collapse field image, the coordinates corresponding to the target points can be found, and their weights can be increased. Weights are set to give priority to splicing these areas and provide registration data for the 3D scanning image; corresponding to the areas where obvious collapse data can be found in the InSAR collapse field image and the 3D local collapse field image, it is indicated that the area may have collapse phenomenon, or it may be error data, so priority is given to splicing this area to reduce the impact of other data on the data in this area; for some areas in the monitoring range, there may be a risk of collapse due to terrain and historical factors. The areas prone to collapse in the monitoring range can be recorded, and the weight of the area can be increased. Priority is given to splicing, and the data in the above-mentioned areas are monitored in detail to ensure the reliability and high accuracy of the data.

[0020] Optionally, the step of weight-marking a plurality of the 3D local collapse field images according to the weight rule and then splicing them to obtain a 3D global collapse field image with weight marking specifically includes:

[0021] Traversing each of the 3D local collapse field images, weight-marking the regions in each of the 3D local collapse field images according to the weight rule, and obtaining a plurality of 3D local collapse field images with weight marks;

[0022] Obtaining all weights on each of the 3D local collapse field images with weight marks, calculating all the obtained weights, and obtaining the total weight of each of the 3D local collapse field images;

[0023] The plurality of 3D local collapse field images with weight marks are spliced ​​in sequence according to the corresponding total weights to obtain a 3D global collapse field image with weight marks.

[0024] It is understandable that the weights are set for the areas in each 3D local collapse field image according to the weight rule to provide a priority for the subsequent stitching work. Because in the process of stitching, the boundary data of the two scanned images are integrated and spliced ​​according to the algorithm, and the boundary data of the two scanned images will affect the integrated data. If there is an error in one of the boundary data, there will be an error in the spliced ​​data; and the spliced ​​integrated data will also be spliced ​​with other boundary data, which will form an accumulation of errors. Using weight rules to limit the stitching order can give priority to the stitching of the areas you want to focus on, which can ensure that the accuracy of the areas of focus will not be affected by multiple integration and splicing, thereby improving the reliability of the monitoring data.

[0025] Optionally, weight marking each region in the 3D local collapse field image according to the weight rule specifically includes:

[0026] Marking a region of the 3D local collapse field image that satisfies one of the weight rules with a corresponding weight;

[0027] For regions of the 3D local collapse field image that satisfy two or more of the weight rules, weight superposition marking is performed;

[0028] For other regions of the 3D local collapse field image that do not meet the weight rule, the weights of the other regions are all set to the initial weights.

[0029] It can be understood that corresponding weights are set for the areas of the 3D local collapse field image that meet the weight rules, and for the areas of the 3D local collapse field image that meet multiple weight rules, the weight superposition is set to splice them in a more prioritized order; for other areas of the 3D local collapse field image that do not meet the weight rules, initial weights are set, and then the twin neural network is used to splice them in sequence according to the total weights of each 3D local collapse field image, to ensure that 3D local collapse field images with high weights are spliced ​​first, thereby ensuring the data reliability of these areas.

[0030] Optionally, the initial weights are all smaller than the weights set in the weight rules.

[0031] It is understandable that the initial weight is smaller than the weight set in the weight rule, which can ensure that the area satisfying the weight rule is spliced ​​first, and other areas are spliced ​​after the area satisfying the weight rule is spliced.

[0032] Optionally, the step of splicing the InSAR collapse field image and the weight-labeled 3D global collapse field image by the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitoring the InSAR seamless collapse field fusion image to obtain the monitoring result of the monitoring range specifically includes:

[0033] Respectively acquiring phase information and amplitude information of the InSAR collapse field image, and encoding the phase information and the amplitude information in different channels through an InSAR data encoder to obtain InSAR collapse field encoded data, wherein the InSAR encoder adopts a convolutional neural network structure;

[0034] The 3D global collapse field image with the weight mark is encoded by a 3D point cloud data encoder to obtain 3D global collapse field encoded data, wherein the 3D point cloud data encoder adopts a point cloud convolutional neural network structure;

[0035] The 3D global collapse field coded data and the InSAR collapse field coded data are input into the twin neural network, the twin neural network extracts information of the area with holes in the InSAR collapse field image from the InSAR collapse field coded data, and uses the 3D global collapse field coded data and the information of the area with holes to splice and fill the area with holes, so as to obtain an InSAR seamless collapse field image;

[0036] The twin neural network extracts a weighted area from the 3D global collapse field coded data as a selected area, obtains a selected area corresponding to the InSAR seamless collapse field image according to the selected area, and uses the 3D global collapse field coded data to perform permutation fusion on the corresponding selected area in the InSAR seamless collapse field image to obtain an InSAR seamless collapse field fused image;

[0037] The InSAR seamless collapse field fusion images at different times are monitored to obtain monitoring results of the monitoring range.

[0038] It can be understood that by using different encoders to encode the InSAR collapse field image and the 3D global collapse field image, different data can be processed correspondingly, so that the characteristics of the two data can be captured more accurately; by using the twin neural network to process the InSAR collapse field image and the 3D global collapse field image, the joint feature vector of the two data can be extracted quickly and accurately, so that by calculating the similarity between the joint feature vectors, the data changes at different times can be monitored and the collapse area can be accurately captured.

[0039] According to a second aspect of the present application, a ground collapse monitoring system based on twin neural network fusion is provided, comprising:

[0040] A range acquisition module, used to acquire a monitoring range of ground collapse, in which control points and target points are pre-arranged;

[0041] An InSAR data acquisition-processing module, used for acquiring a real-time InSAR scanning image of the monitoring range, performing a first preprocessing on the InSAR scanning image, and acquiring an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR;

[0042] A 3D scanning local data acquisition-processing module, used to acquire a plurality of 3D local scanning images of the monitoring range in real time, perform a second preprocessing on the plurality of 3D local scanning images, and process the plurality of 3D local scanning images that have completed the second preprocessing through a twin neural network to acquire a plurality of 3D local collapse field images, wherein the 3D local scanning images are acquired through 3D laser scanning;

[0043] A 3D scanning global data acquisition module is used to formulate a weight rule according to the control point and the target point, the InSAR collapse field image and a plurality of the 3D local collapse field images, and to perform weight marking on the plurality of the 3D local collapse field images according to the weight rule and then to splice them to obtain a 3D global collapse field image with weight marking;

[0044] The splicing monitoring module is used to splice the InSAR collapse field image and the 3D global collapse field image with weight marking through the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitor the InSAR seamless collapse field fusion image to obtain the monitoring result of the monitoring range.

[0045] According to the third aspect of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement a ground collapse monitoring method based on twin neural network fusion as described in the first aspect above.

[0046] According to a fourth aspect of the present application, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed, a ground collapse monitoring method based on twin neural network fusion as described in the first aspect above is implemented.

[0047] Based on any of the above aspects, a ground collapse monitoring method and system based on twin neural network fusion provided in the embodiment of the present application can first obtain the monitoring range of ground collapse, then obtain InSAR collapse field images, and preprocess them; obtain several 3D local collapse field images; splice several of the 3D local collapse field images according to the weight rule through the twin neural network to obtain a 3D global collapse field image with a weight mark, and preprocess it; finally, the InSAR collapse field image and the 3D global collapse field image with a weight mark are spliced ​​through the twin neural network to obtain an InSAR seamless collapse field fusion image, and the InSAR seamless collapse field fusion image is monitored to obtain the monitoring results of the monitoring range. This method can make up for the shortcomings of a single technical means and realize high-precision and full-coverage monitoring of ground collapse. By combining interferometric synthetic aperture radar InSAR interferometry with 3D laser scanning technology, the advantages of both can be fully utilized. The twin neural network and weight rules are used to process the two types of data to achieve comprehensive monitoring of ground collapse, improve the accuracy and reliability of monitoring, and timely discover and warn of potential geological disasters, thereby effectively protecting human life and property. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A flow chart of a ground collapse monitoring method based on twin neural network fusion is provided in this embodiment.

[0050] Figure 2 The following is a flow chart of a method for obtaining weight rules according to this embodiment.

[0051] Figure 3This is a flow chart of a method for obtaining a 3D global collapse field image according to this embodiment.

[0052] Figure 4 This is a diagram of a weight marking situation of this embodiment.

[0053] Figure 5 The flowchart of the method for acquiring and monitoring the InSAR seamless collapse field fusion image is shown in the embodiment.

[0054] Figure 6 A module diagram of a ground collapse monitoring system based on twin neural network fusion provided in this embodiment.

[0055] Figure 7 This is a device structure diagram of the electronic device provided in this embodiment. DETAILED DESCRIPTION

[0056] The drawings of this application are only used for illustrative purposes and should not be construed as limiting the present application. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] Ground collapse is a common geological disaster, which is sudden and hidden, bringing great challenges to monitoring and early warning. The existing technology generally uses interferometric synthetic aperture radar InSAR or 3D laser scanning technology to monitor the ground, but the use of the above two technologies alone has its own advantages and disadvantages. This application can combine the advantages of the two technologies, avoid their shortcomings, and use twin neural networks and formulated weight rules to improve the accuracy of data processing, so as to achieve high-precision and full-coverage monitoring of ground collapse.

[0060] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below in conjunction with the accompanying drawings.

[0061] For example, Figure 1 A method flow chart of a ground collapse monitoring method based on twin neural network fusion provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0062] S110, obtaining a monitoring range of ground collapse, and pre-arranging control points and target points in the monitoring range;

[0063] It is understandable that obtaining the monitoring range of ground collapse and defining the scanning boundary for subsequent scanning work can provide clear boundary targets for the range that needs to be monitored, and also play a huge guiding role in the subsequent stitching and registration of the two scanned images; the deployment of control points can serve as a benchmark for the processing of subsequent scanned images. Preferably, the control points need to be set in landmarks where the terrain is not easily changed, so that they can be compared in multiple scanned images. It is understandable that it is necessary to obtain accurate attribute information such as height data of the control points in advance, and compare the control point information of multiple scanned images with the accurate control point information, so as to measure the error degree of the scanned image; it is understandable that the target points are used for the registration of point cloud data.

[0064] S120, acquiring a real-time InSAR scanning image of the monitoring range, performing a first preprocessing on the InSAR scanning image, and acquiring an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR;

[0065] It is understandable that the InSAR scanning image obtained by the interferometric synthetic aperture radar InSAR corresponds to a single moment, which includes the attribute information of the ground in the monitoring range at the current moment. Preferably, the attribute information includes the phase information and amplitude information of the ground in the monitoring range; it is understandable that the InSAR collapse field image obtained based on the InSAR scanning image corresponds to a time period, and the InSAR collapse field image reflects the collapse data of the ground in the monitoring range.

[0066] Exemplarily, the present application provides an embodiment to simulate the acquisition of InSAR collapse field images:

[0067] At time t0, the InSAR scanning image at time t0 is obtained by using the interferometric synthetic aperture radar InSAR;

[0068] At time t1, an InSAR scanning image at time t1 is obtained by using an interferometric synthetic aperture radar InSAR; it is understandable that the time interval between time t0 and time t1 needs to be set in advance, and preferably, the time interval is set to a few days or several months;

[0069] It is understandable that in order to monitor ground deformation, it is usually necessary to perform imaging acquisition multiple times at the same track and angle. The InSAR scanned image is preprocessed, including but not limited to one or more of the existing image processing technologies of image registration, noise removal, interferogram generation, and phase unwrapping:

[0070] Image registration: Geometrically align multiple acquired InSAR scans to ensure they cover the same ground area;

[0071] Noise removal: Eliminate errors due to atmospheric interference, ionospheric effects and other noise sources through filtering;

[0072] Generate an interference pattern:

[0073] Interference pattern calculation: The interference pattern is obtained by multiplying the two registered InSAR scan images by complex numbers. The interference pattern contains phase information and reflects the ground height and deformation information;

[0074] Removing flat-earth effects: Using digital elevation models (DEMs) to remove flat-earth effects in the interferogram, thereby highlighting ground deformations;

[0075] Phase unwrapping: Since the phase value in the interferogram varies in the range of [-π,π], phase unwrapping is required to obtain continuous phase information; this step can obtain the actual surface displacement information.

[0076] By comparing the preprocessed InSAR scanning image at time t0 and the InSAR scanning image at time t1, the area where the phase, amplitude and other parameters change can be obtained, the phase difference information of the attributes is calculated, the InSAR collapse data of the monitoring range in the time period from t0 to t1 is obtained, and the InSAR collapse field image is generated.

[0077] It is understandable that the InSAR scanning image obtained by using interferometric synthetic aperture radar scanning is a large-scale image. Due to the wide scanning range, the number of scans can be reduced, but it will also cause large errors in some areas, thereby causing large errors in some areas of the generated InSAR collapse field image.

[0078] Specifically, the InSAR collapse field image is a perforated InSAR collapse field image, wherein the perforated area of ​​the perforated InSAR collapse field image is obtained by removing several error areas using a masking method, wherein the attribute data of the error area meets a preset error threshold.

[0079] It is understandable that, for the identified data that meets the preset error threshold, the present application uses a masking method to remove the above error data. It is understandable that the InSAR collapse field image with the error data removed is a hole-containing InSAR collapse field image, and the holes are areas where the ground attribute information of the area is missing, and the monitoring of the area needs to be supplemented in subsequent work; wherein the error threshold can be used as a reference data for setting the large-scale error caused by phase unwrapping errors, noise, etc. in the interferogram processing step.

[0080] S130, obtaining a plurality of 3D local scanning images of the monitoring range in real time, performing a second preprocessing on the plurality of 3D local scanning images, and processing the plurality of 3D local scanning images that have completed the second preprocessing by means of a twin neural network to obtain a plurality of 3D local collapse field images, wherein the 3D local scanning images are obtained by means of 3D laser scanning;

[0081] It is understandable that the 3D local scanning image obtained by 3D laser scanning technology is local in scope, and the data has certain reliability and high precision, but it requires multiple scanning at different scanning angles and positions to obtain several 3D local scanning images and generate several 3D local collapse field images. The relationship between the 3D local scanning image and the 3D local collapse field image is the same as the relationship between the above-mentioned InSAR scanning image and the InSAR collapse field image, and will not be repeated here.

[0082] It is understandable that the preprocessing of several 3D local scan images requires the use of the data of the control points and the target points to align and fuse the point cloud data of different scanning angles and positions. The classification algorithm is applied to divide the point cloud data into ground points and non-ground points, and the data is filtered; the denoising algorithm is used to remove isolated points and noise in the point cloud data, and the point cloud data of the 3D local scan image is aligned with the existing geographic coordinate system, and the coordinate conversion algorithm is applied to convert the point cloud data to the required geographic coordinate system.

[0083] It is understandable that for a local 3D scanned deformable image, there may be a large amount of redundant information in the point cloud data, such as duplicate points, non-critical areas, etc. If the above preprocessing is performed on all the point cloud data, it will take up too many computing resources and there will be a lot of redundant calculations. Therefore, this application adopts a random area selection mechanism. In the process of selecting multiple local point cloud areas of a local 3D scanned deformable image, a range [n1, n1] is set, and n areas are randomly selected from all point cloud areas for preprocessing calculations, where n∈[n1, n2], and is a random integer. The random area selection mechanism avoids redundant calculations of global point cloud data, improves stitching efficiency while maintaining the diversity of area selection. Using randomly selected n areas for processing can avoid redundancy.

[0084] It is understandable that the twin neural network can compare the similarity of two inputs and capture subtle differences between the two input samples. In this example, for the 3D local scan images of the same area at different times, theoretically, if there is no obvious collapse, the 3D local scan images at two times are very similar. The twin neural network is used to find the coordinates where the terrain data of the 3D local scan images at two times have slight changes. The difference attributes of such coordinates can be calculated to generate several 3D local collapse field images. The use of twin neural networks can liberate the manual comparison process and achieve more precise results.

[0085] It is understandable that for the several 3D local scan images, the amount of data is huge, and scanning multiple local areas will produce a lot of redundant data, and increase the risk of errors. If all 3D local scan images are spliced ​​in sequence, errors will accumulate. Therefore, it is necessary to focus on the areas that need attention and give priority to splicing to maintain the data reliability of the area and achieve accurate monitoring.

[0086] S140, formulating a weight rule according to the control point and the target point, the InSAR collapse field image and the plurality of 3D local collapse field images, weight-marking the plurality of 3D local collapse field images according to the weight rule and then splicing them to obtain a 3D global collapse field image with weight marking;

[0087] Specifically, Figure 2 As shown, the method of formulating the weight rule can be divided into the following cases:

[0088] Taking the control point and the target point as the first selected area;

[0089] taking obvious collapse areas in the InSAR collapse field image and a plurality of the 3D local collapse field images as second selected areas, wherein the attribute data of the obvious collapse areas meet a preset collapse data threshold;

[0090] The area prone to collapse is used as the third selected area, wherein the area prone to collapse meets the preset collapse risk attribute;

[0091] taking a number of high-quality regions in the 3D local collapse field images as fourth selected regions, wherein attribute data of the high-quality regions meets a preset data quality threshold;

[0092] Weights are set for the first selected area, the second selected area, the third selected area and the fourth selected area respectively, and four independent weight rules are formulated for the first selected area, the second selected area, the third selected area and the fourth selected area and their corresponding weights.

[0093] It is understandable that to stitch two scanned images, the boundary data of the two scanned images need to be merged and calculated according to the algorithm to obtain the stitched boundary data, and the two scanned images are combined into one. Therefore, for the boundary data of the two scanned images, if there is an error in one of the boundary data, it will affect the merged boundary data; and some boundary data are stitched more than once, so in the process of stitching, errors will accumulate.

[0094] In the present application, the control points and target points are used as the selected areas of the weight rule, and the areas are spliced ​​preferentially, so as to ensure that the data of the areas corresponding to the control points and target points are not interfered by other data, thereby ensuring the original accuracy and benchmark effect of the area data;

[0095] For an area where obvious collapse data can be found in the InSAR collapse field image and the 3D local collapse field image, it indicates that the area may have a collapse phenomenon or may be error data, so the area is spliced ​​first, and the originality of the data in the area can be preserved first. The subsequent analysis focuses on determining whether the area is a collapse area or error data; the collapse data threshold can be set according to historical experience;

[0096] For some areas within the monitoring range, there may be a risk of collapse due to terrain and historical factors. It is necessary to focus on monitoring the data of the above areas to ensure the reliability and high accuracy of the data and accurately monitor the collapse-prone areas. The collapse risk attribute can be set according to the risk value predicted by experts for each terrain.

[0097] It is understandable that during the stitching process, the noise level and point cloud density of each 3D local collapse field image will be monitored. If the noise in a certain area is low and the data density is high, the above area will be used as the selected area of ​​the weight rule, which can ensure that high-quality data is stitched first and improve the accuracy and stability of the overall stitching; the data quality threshold can be set according to historical experience;

[0098] The weights are set for the selected areas mentioned above respectively, and the weight rules are formulated. It can be understood that the weight values ​​corresponding to each area are preset according to expert experience.

[0099] Specifically, Figure 3 As shown, the weight-marking and splicing of the 3D local collapse field images can be subdivided into the following steps:

[0100] S141, traversing each of the 3D local collapse field images, weight-marking the regions in each of the 3D local collapse field images according to the weight rule, and obtaining a plurality of 3D local collapse field images with weight marks;

[0101] S142, obtaining all weights on each of the 3D local collapse field images with weight marks, calculating all the obtained weights, and obtaining the total weight of each of the 3D local collapse field images;

[0102] S143, the plurality of 3D local collapse field images with weight marks are sequentially spliced ​​according to the corresponding total weights to obtain a 3D global collapse field image with weight marks.

[0103] Specifically, Figure 4 As shown, the weight marking of each region in the 3D local collapse field image according to the weight rule can be subdivided into the following cases:

[0104] Marking a region of the 3D local collapse field image that satisfies one of the weight rules with a corresponding weight;

[0105] For regions of the 3D local collapse field image that satisfy two or more of the weight rules, weight superposition marking is performed;

[0106] For other regions of the 3D local collapse field image that do not meet the weight rule, the weights of the other regions are all set to the initial weights.

[0107] Specifically, the initial weights are all smaller than the weights set in the weight rules.

[0108] It is understandable that the splicing mechanism is:

[0109] Geometric alignment: Using the control point and target point data, the randomly selected 3D local collapse field images are geometrically aligned to ensure seamless splicing between regions. Geometric alignment uses the least squares method for error optimization and combines the ICP (Iterative Closest Point) algorithm to further improve alignment accuracy.

[0110] Weight calculation: During the stitching process, the 3D local collapse field images are stitched one after another according to the weights to ensure the stability and accuracy of the final stitching result.

[0111] It is understandable that after marking all the 3D local collapse field images with weights, they are spliced, and the areas that meet the weight rules can be spliced ​​first, which can achieve the following effects:

[0112] By giving priority to areas that meet the weight rules, high-precision change information in these areas can be determined earlier, reducing the accumulation of local errors in the overall stitching; ensuring the accuracy of important areas and preventing important data from being covered by low-precision point clouds.

[0113] S150. Splicing the InSAR collapse field image and the weight-labeled 3D global collapse field image through the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitoring the InSAR seamless collapse field fusion image to obtain monitoring results of the monitoring range.

[0114] Specifically, Figure 5 As shown, the method for obtaining the InSAR seamless collapse field fusion image and performing monitoring can be subdivided into the following steps:

[0115] S151, respectively acquiring phase information and amplitude information of the InSAR collapse field image, and encoding the phase information and the amplitude information in different channels through an InSAR data encoder to obtain InSAR collapse field encoded data, wherein the InSAR encoder adopts a convolutional neural network structure;

[0116] It is understandable that the InSAR collapse field image data is a two-dimensional image, so the InSAR encoder adopts a convolutional neural network (CNN), and uses the multi-layer convolutional layers and pooling layers therein to extract the surface deformation features in the InSAR collapse field image data for better performance; and obtains the phase information and amplitude information of the InSAR collapse field image respectively, and converts them into different channel inputs to ensure that the twin neural network can capture the complete surface displacement information; preferably, a self-attention mechanism is added to make the model pay more attention to areas with significant terrain changes, thereby improving the ability to capture important features.

[0117] S152, encoding the 3D global collapse field image with the weight mark by a 3D point cloud data encoder to obtain 3D global collapse field encoded data, wherein the 3D point cloud data encoder adopts a point cloud convolutional neural network structure;

[0118] It is understandable that the data of the 3D global collapse field image is characterized by irregular three-dimensional points. The point cloud convolutional neural network structure Point-CNN is a neural network structure specifically for processing point cloud data, which can extract geometric features and local changes in the point cloud; preferably, in order to obtain terrain change information of different scales, a multi-layer receptive field design can be used to extract local features and global deformation features of the point cloud; preferably, since the point cloud data is relatively sparse, sparse convolution (Sparse Convolution) technology is used to reduce the amount of calculation and retain the effective information in the data.

[0119] S153, inputting the 3D global collapse field coded data and the InSAR collapse field coded data into the twin neural network, the twin neural network extracting information of the area with holes in the InSAR collapse field image from the InSAR collapse field coded data, and using the 3D global collapse field coded data and the information of the area with holes to splice and fill the area with holes, so as to obtain an InSAR seamless collapse field image;

[0120] It is understandable that in the previous step, the error area in the InSAR collapse field image is removed by a masking method to form a hole area. In this step, the corresponding hole area is filled with 3D global collapse field encoding data to obtain a seamless collapse field image.

[0121] It is understandable that this application uses the interpolation algorithm - inverse distance weighted method to fill the empty areas, and thins the point cloud as needed to retain key points. For the empty areas caused by missing data or inconsistent resolution during the splicing process, the inverse distance weighted method is used for interpolation and completion.

[0122] S154, the twin neural network extracts a weighted area from the 3D global collapse field coded data as a selected area, obtains a corresponding selected area in the InSAR seamless collapse field image according to the selected area, and uses the 3D global collapse field coded data to perform permutation fusion on the corresponding selected area in the InSAR seamless collapse field image to obtain an InSAR seamless collapse field fused image;

[0123] S155. Monitor the InSAR seamless collapse field fusion images at different times to obtain monitoring results of the monitoring range.

[0124] It is understandable that, using the twin neural network, the InSAR collapse field image and the 3D global collapse field image are fused through the twin neural network after being processed by their respective encoders, including:

[0125] Feature stitching: stitch the features of the InSAR collapse field image and the 3D global collapse field image to form a joint feature vector.

[0126] Shared weights: The twin neural network shares weights so that the data of the two channels share similar feature extraction methods in subsequent processing.

[0127] Similarity calculation: By calculating the similarity between feature vectors, the twin neural network can monitor the data changes of the InSAR seamless collapse field fusion images at different times and accurately capture the collapse area.

[0128] Based on the same inventive concept, the embodiment of the present application also provides a ground collapse monitoring system based on twin neural network fusion, such as Figure 6 As shown, the ground collapse monitoring system based on twin neural network fusion may include:

[0129] Range acquisition module 311, InSAR data acquisition-processing module 312, 3D scanning local data acquisition-processing module 313, 3D scanning global data acquisition module 314, splicing monitoring module 315. Among them:

[0130] A range acquisition module 311 is used to acquire a monitoring range of ground collapse, in which control points and target points are pre-arranged;

[0131] In this embodiment, the range acquisition module 311 can be used to perform Figure 1 As shown in step S110 , for a detailed description of the range acquisition module 311 , reference may be made to the description of step S110 .

[0132] An InSAR data acquisition-processing module 312 is used to acquire a real-time InSAR scanning image of the monitoring range, perform a first preprocessing on the InSAR scanning image, and acquire an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR;

[0133] In this embodiment, the InSAR data acquisition-processing module 312 can be used to perform Figure 1 As shown in step S120 , for a detailed description of the InSAR data acquisition-processing module 312 , reference may be made to the description of step S120 .

[0134] 3D scanning local data acquisition-processing module 313, used to acquire a plurality of 3D local scanning images of the monitoring range in real time, perform a second preprocessing on the plurality of 3D local scanning images, and process the plurality of 3D local scanning images that have completed the second preprocessing through a twin neural network to acquire a plurality of 3D local collapse field images, wherein the 3D local scanning images are acquired through 3D laser scanning;

[0135] In this embodiment, the 3D scanning local data acquisition-processing module 313 can be used to perform Figure 1 As shown in step S130 , for a detailed description of the 3D scanning local data acquisition-processing module 313 , reference may be made to the description of step S130 .

[0136] The 3D scanning global data acquisition module 314 is used to formulate a weight rule according to the control point and the target point, the InSAR collapse field image and the plurality of 3D local collapse field images, and to perform weight marking on the plurality of 3D local collapse field images according to the weight rule and then to splice them to obtain a 3D global collapse field image with weight marking;

[0137] In this embodiment, the 3D scanning global data acquisition module 314 can be used to perform Figure 1 As shown in step S140 , for a detailed description of the 3D scanning global data acquisition module 314 , reference may be made to the description of step S140 .

[0138] A splicing monitoring module 315 is used to splice the InSAR collapse field image and the 3D global collapse field image with weight markings through the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitor the InSAR seamless collapse field fusion image to obtain a monitoring result of the monitoring range;

[0139] In this embodiment, the splicing monitoring module 315 can be used to perform Figure 1As shown in step S150 , for a detailed description of the splicing monitoring module 315 , reference may be made to the description of step S150 .

[0140] Based on the same inventive concept, this embodiment also provides an electronic device, Figure 7 The structural diagram of the electronic device of this embodiment is shown, including a memory 21 and a processor 22, the memory 21 stores computer-readable instructions, and the processor 22 executes the computer-readable instructions to implement the ground collapse monitoring method based on twin neural network fusion of this embodiment.

[0141] Preferably, the electronic device further includes a bus 23 and a communication interface 24 , and the processor 22 , the communication interface 24 and the memory 21 are connected via the bus 23 .

[0142] The memory 21 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 24 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 23 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus 23 may be divided into an address bus, a data bus, a control bus, etc. (not fully drawn in the figure).

[0143] The processor 22 may be an integrated circuit chip with signal processing capability. In the specific implementation process, each step in the above method embodiment may be completed by the hardware integrated logic circuit or software instructions in the processor 22. The above processor 22 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, which may implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor 22 may also be any conventional processor 22, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 21, and the processor 22 reads the information in the memory 21 and completes the steps of the method of the above embodiment in combination with its hardware.

[0144] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor 22, the computer-executable instructions prompt the processor 22 to implement the above-mentioned ground collapse monitoring method based on twin neural network fusion. The specific implementation can be found in the embodiment and will not be repeated here.

[0145] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0146] Obviously, the above embodiments of the present application are merely examples for clearly illustrating the technical solution of the present application, and are not intended to limit the specific implementation methods of the present application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A ground subsidence monitoring method based on twin neural network fusion, characterized in that: Specifically include: Acquire a monitoring range of ground collapse, and pre-deploy control points and target points in the monitoring range; Acquire a real-time InSAR scanning image of the monitoring range, perform a first preprocessing on the InSAR scanning image, and acquire an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR; Acquire a plurality of real-time 3D local scanning images of the monitoring range, perform a second preprocessing on the plurality of 3D local scanning images, and process the plurality of 3D local scanning images that have completed the second preprocessing by means of a twin neural network to acquire a plurality of 3D local collapse field images, wherein the 3D local scanning images are acquired by means of 3D laser scanning; Formulate a weight rule according to the control point and the target point, the InSAR collapse field image and the plurality of 3D local collapse field images, and perform weight marking on the plurality of 3D local collapse field images according to the weight rule and then splice them to obtain a 3D global collapse field image with weight marking; The InSAR collapse field image and the weight-labeled 3D global collapse field image are spliced ​​by the twin neural network to obtain an InSAR seamless collapse field fusion image, and the InSAR seamless collapse field fusion image is monitored to obtain the monitoring result of the monitoring range.

2. A ground subsidence monitoring method based on twin neural network fusion according to claim 1, characterized in that: The InSAR collapse field image is a perforated InSAR collapse field image, wherein the perforated region of the perforated InSAR collapse field image is obtained by removing a number of error regions using a masking method, wherein the attribute data of the error region meets a preset error threshold.

3. The ground subsidence monitoring method based on twin neural network fusion according to claim 1 is characterized in that: The formulating of weight rules according to the control points and the target points, the InSAR collapse field images and the plurality of 3D local collapse field images specifically includes: Taking the control point and the target point as the first selected area; taking obvious collapse areas in the InSAR collapse field image and a plurality of the 3D local collapse field images as second selected areas, wherein the attribute data of the obvious collapse areas meet a preset collapse data threshold; The area prone to collapse is used as the third selected area, wherein the area prone to collapse meets the preset collapse risk attribute; taking a number of high-quality regions in the 3D local collapse field images as fourth selected regions, wherein attribute data of the high-quality regions meets a preset data quality threshold; Weights are set for the first selected area, the second selected area, the third selected area and the fourth selected area respectively, and four independent weight rules are formulated for the first selected area, the second selected area, the third selected area and the fourth selected area and their corresponding weights.

4. The ground subsidence monitoring method based on twin neural network fusion according to claim 1 is characterized in that: The step of weight-marking a plurality of the 3D local collapse field images according to the weight rule and then splicing them to obtain a 3D global collapse field image with weight marking specifically includes: Traversing each of the 3D local collapse field images, weight-marking the regions in each of the 3D local collapse field images according to the weight rule, and obtaining a plurality of 3D local collapse field images with weight marks; Obtaining all weights on each of the 3D local collapse field images with weight marks, calculating all the obtained weights, and obtaining the total weight of each of the 3D local collapse field images; The plurality of 3D local collapse field images with weight marks are spliced ​​in sequence according to the corresponding total weights to obtain a 3D global collapse field image with weight marks.

5. A ground subsidence monitoring method based on twin neural network fusion according to claim 4, characterized in that: The step of weighting each region in the 3D local collapse field image according to the weight rule specifically includes: Marking a region of the 3D local collapse field image that satisfies one of the weight rules with a corresponding weight; For regions of the 3D local collapse field image that satisfy two or more of the weight rules, weight superposition marking is performed; For other regions of the 3D local collapse field image that do not meet the weight rule, the weights of the other regions are all set to the initial weights.

6. A ground subsidence monitoring method based on twin neural network fusion according to claim 5, characterized in that: The initial weights are all smaller than the weights set in the weight rules.

7. The ground subsidence monitoring method based on twin neural network fusion according to claim 1 is characterized in that: The step of splicing the InSAR collapse field image and the weight-marked 3D global collapse field image by the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitoring the InSAR seamless collapse field fusion image to obtain the monitoring result of the monitoring range specifically includes: Respectively acquiring phase information and amplitude information of the InSAR collapse field image, and encoding the phase information and the amplitude information in different channels through an InSAR data encoder to obtain InSAR collapse field encoded data, wherein the InSAR encoder adopts a convolutional neural network structure; The 3D global collapse field image with the weight mark is encoded by a 3D point cloud data encoder to obtain 3D global collapse field encoded data, wherein the 3D point cloud data encoder adopts a point cloud convolutional neural network structure; The 3D global collapse field coded data and the InSAR collapse field coded data are input into the twin neural network, the twin neural network extracts information of the area with holes in the InSAR collapse field image from the InSAR collapse field coded data, and uses the 3D global collapse field coded data and the information of the area with holes to splice and fill the area with holes, so as to obtain an InSAR seamless collapse field image; The twin neural network extracts a weighted area from the 3D global collapse field coded data as a selected area, obtains a corresponding selected area in the InSAR seamless collapse field image according to the selected area, and uses the 3D global collapse field coded data to perform permutation fusion on the corresponding selected area in the InSAR seamless collapse field image to obtain an InSAR seamless collapse field fused image; The InSAR seamless collapse field fusion images at different times are monitored to obtain monitoring results of the monitoring range.

8. A ground subsidence monitoring system based on twin neural network fusion, characterized in that: include: A range acquisition module, used to acquire a monitoring range of ground collapse, in which control points and target points are pre-arranged; An InSAR data acquisition-processing module, used for acquiring a real-time InSAR scanning image of the monitoring range, performing a first preprocessing on the InSAR scanning image, and acquiring an InSAR collapse field image according to the InSAR scanning image after the first preprocessing, wherein the InSAR scanning image is acquired by an interferometric synthetic aperture radar InSAR; A 3D scanning local data acquisition-processing module, used to acquire a plurality of 3D local scanning images of the monitoring range in real time, perform a second preprocessing on the plurality of 3D local scanning images, and process the plurality of 3D local scanning images that have completed the second preprocessing through a twin neural network to acquire a plurality of 3D local collapse field images, wherein the 3D local scanning images are acquired through 3D laser scanning; A 3D scanning global data acquisition module is used to formulate a weight rule according to the control point and the target point, the InSAR collapse field image and a plurality of the 3D local collapse field images, and to perform weight marking on the plurality of the 3D local collapse field images according to the weight rule and then to splice them to obtain a 3D global collapse field image with weight marking; The splicing monitoring module is used to splice the InSAR collapse field image and the 3D global collapse field image with weight marking through the twin neural network to obtain an InSAR seamless collapse field fusion image, and monitor the InSAR seamless collapse field fusion image to obtain the monitoring result of the monitoring range.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement a ground collapse monitoring method based on twin neural network fusion as described in any one of claims 1-7.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, it implements a ground collapse monitoring method based on twin neural network fusion as described in any one of claims 1 to 7.

Images