Waste dump area change monitoring method and device, electronic equipment, storage medium and product

Abstract

The invention provides a waste dump area change monitoring method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: generating a plurality of sparse three-dimensional point cloud images corresponding to a waste dump through employing a motion structure recovery SFM technology based on two-dimensional images of a plurality of image control points preset in the waste dump; generating a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images; generating a dense three-dimensional point cloud image of the dumping site based on the plurality of initial local patches; and comparing the dense three-dimensional point cloud images of the dumping site at different moments, and determining a change area of the dumping site.

Description

Technical Field

[0001] The present disclosure relates to the field of mine surveying technology, and in particular to a method, device, electronic equipment, storage medium and product for monitoring regional changes in a spoil dump. Background Art

[0002] In the field of open-pit mining, the stability of the mine spoil dump is an important factor in ensuring safe production and construction.

[0003] In related technologies, the Structure From Motion (SFM) 3D reconstruction technology is usually used to generate sparse 3D point cloud images of the spoil dump. The relative position relationship, shape, and size of the objects in the sparse 3D point cloud images are basically correctly restored, and the structure and texture of the landforms are clear. However, due to algorithm defects, SFM mainly relies on feature points in sparse 3D point cloud images, which are often sparsely distributed, resulting in a relatively sparse 3D model of the reconstructed spoil dump. For scenes that require high resolution or complex details, SFM is difficult to directly meet the requirements.

[0004] In summary, due to the low density of the sparse 3D point cloud, it cannot reflect all the details of the photographed spoil dump. Therefore, it is necessary to process the sparse 3D point cloud of the spoil dump to obtain a more complete dense 3D point cloud image. Summary of the invention

[0005] The present invention provides a method, device, electronic equipment, storage medium and product for monitoring regional changes in a spoil dump to solve the problems in the related art.

[0006] A first aspect of the present disclosure provides a method for monitoring regional changes in a spoil dump, the method comprising:

[0007] Based on the two-dimensional images of multiple image control points preset in the spoil dump, a corresponding multiple sparse three-dimensional point cloud images of the spoil dump are generated by using the motion structure recovery SFM technology;

[0008] Based on the plurality of sparse three-dimensional point cloud images, generating a plurality of initial local patches;

[0009] Based on the multiple initial local patches, a dense three-dimensional point cloud image of the spoil dump is generated;

[0010] The dense three-dimensional point cloud images of the spoil dump at different times are compared to determine the changed area of ​​the spoil dump.

[0011] In one embodiment, the method of generating a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump using the motion structure recovery SFM technology based on the two-dimensional images of a plurality of image control points preset in the spoil dump includes:

[0012] According to the spatial positions of the image control points, setting flight parameters for the drone that collects the plurality of image control points of the spoil dump;

[0013] Generate a collection route for the drone based on the flight parameters;

[0014] Controlling the drone to collect two-dimensional images of a plurality of image control points along the collection route;

[0015] Using real-time kinematic RTK technology to obtain the true three-dimensional coordinates of the plurality of image control points;

[0016] Based on the two-dimensional images of the plurality of image control points and the corresponding real three-dimensional coordinates, a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump are generated using the motion structure recovery SFM technology.

[0017] In one embodiment, generating a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images comprises:

[0018] Extracting feature points of each sparse three-dimensional point cloud image from each sparse three-dimensional point cloud image using Harris and DoG operators;

[0019] Matching feature points of different sparse three-dimensional point cloud images by a feature matching algorithm to determine matching point pairs;

[0020] Based on the matching point pairs, a plurality of initial local patches are generated by a triangulation method.

[0021] In one embodiment, generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches includes:

[0022] Extracting feature information of the initial local patch using a convolutional neural network Unet;

[0023] Taking the feature information as input of a classic multi-view matching algorithm PMVS, and using the PMVS to generate a feature map corresponding to the initial local patch;

[0024] Based on the feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

[0025] In one embodiment, the generating of a dense three-dimensional point cloud image of the spoil dump using the PMVS based on the feature map includes:

[0026] Using the PMVS to perform disparity consistency detection on the feature maps under multiple viewing angles, and deleting feature maps that do not conform to disparity consistency from the multiple feature maps to obtain remaining feature maps;

[0027] Based on the remaining feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

[0028] In one embodiment, after generating the dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches, the spoil dump area change monitoring method proposed in the first aspect of the present disclosure further includes:

[0029] The dense three-dimensional point cloud image of the spoil dump is used to generate a digital orthophoto DOM and a digital elevation model DEM of the spoil dump.

[0030] The second aspect of the present disclosure provides a device for monitoring regional changes in a dump site, the device comprising:

[0031] A sparse three-dimensional point cloud image generating unit, used for generating a plurality of corresponding sparse three-dimensional point cloud images of the spoil dump using a motion structure recovery SFM technology based on two-dimensional images of a plurality of image control points preset in the spoil dump;

[0032] An initial local patch generating unit, configured to generate a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images;

[0033] A dense three-dimensional point cloud image generating unit, used for generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches;

[0034] The determination unit is used to compare the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0035] A third aspect of the present disclosure provides an electronic device, including:

[0036] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the embodiment of the first aspect of the present disclosure.

[0037] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0038] A fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect embodiment of the present disclosure when executed by a processor.

[0039] In summary, the present disclosure proposes a method, device, electronic device, storage medium and product for monitoring changes in the spoil dump area. The method includes: based on a two-dimensional image of multiple image control points preset in the spoil dump, using the motion structure recovery SFM technology to generate a corresponding multiple sparse three-dimensional point cloud images of the spoil dump; based on the multiple sparse three-dimensional point cloud images, generating a multiple initial local patches; based on the multiple initial local patches, generating a dense three-dimensional point cloud image of the spoil dump; comparing the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0040] The solution provided by the present disclosure can generate a dense three-dimensional point cloud image based on a sparse three-dimensional point cloud image of a spoil dump, and determine the changed area of ​​the spoil dump by comparing the dense three-dimensional point cloud images of the spoil dump at different times.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0043] Figure 1 A schematic diagram of a flow chart of a method for monitoring changes in a spoil dump area provided in an embodiment of the present disclosure;

[0044] Figure 2 A schematic diagram of a process for generating a sparse three-dimensional point cloud image provided by an embodiment of the present disclosure;

[0045] Figure 3 A schematic diagram of a flow chart of a method for generating an initial local patch provided in an embodiment of the present disclosure;

[0046] Figure 4 A schematic diagram of a process for generating a dense three-dimensional point cloud image provided by an embodiment of the present disclosure;

[0047] Figure 5 A schematic diagram of the structure of a device for monitoring changes in a spoil dump area provided in an embodiment of the present disclosure;

[0048] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0050] The present disclosure proposes a method, device, electronic device, storage medium and product for monitoring changes in a spoil dump area. The method includes: based on a two-dimensional image of a plurality of image control points preset in the spoil dump, using the motion structure recovery (SFM) technology to generate a corresponding plurality of sparse three-dimensional point cloud images of the spoil dump; based on the plurality of sparse three-dimensional point cloud images, generating a plurality of initial local patches; based on the plurality of initial local patches, generating a dense three-dimensional point cloud image of the spoil dump; and comparing the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0051] The solution provided by the present disclosure can generate a dense three-dimensional point cloud image based on a sparse three-dimensional point cloud image of a spoil dump, and determine the changed area of ​​the spoil dump by comparing the dense three-dimensional point cloud images of the spoil dump at different times.

[0052] The present disclosure is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] like Figure 1 As shown, Figure 1 The flowchart of the method for monitoring the change of the spoil dump area provided by the embodiment of the present disclosure is as follows. The method for monitoring the change of the spoil dump area provided by the embodiment of the present disclosure comprises the following steps:

[0054] Step 101, based on the two-dimensional images of multiple image control points preset in the spoil dump, generate multiple corresponding sparse three-dimensional point cloud images of the spoil dump using the motion structure recovery SFM technology;

[0055] In one embodiment, the spoil dump refers to a spoil dump in a mining area.

[0056] In one embodiment, if Figure 2 As shown, step 101 includes:

[0057] Step 201, setting flight parameters for a drone that collects a plurality of the image control points of the spoil dump according to the spatial positions of the image control points;

[0058] In one embodiment, the image control points are arranged at obvious feature points, feature corner points, intersection points of nearly orthogonal linear features, or fixed point-shaped features in the spoil dump.

[0059] In one embodiment, the image control points should be evenly distributed in the spoil dump to ensure that the complete topography and landform of the spoil dump can be acquired.

[0060] In one embodiment, the image control points of the strip area in the spoil dump may be arranged along an S-shaped or Z-shaped route.

[0061] In one embodiment, the flight parameters include the coordinates of key points on the route, flight altitude, speed, return point, etc. It even includes safety parameters such as the uncontrolled behavior of the drone and the emergency shutdown method to ensure the safety during the flight.

[0062] Step 202, generating a collection route of the UAV based on the flight parameters;

[0063] In one embodiment, the method of generating a collection route of a drone based on preset flight parameters belongs to existing mature technology and will not be described in detail here.

[0064] Step 203, controlling the drone to collect two-dimensional images of a plurality of image control points along the collection route;

[0065] In one embodiment, when the drone moves above an image control point, it is necessary to capture multiple two-dimensional images from different angles for each image control point.

[0066] Step 204, using real-time kinematic RTK technology to obtain the real three-dimensional coordinates of the plurality of image control points;

[0067] In one embodiment, real-time kinematic (RTK) technology is a measurement method that can obtain centimeter-level positioning accuracy in real time in the field. It consists of three parts: a base station receiver, a data link, and a mobile station receiver. The base station continuously observes the satellite and sends its observation data and station information to the mobile station in real time through a radio transmission device. While the mobile station receives the GPS satellite signal, it also receives the data transmitted by the base station, and then calculates the three-dimensional coordinates and accuracy of the mobile station in real time based on the principle of relative positioning.

[0068] Step 205: Based on the two-dimensional images of the plurality of image control points and the corresponding real three-dimensional coordinates, a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump are generated using a motion structure recovery (SFM) technology.

[0069] In one embodiment, using the motion structure recovery SFM technology to generate multiple sparse three-dimensional point cloud images corresponding to the spoil yard includes: using a computer vision algorithm (such as SIFT, SURF or ORB, etc.) to extract feature points from the two-dimensional image of each image control point, and matching the feature points in the two-dimensional images of different image control points through a feature matching algorithm (such as FLANN or BFMatcher, etc.) to establish a corresponding relationship between them.

[0070] Then, the real three-dimensional coordinates of each image control point and the above-mentioned matched feature points are used to solve the posture of the camera mounted on the drone for taking two-dimensional images of the image control points through the PnP (Perspective-n-Point) algorithm, specifically, the rotation matrix R and translation vector T of the camera.

[0071] Finally, the camera pose and the matched feature points are used to calculate the coordinates of the 3D points that constitute the sparse 3D point cloud image using a triangulation algorithm.

[0072] Step 102, generating a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images;

[0073] In one embodiment, if Figure 3 As shown, step 102 includes:

[0074] Step 301, extracting feature points of each sparse three-dimensional point cloud image from each sparse three-dimensional point cloud image using Harris and DoG operators;

[0075] In one embodiment, when Harris and DoG operators are used to extract feature points from a sparse 3D point cloud image, the sparse 3D point cloud image needs to be converted into a grayscale image, such as a height map or a density map.

[0076] In one embodiment, feature points extracted from a sparse three-dimensional point cloud image using the Harris operator are Harris corner points, and feature points extracted from a sparse three-dimensional point cloud image using the DoG operator are local extreme points in the sparse three-dimensional point cloud image, such as maximum points or minimum points.

[0077] Step 302, matching the feature points of different sparse three-dimensional point cloud images by a feature matching algorithm to determine matching point pairs;

[0078] In one embodiment, different feature points may be matched using similarity metrics between different feature points, for example, feature points whose Euclidean distance is less than a preset distance threshold are used as matching point pairs.

[0079] In one embodiment, after the matching point pairs are determined, the abnormal point pairs in the matching point pairs may be screened out by using a nearest neighbor distance ratio test (Lowe's ratio test, Lrt) algorithm.

[0080] Step 303: Generate multiple initial local patches by triangulation method based on the matching point pairs.

[0081] In one embodiment, the method of generating a local patch by triangulation is a well-known existing technology and will not be described in detail here.

[0082] In one embodiment, the initial local patch includes the three-dimensional position information, normal vector information and color information of the local patch. The initial local patch can be expressed by a mathematical expression as follows:

[0083] P={(X,Y,Z),(n x ,n y ,n z ),color};

[0084] Among them, (X, Y, Z) is the three-dimensional position information of the initial local patch, (n x ,n y ,n z ) is the normal vector information of the initial local patch, and color is the color information of the initial local patch.

[0085] Step 103, generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches;

[0086] In one embodiment, if Figure 4 As shown, step 103 includes:

[0087] Step 401, extracting feature information of the initial local patch using a convolutional neural network Unet;

[0088] In one embodiment, the initial local patch information in step 303 may be used as input to a convolutional neural network Unet to extract feature information of the initial local patch.

[0089] Step 402, using the feature information as input of a classic multi-view matching algorithm PMVS, and using the PMVS to generate a feature map corresponding to the initial local patch;

[0090] In one embodiment, the feature information is input into the PMVS as a partial weight of the PMVS, and it is iterated and expanded to obtain a feature map output by the PMVS; the feature map can be expressed by a mathematical expression as follows:

[0091]

[0092] Among them, X[i,j] is the pixel value corresponding to the input feature information, W[m,n] is the weight of the convolution kernel (filter) of PMVS, and Y[i,j] is the feature map output by PMVS.

[0093] Step 403: Based on the feature map, use the PMVS to generate a dense three-dimensional point cloud image of the spoil dump.

[0094] In one embodiment, step 403 includes:

[0095] Using the PMVS to perform disparity consistency detection on the feature maps under multiple viewing angles, and deleting feature maps that do not conform to disparity consistency from the multiple feature maps to obtain remaining feature maps;

[0096] Based on the remaining feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

[0097] In one embodiment, the parallax consistency condition is expressed by a mathematical expression:

[0098] Σ(Ii(u,v)-Ij(u,v)) 2 <∈;

[0099] Among them, Ii(u,v) and Ij(u,v) are the brightness values ​​of the corresponding pixels (u,v) in different feature maps, and ∈ is the disparity consistency threshold.

[0100] Step 104 , comparing the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0101] In one embodiment, after generating the dense three-dimensional point cloud image of the spoil dump, the dense three-dimensional point cloud image of the spoil dump can be used to generate a digital orthophoto DOM and a digital elevation model DEM of the spoil dump.

[0102] In summary, the present invention proposes a method for monitoring changes in the spoil dump area, including: based on the two-dimensional images of multiple image control points preset in the spoil dump, using the motion structure recovery SFM technology to generate a corresponding multiple sparse three-dimensional point cloud images of the spoil dump; based on the multiple sparse three-dimensional point cloud images, generating a multiple initial local patches; based on the multiple initial local patches, generating a dense three-dimensional point cloud image of the spoil dump; comparing the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0103] The solution provided by the present disclosure can generate a dense three-dimensional point cloud image based on a sparse three-dimensional point cloud image of a spoil dump, and determine the changed area of ​​the spoil dump by comparing the dense three-dimensional point cloud images of the spoil dump at different times.

[0104] In order to implement the method for monitoring changes in the spoil dump area provided by the embodiment of the present disclosure, the embodiment of the present disclosure also provides a device for monitoring changes in the spoil dump area, such as Figure 5 shown. Figure 5 This is a schematic diagram of the structure of a spoil dump area change monitoring device provided in an embodiment of the present disclosure. The spoil dump area change monitoring device 500 includes:

[0105] A sparse 3D point cloud image generating unit 501 is used to generate a plurality of sparse 3D point cloud images corresponding to the spoil dump using a motion structure recovery SFM technology based on a plurality of 2D images of image control points preset in the spoil dump;

[0106] An initial local patch generating unit 502, configured to generate a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images;

[0107] A dense three-dimensional point cloud image generating unit 503 is used to generate a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches;

[0108] The determination unit 504 is used to compare the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

[0109] In one embodiment, the sparse 3D point cloud image generating unit 501 is specifically used to:

[0110] According to the spatial positions of the image control points, setting flight parameters for the drone that collects the plurality of image control points of the spoil dump;

[0111] Generate a collection route for the drone based on the flight parameters;

[0112] Controlling the drone to collect two-dimensional images of a plurality of image control points along the collection route;

[0113] Using real-time kinematic RTK technology to obtain the true three-dimensional coordinates of the plurality of image control points;

[0114] Based on the two-dimensional images of the plurality of image control points and the corresponding real three-dimensional coordinates, a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump are generated using the motion structure recovery SFM technology.

[0115] In one embodiment, the initial local patch generating unit 502 is specifically configured to:

[0116] Extracting feature points of each sparse three-dimensional point cloud image from each sparse three-dimensional point cloud image using Harris and DoG operators;

[0117] Matching feature points of different sparse three-dimensional point cloud images by a feature matching algorithm to determine matching point pairs;

[0118] Based on the matching point pairs, a plurality of initial local patches are generated by a triangulation method.

[0119] In one embodiment, the dense 3D point cloud image generating unit 503 is specifically used to:

[0120] Extracting feature information of the initial local patch using a convolutional neural network Unet;

[0121] Taking the feature information as input of a classic multi-view matching algorithm PMVS, and using the PMVS to generate a feature map corresponding to the initial local patch;

[0122] Based on the feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

[0123] In one embodiment, the dense 3D point cloud image generating unit 503 is further specifically used for:

[0124] Using the PMVS to perform disparity consistency detection on the feature maps under multiple viewing angles, and deleting feature maps that do not conform to disparity consistency from the multiple feature maps to obtain remaining feature maps;

[0125] Based on the remaining feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

[0126] In one embodiment, the spoil dump area change monitoring device 500 further includes a DOM and DEM generation unit, and the DOM and DEM generation unit is used to:

[0127] The dense three-dimensional point cloud image of the spoil dump is used to generate a digital orthophoto DOM and a digital elevation model DEM of the spoil dump.

[0128] It should be noted that: the spoil dump area change monitoring device provided in the above embodiment only uses the division of the above program modules as an example when monitoring the spoil dump area changes. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the spoil dump area change monitoring device is divided into different program modules to complete all or part of the processing described above. In addition, the spoil dump area change monitoring device provided in the above embodiment and the spoil dump area change monitoring method embodiment provided in the embodiment of the present disclosure belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0129] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the electronic device 600 includes at least one processor 602; and a memory 601 that is communicatively connected to the at least one processor 602; wherein the memory 601 stores instructions that can be executed by the at least one processor 602, and the instructions are executed by the at least one processor 602 to implement the steps of the DCS-based thermal equipment inspection method described in the embodiment of the present disclosure.

[0130] Optionally, the electronic device may specifically be a DCS-based thermal equipment inspection device of an embodiment of the present application, and the electronic device may implement the corresponding processes implemented by the DCS-based thermal equipment inspection device in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0131] It is understood that the electronic device also includes a communication interface 603. The various components in the electronic device are coupled together through a bus system 604. It is understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604 .

[0132] It can be understood that the memory 601 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 601 described in the embodiment of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0133] The method disclosed in the above embodiment of the present disclosure can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 602 or the instruction in the form of software. The above processor 602 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 601, and the processor 602 reads the information in the memory 601 and completes the steps of the above method in combination with its hardware.

[0134] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors, or other electronic components to execute the aforementioned method.

[0135] The embodiment of the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to implement the steps of the DCS-based thermal equipment inspection method described in the embodiment of the present disclosure when executed.

[0136] The embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the DCS-based thermal equipment inspection method described in the embodiment of the present disclosure.

[0137] Optionally, the computer-readable storage medium can be applied to the DCS-based thermal equipment inspection device in the embodiments of the present application, and the computer instructions enable the computer to execute the corresponding processes implemented by the DCS-based thermal equipment inspection device in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0139] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0140] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0141] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0142] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0143] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for monitoring regional changes in a spoil dump, characterized in that: include: Based on the two-dimensional images of multiple image control points preset in the spoil dump, a corresponding multiple sparse three-dimensional point cloud images of the spoil dump are generated by using the motion structure recovery SFM technology; Based on the plurality of sparse three-dimensional point cloud images, generating a plurality of initial local patches; Based on the multiple initial local patches, a dense three-dimensional point cloud image of the spoil dump is generated; The dense three-dimensional point cloud images of the spoil dump at different times are compared to determine the changed area of ​​the spoil dump.

2. The method according to claim 1, characterized in that The method of generating a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump using the motion structure recovery SFM technology based on the two-dimensional images of a plurality of image control points preset in the spoil dump includes: According to the spatial positions of the image control points, setting flight parameters for the drone that collects the plurality of image control points of the spoil dump; Generate a collection route for the drone based on the flight parameters; Controlling the drone to collect two-dimensional images of a plurality of image control points along the collection route; Using real-time kinematic RTK technology to obtain the true three-dimensional coordinates of the plurality of image control points; Based on the two-dimensional images of the plurality of image control points and the corresponding real three-dimensional coordinates, a plurality of sparse three-dimensional point cloud images corresponding to the spoil dump are generated using the motion structure recovery SFM technology.

3. The method according to claim 1, characterized in that: The step of generating a plurality of initial local patches based on the plurality of sparse three-dimensional point cloud images comprises: Extracting feature points of each sparse three-dimensional point cloud image from each sparse three-dimensional point cloud image using Harris and DoG operators; Matching feature points of different sparse three-dimensional point cloud images by a feature matching algorithm to determine matching point pairs; Based on the matching point pairs, a plurality of initial local patches are generated by a triangulation method.

4. The method according to claim 1, characterized in that: The step of generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches comprises: Extracting feature information of the initial local patch using a convolutional neural network Unet; Taking the feature information as input of a classic multi-view matching algorithm PMVS, and using the PMVS to generate a feature map corresponding to the initial local patch; Based on the feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

5. The method according to claim 4, characterized in that The method of generating a dense three-dimensional point cloud image of the spoil dump using the PMVS based on the feature map includes: Using the PMVS to perform disparity consistency detection on the feature maps under multiple viewing angles, and deleting feature maps that do not conform to disparity consistency from the multiple feature maps to obtain remaining feature maps; Based on the remaining feature map, the PMVS is used to generate a dense three-dimensional point cloud image of the spoil dump.

6. The method according to claim 1, characterized in that After generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches, the method further comprises: The dense three-dimensional point cloud image of the spoil dump is used to generate a digital orthophoto DOM and a digital elevation model DEM of the spoil dump.

7. A device for monitoring changes in spoil dump areas, characterized in that: include: A sparse three-dimensional point cloud image generating unit, used to generate a plurality of corresponding sparse three-dimensional point cloud images of the spoil dump using a motion structure recovery SFM technology based on two-dimensional images of a plurality of image control points preset in the spoil dump; An initial local patch generating unit, used for generating a plurality of initial local patches based on a plurality of the sparse three-dimensional point cloud images; A dense three-dimensional point cloud image generating unit, used for generating a dense three-dimensional point cloud image of the spoil dump based on the multiple initial local patches; The determination unit is used to compare the dense three-dimensional point cloud images of the spoil dump at different times to determine the changed area of ​​the spoil dump.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images