An open-pit mine production plan visualization system based on orthophoto image and DXF vector overlay

Abstract

The application discloses a strip mine production plan visualization system based on orthographic image and DXF vector superposition, belongs to the three-dimensional model technical field in image data processing, and particularly relates to a geographic information system. The system specifically comprises a data processing module, a coordinate fusion module, a dynamic updating module and a visualization module. The data processing module pre-processes original orthographic images and DXF vector data. The coordinate fusion module performs precision superposition to generate a data fusion model of a unified coordinate system. The dynamic updating module extracts real-time production data to update the data fusion model and simulates production plan evolution. The visualization module renders and visualizes the data fusion model and dynamic data in a three-dimensional scene. The application has the technical advantage of production plan visualization over a traditional system, solves the problem of insufficient production plan visualization degree of the traditional system, and achieves a highly visualized technical effect.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional modeling technology in image data processing, and more particularly to geographic information systems. Background Technology

[0002] In the early stages of open-pit mining, visualization primarily relied on two-dimensional drawings. With the rise of large-scale open-pit mining, production planning at this stage depended on CAD-drawn two-dimensional topographic maps and engineering drawings, using static elements such as mining boundaries, blasting zone coordinates, and transportation routes to guide production. Currently, open-pit mining production planning and on-site execution still heavily depend on a direct understanding and precise location of information such as surface morphology, boundaries, transportation channels, and blasting areas. Orthophotos provide high-resolution surface textures, terrain undulations, and temporal information; DXF vector data carries key geometric elements such as design boundaries, channels, engineering implementation areas, and equipment locations. In recent years, the field has entered an exploration phase of "multi-source data fusion visualization." With the popularization of the smart mine concept, the high-fidelity topographic presentation of orthophotos and the engineering planning annotations of DXF vectors complement each other. The industry has begun to experiment with the superposition and integration of these two methods, which will greatly improve the accuracy and efficiency of design review, resource allocation, transportation planning, and on-site command.

[0003] However, with current technology, traditional open-pit mine visualization systems mostly use single two-dimensional drawings or static three-dimensional models, lacking real-time data fusion capabilities. This leads to insufficient adaptability to complex terrain and insufficient visualization and monitoring capabilities for abnormal situations during production. Summary of the Invention

[0004] The purpose of this invention is to disclose a visualization system for open-pit mine production planning based on the overlay of orthophotos and DXF vector data. The system achieves accurate overlay and real-time rendering of orthophotos and DXF vector data through multi-source data fusion, adaptive coordinate matching mechanism and dynamic projection transformation technology. The system is composed of multi-modal modules connected in sequence. The output parameters of each module drive the operation of subsequent modules, thereby solving the problems of traditional systems lacking real-time data fusion capabilities, resulting in insufficient adaptability to complex terrain and insufficient visualization and monitoring capabilities for abnormal situations during production. The system achieves technical effects of adapting to complex terrain and high visualization.

[0005] The specific plan is as follows:

[0006] This invention discloses a visualization system for open-pit mine production planning based on the overlay of orthophotos and DXF vectors. The system specifically includes: a data processing module, a coordinate fusion module, a dynamic update module, and a visualization module; the above modules are connected in sequence.

[0007] The data processing module preprocesses and extracts features from the original orthophoto and DXF vector data, and outputs a feature matching matrix.

[0008] The coordinate fusion module performs precision superposition of data based on the feature matching matrix to generate a data fusion model with coordinate system one, and the model outputs a set of coordinate transformation parameters.

[0009] The dynamic update module, based on the coordinate transformation parameter set, introduces real-time production data to update the data fusion model, simulates the evolution of the production plan, and outputs the production plan layer and early warning signal dataset.

[0010] The visualization module renders and visualizes the production plan layer and early warning signal dataset in a 3D scene.

[0011] Furthermore, the data processing module processes data including orthophoto data, DXF vector files, and mine geographic information data acquired based on airborne lidar; using geometric correction and DXF vector topology restoration, based on the mine geographic information data, it processes the orthophoto data and DXF vector file data to make them accurately aligned on the same reference plane and form a feature point matching matrix.

[0012] Furthermore, the coordinate fusion module includes a feature point matching and verification submodule, a coordinate parameter transformation submodule, and a pixel vector binding submodule;

[0013] The feature point matching verification submodule reads the feature point matching matrix and checks the number and quality of matching pairs to verify the consistency between the georeferenced information of the orthophoto and the DXF vector data.

[0014] The coordinate parameter transformation submodule uses thin-plate spline functions to calculate the nonlinear transformation from image pixel coordinates to vector world coordinates, solves for the TPS transformation parameters, and outputs them.

[0015] The pixel vector binding submodule converts the coordinates of each pixel in the orthophoto into world coordinates through TPS transformation, assigns a unique ID to each vector feature, establishes the association between the vector feature and the image pixel, determines the image pixel area it covers, and further establishes a pixel-vector binding table to record each vector feature ID and its corresponding image pixel area.

[0016] Furthermore, the coordinate fusion module also includes a spatial index construction submodule and a model generation submodule;

[0017] The spatial index construction submodule includes a vector feature ID allocation unit, a spatial association establishment unit, and an association strategy unit;

[0018] The vector feature ID allocation unit assigns a unique identifier to each DXF vector feature and establishes a feature attribute index table.

[0019] For each vector feature, the spatial association establishment unit calculates its coverage area on the image and further uses a scan line filling algorithm to determine the pixel area corresponding to the vector polygon.

[0020] The association strategy unit constructs a pyramid-structured association index, which includes L0 for precise query, L1 for fast display, and L2 for overview, with resolutions of original resolution → 1 / 4 resolution → 1 / 16 resolution in sequence.

[0021] The model generation submodule creates a raster-vector fusion data structure and further performs boundary smoothing to eliminate jagged edges. The smoothing formula is: C 混 =α·C 矢原 +(1-α)·C 正原 C 混 Indicates the final displayed mixed color; C 矢原 Represents the original color of the vector element; C 正原 The original pixel color of the orthophoto is represented by α; the mixing factor is represented by α; a data fusion model is established by constructing sub-modules and model generation sub-modules based on the above spatial index.

[0022] Furthermore, the coordinate fusion module also includes a quality assessment submodule;

[0023] The quality assessment submodule evaluates the fusion effect based on the data fusion model, the original feature point matching matrix and the TPS transformation result. The quality assessment submodule includes a consistency check unit, an accuracy verification unit and a visual effect evaluation unit.

[0024] The consistency check unit randomly selects a set of verification points and calculates the reprojection error.

[0025] The accuracy verification unit uses an independent control point set to verify the fusion accuracy and calculates the position accuracy index, which includes planar accuracy and elevation accuracy.

[0026] The visual effect evaluation unit uses the Structural Similarity Index (SSIM) to evaluate the superposition effect. The SSIM formula is as follows:

[0027]

[0028] Where SSIM(x, y) represents the similarity index of the two images to be compared; (2μ x μ y +c1)(2σ xy+c2) represents the molecular part, namely the combined factor of brightness, contrast, and structural similarity; c1 and c2 represent constants; μ x μ represents the average pixel value of image patch x. y This represents the average pixel value of image patch y; This indicates that the molecules are being normalized. Represents the variance of image patch x; σ represents the variance of the image y; xy It represents the covariance between image patches x and y; it integrates geometric accuracy and visual quality through multi-dimensional quality assessment.

[0029] Furthermore, the dynamic update module includes a data stream submodule, a change detection and difference localization submodule, and a data interpolation extrapolation submodule;

[0030] The data stream submodule includes a data acquisition unit, a parsing unit, and an associated output unit;

[0031] The data acquisition unit obtains GPS coordinates via the MQTT protocol, periodically acquires updated production data from the database, periodically acquires radar point cloud data via FTP, and obtains a data fusion model through the aforementioned coordinate fusion module.

[0032] The analysis unit converts GPS coordinates into a unified coordinate system used by the data fusion model, filters and thins the geological radar point cloud, and extracts information on changes in the mining face.

[0033] The associated output unit spatially associates the device location with the vector elements in the data fusion model to determine the road segment where the device is located and outputs the parsed standardized data and the associated results.

[0034] The change detection and differential location submodule performs multi-dimensional processing based on the standardized data and correlation results output above, including detection of changes in the mining face, detection of changes in equipment location, and analysis of changes in production output. The processing result is a list of change areas.

[0035] The data interpolation and extrapolation submodule performs interpolation and extrapolation based on the list of changed areas and historical production data. The process includes Kriging interpolation, temporal extrapolation and spatial propagation simulation. The result is the production status of the entire area after interpolation and extrapolation.

[0036] Furthermore, the dynamic update module also includes a layer generation and update submodule and an early warning signal submodule;

[0037] The layer generation and update submodule constructs a dynamic layer based on the interpolated extrapolated production status of the entire area and uses an incremental update mechanism to update the model blocks corresponding to the changed areas.

[0038] The warning signal submodule is based on an updated model and sets up a multi-level warning mechanism, which is divided into different levels according to the severity and adopts different notification methods, including information notification and audio notification.

[0039] Furthermore, the visualization module includes a data loading submodule and a scene graph construction and rendering submodule;

[0040] The data loading submodule prepares data based on dynamic layers, updated models, and early warning data. Its processing includes data conversion and analysis, audio signal processing, and model optimization. The processing results are a 3D scene map, a set of LOD models, and a spatial audio object library.

[0041] The scene graph construction and rendering submodule is based on the aforementioned 3D scene graph, LOD model set, and spatial audio object library to construct and render the scene. Its processing includes scene graph construction → LOD rendering → spatial audio rendering → dynamic data rendering. The processing result is the rendered 3D scene and real-time audio data.

[0042] Furthermore, the visualization module also includes a 3D output submodule;

[0043] The 3D output submodule, based on the rendered 3D scene and real-time audio data, uses a multi-view output mode to materialize the rendered 3D scene data and present it on the system page.

[0044] Compared with the prior art, the present invention has at least one of the following technical effects:

[0045] 1. This invention implements modular partitioned storage processing for all system data, which to some extent solves the problem of traditional systems lacking real-time data integration capabilities, resulting in fragmented and isolated system data.

[0046] 2. The data processing module of this invention utilizes multi-source data preprocessing and feature extraction technology to extract feature points from the original data to form a feature matching matrix. Compared with traditional systems, it has the technical advantage of multimodal data detail feature extraction and solves the problem of traditional systems lacking data detail feature extraction.

[0047] 3. The coordinate fusion module of this invention uses thin plate spline function for nonlinear transformation, which can adapt to local deformation caused by complex terrain and achieve accuracy superposition. Compared with traditional systems, it has the technical advantage of multi-layer data accuracy superposition and solves the problem of insufficient adaptability of traditional system data to complex terrain.

[0048] 4. The dynamic update module of this invention uses multiple data streams and algorithm technology to simulate and optimize the production plan in real time, and uses an incremental update mechanism to update the changed area in real time, which greatly improves the system response speed and processing efficiency. Compared with the traditional system, it has the technical advantage of real-time simulation and optimization of the production plan, and solves the problem that the traditional system cannot reflect the changes in the production site in real time, resulting in the production plan being out of sync with reality.

[0049] 5. The visualization module of this invention utilizes LOD multi-resolution rendering and instantiation technology to significantly improve rendering efficiency and output WebGL standard format. It can visualize and render production plans in a browser. Compared with traditional systems, it has a high technical advantage in visualization rendering and solves the problem of insufficient visualization and monitoring capabilities of traditional systems for abnormal situations during production. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0051] Figure 1 This is an architecture diagram of an open-pit mine production planning visualization system based on the overlay of orthophotos and DXF vectors, according to the present invention.

[0052] Figure 2 This is a diagram of the data processing module architecture of the present invention;

[0053] Figure 3 This is a diagram of the coordinate fusion module architecture of the present invention;

[0054] Figure 4 This is a diagram of the dynamic update module architecture of the present invention;

[0055] Figure 5 This is a visual module architecture diagram of the present invention. Detailed Implementation

[0056] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0057] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0058] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0059] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0060] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0061] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0062] The specific implementation method is as follows:

[0063] See appendix Figure 1 A visualization system for production planning in open-pit mines based on the overlay of orthophotos and DXF vectors, the system specifically includes: a data processing module, a coordinate fusion module, a dynamic update module, and a visualization module; the above modules are connected in sequence.

[0064] The data processing module preprocesses and extracts features from the original orthophoto and DXF vector data, and outputs a feature matching matrix.

[0065] The coordinate fusion module performs precision superposition of data based on the feature matching matrix to generate a data fusion model with coordinate system one, and the model outputs a set of coordinate transformation parameters.

[0066] The dynamic update module, based on the coordinate transformation parameter set, introduces real-time production data to update the data fusion model, simulates the evolution of the production plan, and outputs the production plan layer and early warning signal dataset.

[0067] The visualization module renders and visualizes the production plan layer and early warning signal dataset in a 3D scene.

[0068] The aforementioned open-pit mine production planning visualization system, based on orthophotos and DXF vector overlay, employs a modular design, subdividing each function into functional points from modules to sub-modules. Each module has a specific function, and they are interconnected functional modules that form a complete system through their module network, capable of visually presenting the open-pit mine production plan. This system incorporates deep learning algorithms and combines various data processing technologies, and its visualization level has been deeply optimized to meet the requirements for three-dimensional visualization of the production plan during the open-pit mine production process.

[0069] For specific implementation details, please refer to the appendix. Figure 2 The data processing module processes data including orthophoto data, DXF vector files, and mine geographic information data acquired by airborne lidar; using geometric correction and DXF vector topology repair, based on the mine geographic information data, it processes the orthophoto data and DXF vector file data to make them accurately aligned on the same reference plane and form a feature point matching matrix.

[0070] The above-mentioned process, which involves processing the orthophoto using geometric correction based on mine geographic information data and processing the DXF vector file data using DXF vector topology restoration, includes the following steps:

[0071] Step 1) Input the orthophoto, raw DXF vector file data, mine geographic information data (DEM), and data source metadata (coordinate system, acquisition time, and resolution) into the module, and unify the data format and extract specific metadata;

[0072] Step 2) Based on the data from Step 1, the image distortion caused by the image acquisition camera is corrected using the Brown-Lacondy model; the projection error caused by the terrain is corrected using orthorectification based on DEM, and further radiometric normalization is performed using Wallis filtering; finally, the geometrically corrected orthorectified image is obtained.

[0073] Step 3) Based on the results of Step 2, feature detection is performed using an improved SIFT. The improvement lies in introducing a Gaussian-Laplacian pyramid (LoG) instead of the difference in Gaussians (DoG) to enhance the stability of feature points. The process involves multi-scale decomposition of the image, calculating the Laplacian response of each scale layer to construct a Gaussian-Laplacian pyramid, further finding extreme points in the scale space and using the Hessian matrix to remove edge response points, and further using the gradient direction histogram to generate feature descriptors. Finally, an enhanced SIFT feature point set (including location, scale, orientation, and descriptor), a feature quality assessment report, and feature distribution statistical analysis data are obtained.

[0074] Step 4) Based on the data from Step 1, parse the DXF file to read the entity data and extract its coordinates and attributes;

[0075] Step 5) Based on the results of Step 4, perform topology checking and repair. The process is as follows: For closed but not closed polygons, check the distance between the first and last points of the polyline; if it is less than the threshold, automatically close the polygon; for redundant vertices, use the Douglas-Peucker algorithm to simplify the polygon and delete collinear redundant vertices; for intersection boundary repair, use the planar scanning algorithm to detect line segment intersections and automatically break the intersecting line segments; verify the consistency between the DXF coordinate system and the image coordinate system and automatically convert to the Unified Coordinate System (UTM); finally, obtain the topology-repaired DXF vector data.

[0076] Step 6) Based on the results of Step 3 and Step 5, use kd-tree nearest neighbor search (FLANN) to perform initial feature point matching, and further use the improved RANSAC algorithm (combined with geographical constraints) to remove mismatched points; finally, store the matched feature point pairs in matrix form, with each row representing a matching pair, including image pixel coordinates and vector world coordinates; further store the matched feature point pairs in matrix form, with each row representing a matching pair, including image pixel coordinates and vector world coordinates to obtain the feature point matching matrix.

[0077] For specific implementation details, please refer to the appendix. Figure 3 The coordinate fusion module includes a feature point matching and verification submodule, a coordinate parameter transformation submodule, and a pixel vector binding submodule;

[0078] The feature point matching verification submodule reads the feature point matching matrix and checks the number and quality of matching pairs to verify the consistency between the georeferenced information of the orthophoto and the DXF vector data.

[0079] The coordinate parameter transformation submodule uses thin-plate spline functions to calculate the nonlinear transformation from image pixel coordinates to vector world coordinates, solves for the TPS transformation parameters, and outputs them.

[0080] The pixel vector binding submodule converts the coordinates of each pixel in the orthophoto into world coordinates through TPS transformation, assigns a unique ID to each vector feature, establishes the association between the vector feature and the image pixel, determines the image pixel area it covers, and further establishes a pixel-vector binding table to record each vector feature ID and its corresponding image pixel area.

[0081] The energy function for solving the TPS transform described above is constructed as follows: E TPS The energy function of the thin-plate spline is represented (the objective is to minimize this energy value); n represents the number of matching point pairs; v i P represents the coordinates of the i-th control point in the target coordinate system (vector coordinate system); i f(P) represents the coordinates of the i-th control point in the source coordinate system (image pixel coordinate system); i ) represents the transformation function f at point P. i The value at; ||v i -f(P i )|| 2 Let f(i) represent the square of the Euclidean distance (measuring the transformation error at the i-th control point); λ represents the regularization parameter; J(f) represents the energy curvature term; its formula is:

[0082]

[0083] Where J(f) represents the energy bending term; It represents the second partial derivative of the function f with respect to x (curvature in the x-direction); The mixed second-order partial derivative (degree of distortion) of the function f is represented. The second-order partial derivative of function f with respect to y (curvature in the y-direction) is used to establish a precise mapping relationship from image pixel coordinates to world coordinates, solve the nonlinear deformation caused by terrain undulation, shooting angle, etc., and finally obtain the TPS transformation parameters.

[0084] In practice, the coordinate fusion module also includes a spatial index construction submodule and a model generation submodule;

[0085] The spatial index construction submodule includes a vector feature ID allocation unit, a spatial association establishment unit, and an association strategy unit;

[0086] The vector feature ID allocation unit assigns a unique identifier to each DXF vector feature and establishes a feature attribute index table.

[0087] For each vector feature, the spatial association establishment unit calculates its coverage area on the image and further uses a scan line filling algorithm to determine the pixel area corresponding to the vector polygon.

[0088] The association strategy unit constructs a pyramid-structured association index, which includes L0 for precise querying, L1 for fast display, and L2 for overview, with resolutions ranging from the original resolution to 1 / 4 resolution to 1 / 16 resolution.

[0089] The model generation submodule creates a raster-vector fusion data structure and further performs boundary smoothing to eliminate jagged edges. The smoothing formula is: C 混 =α·C 矢原 +(1-α)·C 正原 C 混 Indicates the final displayed mixed color; C 矢原 Represents the original color of the vector element; C 正原 The original pixel color of the orthophoto is represented by α; the mixing factor is represented by α; a data fusion model is established by constructing sub-modules and model generation sub-modules based on the above spatial index.

[0090] The spatial association establishment unit described above calculates the coverage area of ​​each vector feature on the image using the following formula:

[0091] R 矢 =f -1 (R 覆 ), where R 矢 Indicates the area covered by vector features in the image pixel coordinate system; f -1 Represents the inverse TPS transform; R 覆 This indicates the area covered by a vector feature in the world coordinate system.

[0092] The above model generation submodule uses a block storage mechanism to divide a large mining area into several blocks, implementing a lazy loading mechanism that loads only the fused data within the visible area.

[0093] In practice, the coordinate fusion module also includes a quality assessment submodule;

[0094] The quality assessment submodule evaluates the fusion effect based on the data fusion model, the original feature point matching matrix and the TPS transformation result. The quality assessment submodule includes a consistency check unit, an accuracy verification unit and a visual effect evaluation unit.

[0095] The consistency check unit randomly selects a set of verification points and calculates the reprojection error.

[0096] The accuracy verification unit uses an independent control point set to verify the fusion accuracy and calculates the position accuracy index, which includes planar accuracy and elevation accuracy.

[0097] The visual effect evaluation unit uses the Structural Similarity Index (SSIM) to evaluate the superposition effect. The SSIM formula is as follows:

[0098]

[0099] Where SSIM(x, y) represents the similarity index of the two images to be compared; (2μ x μ y +c1)(2σ xy +c2) represents the molecular part, namely the combined factor of brightness, contrast, and structural similarity; c1 and c2 represent constants; μ x μ represents the average pixel value of image patch x. y This represents the average pixel value of image patch y; This indicates that the molecules are being normalized. Represents the variance of image patch x; σ represents the variance of the image y; xy It represents the covariance between image patches x and y; it integrates geometric accuracy and visual quality through multi-dimensional quality assessment.

[0100] The calculation formula for the reprojection error described in the above consistency check unit is as follows:

[0101]

[0102] in The reprojection error is represented by 're'; m represents the number of verification points; v j f(v) represents the world coordinates of the j-th verification point; j ) indicates the verification point v j Apply forward transform; f -1 f represents the inverse transformation function (i.e., from world coordinates to pixel coordinates); ||...|| represents the Euclidean norm (i.e., the vector magnitude); -1 (f(v j )) indicates a "round trip" transformation of the verification point; the Euclidean norm, for two-dimensional coordinates ((x1, x2) and (y1, y2)), calculates the straight-line distance between two points, and its calculation formula is: , where ||(x1, y1) - (x2, y2)|| represent the coordinates of two points on a two-dimensional plane; (x1 - x2) represents the difference in coordinates of the two points on the x-axis (i.e., the horizontal distance); (y1 - y2) represents the difference in coordinates of the two points on the y-axis (i.e., the vertical distance); the calculation principle on the right is an application of the Pythagorean theorem, and its significance is to quantify the actual distance of spatial position error.

[0103] The aforementioned accuracy verification unit calculates position accuracy indicators, including planar accuracy and elevation accuracy. The formula for calculating the planar accuracy indicator is as follows: , where σ xyThe plane accuracy index is represented by Δx and Δy; Δx and Δy represent coordinate residuals (the difference between observed and actual values); n represents the number of verification points; the coefficient 2 in the denominator indicates that there are two components (i.e., x and y); ∑ represents the summation sign; the formula for calculating the elevation accuracy index is: , where σ z Δz represents the elevation accuracy index; Δz represents the coordinate residual (the difference between the observed value and the actual value); n represents the number of verification points.

[0104] For specific implementation details, please refer to the appendix. Figure 4 The dynamic update module includes a data stream submodule, a change detection and difference localization submodule, and a data interpolation extrapolation submodule.

[0105] The data stream submodule includes a data acquisition unit, a parsing unit, and an associated output unit;

[0106] The data acquisition unit obtains GPS coordinates via the MQTT protocol, periodically acquires updated production data from the database, periodically acquires radar point cloud data via FTP, and obtains a data fusion model through the aforementioned coordinate fusion module.

[0107] The analysis unit converts GPS coordinates into a unified coordinate system used by the data fusion model, filters and thins the geological radar point cloud, and extracts information on changes in the mining face.

[0108] The associated output unit spatially associates the device location with the vector elements in the data fusion model to determine the road segment where the device is located and outputs the parsed standardized data and the associated results.

[0109] The change detection and differential location submodule performs multi-dimensional processing based on the standardized data and correlation results output above, including detection of changes in the mining face, detection of changes in equipment location, and analysis of changes in production output. The processing result is a list of change areas.

[0110] The data interpolation and extrapolation submodule performs interpolation and extrapolation based on the list of changed areas and historical production data. The process includes Kriging interpolation, temporal extrapolation and spatial propagation simulation. The result is the production status of the entire area after interpolation and extrapolation.

[0111] The aforementioned change detection and difference localization submodule, which includes multi-dimensional data processing such as mining face change detection, equipment location change detection, and production change analysis, implements the following steps:

[0112] Step 1) Detect changes in the mining face using the frame difference method. The calculation formula is as follows:

[0113] D(x,y,t)=|I(x,y,t)-I(x,y,t-Δt)|>τ 阈

[0114] Where D(x,y,t) represents the binary result image of change detection (value 0 or 1); I(x,y,t) represents the pixel intensity value of the image at position (x,y) at time t; I(x,y,t-Δt) represents the pixel intensity value of the image at the same position (x,y) at the previous time (t-Δt); Δt represents the time interval; |...| represents the absolute value operation; τ 阈 This represents the change detection threshold. This value is a dynamic threshold determined by multiple factors, including weather factors, seasonal factors, and the characteristics of surface materials in the mine. The specific setting logic is as follows:

[0115] Weather factors: Sunny (stable lighting, minimal shadow interference, set to τ) 阈 =15), Cloudy / Overcast (uniform lighting with no shadows, but overall contrast reduced, set to τ) 阈 =10) and complex weather with many shadows (unstable lighting and large shadow interference, set to τ) 阈 =35);

[0116] Seasonal factors: Spring / Autumn season (transitional instability is set to τ) 阈 =50), Summer (increase the threshold to reduce the disturbance of vegetation change by τ) 阈 =25) and winter (surface and mining face characteristics are stable as τ) 阈 =15);

[0117] Mining surface material characteristics: Homogeneous material zone (since the variation mainly comes from topographic excavation, moderate pixel differences are set to τ). 阈 =15) and the high-contrast material alternation zone (the boundary zone between coal and limestone, where the material itself varies greatly, set as τ). 阈 =45);

[0118] This formula can distinguish between real changes and minor fluctuations caused by noise, ensuring that only meaningful changes in production activities are detected.

[0119] Step 2) Extract the comprehensive feature vector of the changed region, including the area of ​​the changed region, the perimeter of the changed region, the shape compactness index, the elevation change characteristics, and production-related characteristics. The formula for calculating the shape compactness index is: , where f 紧 Indicates the shape compactness index; p 2 The area represents the perimeter of the region of change; π represents the mathematical constant pi; 4π·area represents 4π times the area of ​​the equivalent circle.

[0120] Step 3) Identify change types based on a random forest classifier to form a list of change areas, including [mining progress, spoil heap expansion, transport road changes, equipment relocation].

[0121] The above-mentioned data interpolation and extrapolation submodule includes Kriging interpolation, temporal extrapolation, and spatial propagation simulation. The implementation steps are as follows:

[0122] Step 1) Construct a three-dimensional spatiotemporal variability function. The modeling formula for the constructed function is:

[0123]

[0124] Where γ(h) s h t ) represents the value of the spatiotemporal variability function; h s and h t N(h) represents time and spatial distance, respectively. s h t ) represents the distance pair (h) in a specific time and space. s h t The number of data pairs within ); ∑[z(s i , t i )-z(s j ,t j )]² represents the sum of squares of the differences between all pairs of points that satisfy the distance condition; z(s) i , t i ) indicates at position s i and time t i The observed values; the purpose of this formula is to quantify the spatial and temporal correlation structure of mine production data;

[0125] Step 2) Based on historical production data, changes in mining face, and the variogram model constructed in Step 1 above, interpolate the production distribution of the entire region to generate a production distribution heat map;

[0126] Step 3) Use the ARIMA model to make time series predictions of output, extrapolate the production status in the future, and combine it with the planning data to calculate the estimated time to complete the plan.

[0127] In practice, the dynamic update module also includes a layer generation and update submodule and an early warning signal submodule;

[0128] The layer generation and update submodule constructs a dynamic layer based on the interpolated extrapolated production status of the entire area and uses an incremental update mechanism to update the model blocks corresponding to the changed areas.

[0129] The warning signal submodule is based on an updated model and sets up a multi-level warning mechanism, which is divided into different levels according to the severity and adopts different notification methods, including information notification and audio notification.

[0130] The specific implementation steps for constructing a dynamic layer and using an incremental update mechanism to update the model blocks corresponding to the changed areas are as follows:

[0131] Step 1) Create a new layer to store dynamic production plan data;

[0132] Step 2) Based on Step 1, convert the interpolation and extrapolation results into vector features (contour lines, advance boundaries) and raster layers (heatmaps);

[0133] Step 3) Based on Step 2, merge the changed areas into the data fusion model and update the mining face boundary and transportation route;

[0134] Step 4) Based on Step 3, use the incremental update mechanism to update only the model blocks corresponding to the changed areas;

[0135] Step 5) Based on steps 1-4 above, create a version for each update, and record the timestamp and update content.

[0136] The above-mentioned multi-level early warning mechanism implements tiered alarms, and the specific implementation steps are as follows:

[0137] Step 1) Set early warning rules, including equipment exceeding the preset area, output below the threshold, and mining face advancing too fast. Real-time data is matched with the rules to trigger the early warning mechanism.

[0138] Step 2) Based on Step 1, assign audio prompts to the warning signals and bind the audio to spatial coordinates. When the administrator views the 3D scene, the warning audio at the corresponding location is played according to the viewing direction.

[0139] Step 3) Based on Step 1-2, when an alert is triggered, push the alert information to the client through the data interface. The content includes the alert location, type, level, and recommended measures. The data is stored and sent in JSON format.

[0140] For specific implementation details, please refer to the appendix. Figure 5 The visualization module includes a data loading submodule and a scene graph construction and rendering submodule;

[0141] The data loading submodule prepares data based on dynamic layers, updated models, and early warning data. Its processing includes data conversion and analysis, audio signal processing, and model optimization. The processing results are a 3D scene map, a set of LOD models, and a spatial audio object library.

[0142] The scene graph construction and rendering submodule is based on the aforementioned 3D scene graph, LOD model set, and spatial audio object library to construct and render the scene. Its processing includes scene graph construction → LOD rendering → spatial audio rendering → dynamic data rendering. The processing result is the rendered 3D scene and real-time audio data.

[0143] In practice, the visualization module also includes a 3D output submodule;

[0144] The 3D output submodule, based on the rendered 3D scene and real-time audio data, uses a multi-view output mode to materialize the rendered 3D scene data and present it on the system page.

[0145] The specific steps for preparing and processing the above data are as follows:

[0146] Step 1) The data transformation and analysis involves parsing the dynamic production plan layer and extracting the mine status at each moment in the time series; further, the vector data is converted into three-dimensional geometry (that is, the mining boundary is converted into polygons on the elevation terrain and stretched into a three-dimensional body); further, orthophotos are loaded as terrain textures and mapped onto the three-dimensional terrain according to the output of the coordinate fusion module;

[0147] Step 2) The audio signal processing involves parsing the real-time warning signal dataset, binding the audio file to spatial coordinates, and playing it as spatial audio in the three-dimensional scene;

[0148] Step 3) The model optimization simplifies the 3D model and uses LOD technology to generate models with multiple resolutions so that dynamic switching can be implemented according to the viewpoint distance.

[0149] The above-described scene construction and rendering process involves the following specific steps:

[0150] Step 1) Scene graph construction: Construct a hierarchical scene graph, including root node, terrain node, vector element node and dynamic layer node. Further, map the orthophoto texture onto the terrain geometry to form the basic terrain. Further, add vector elements (mining boundary and transportation road) to the scene in three-dimensional geometry and set the color and transparency according to the attributes.

[0151] Step 2) The LOD rendering, based on the distance of the image acquisition device, selects an appropriate LOD level for each model, calculates the distance between the device and the object, and further selects the pre-generated model with the corresponding level of detail to achieve a smooth LOD transition and avoid sudden switching;

[0152] Step 3) The spatial audio rendering calculates the relative position of each audio source based on the position and orientation of the listener (i.e., the image acquisition device), and generates a stereo effect using HRTF (Head-Related Transfer Function).

[0153] Step 4) The dynamic data rendering, based on the progress of the production plan, dynamically displays the changes in the mining face.

[0154] The above-mentioned multi-view output mode is used to materialize and present the data on the system page. The implementation steps are as follows:

[0155] Step 1) Using the multi-view output mode, the main view, top view and section view are generated simultaneously and displayed on the same interface, and the view can be output to multiple display devices;

[0156] Step 2) Set up the API interface so that module data can be accessed by calling the interface through external devices, and 3D visualization on the browser side can be achieved using WebGL technology.

[0157] Working Principle: The system uses a data processing module as the foundation for raw data processing. This module ensures the geometric, radiometric, and topological consistency of multi-source data through automated algorithm processes and incorporates geographic information data. A coordinate fusion module is the core module for establishing the data fusion model. Utilizing a thin-plate spline TPS transformation model, it precisely overlays orthophotos and DXF vector data onto a unified coordinate system, establishing a mapping between pixels and vector elements to generate a raster-vector data fusion model. A dynamic update module is the core data analysis and update module. This module introduces real-time production data through data interface technology and uses spatiotemporal data analysis and change detection algorithms to dynamically update the data fusion model and simulate the evolution of production plans. A visualization module is the core presentation module. This module renders the data fusion model and dynamic data in a 3D scene, supporting multi-view output through LOD multi-resolution technology and real-time projection transformation to complete the system's visualization design.

[0158] Those skilled in the art will recognize that the modules, units, and algorithm implementation steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A visualization system for production planning in open-pit mines based on the overlay of orthophotos and DXF vectors, characterized in that, The system specifically includes: a data processing module, a coordinate fusion module, a dynamic update module, and a visualization module, which are connected in sequence. The data processing module preprocesses and extracts features from the original orthophoto and DXF vector data, and outputs a feature matching matrix. The coordinate fusion module performs precision superposition of data based on the feature matching matrix to generate a data fusion model with coordinate system one, and the model outputs a set of coordinate transformation parameters. The dynamic update module, based on the coordinate transformation parameter set, introduces real-time production data to update the data fusion model, simulates the evolution of the production plan, and outputs the production plan layer and early warning signal dataset. The visualization module renders and visualizes the production plan layer and the early warning signal dataset in a three-dimensional scene. The data processing module processes data including orthophoto data, DXF vector file data, and mine geographic information data acquired by airborne lidar; using geometric correction and DXF vector topology repair, based on the mine geographic information data, it processes the orthophoto data and DXF vector file data to make them accurately aligned on the same reference plane and form a feature point matching matrix.

2. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 1, characterized in that, The coordinate fusion module includes a feature point matching and verification submodule, a coordinate parameter transformation submodule, and a pixel vector binding submodule; The feature point matching verification submodule reads the feature point matching matrix, checks the number and quality of matching pairs, and verifies the consistency between the georeferenced information of the orthophoto and the DXF vector data. The coordinate parameter transformation submodule uses thin-plate spline functions to calculate the nonlinear transformation from image pixel coordinates to vector world coordinates, solves for the TPS transformation parameters, and outputs them. The pixel vector binding submodule converts the coordinates of each pixel in the orthophoto into world coordinates through TPS transformation, assigns a unique ID to each vector feature, establishes the association between the vector feature and the image pixel, determines the image pixel area it covers, and further establishes a pixel-vector binding table to record each vector feature ID and its corresponding image pixel area.

3. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 2, characterized in that, The coordinate fusion module also includes a spatial index construction submodule and a model generation submodule; The spatial index construction submodule includes a vector feature ID allocation unit, a spatial association establishment unit, and an association strategy unit; The vector feature ID allocation unit assigns a unique identifier to each DXF vector feature and establishes a feature attribute index table. For each vector feature, the spatial association establishment unit calculates its coverage area on the image and further uses a scan line filling algorithm to determine the pixel area corresponding to the vector polygon. The association strategy unit constructs a pyramid-structured association index, which includes L0 for precise query, L1 for fast display, and L2 for overview, with resolutions of original resolution → 1 / 4 resolution → 1 / 16 resolution in sequence. The model generation submodule creates a raster-vector fusion data structure and further performs boundary smoothing to eliminate jagged edges. The smoothing formula is: C 混 =α·C 矢原 +(1-α)·C 正原 C 混 Indicates the final displayed mixed color; C 矢原 Represents the original color of the vector element; C 正原 The original pixel color of the orthophoto is represented by α; the mixing factor is represented by α; a data fusion model is established through a spatial index construction submodule and a model generation submodule.

4. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 3, characterized in that, The coordinate fusion module also includes a quality assessment submodule; The quality assessment submodule evaluates the fusion effect based on the data fusion model, the original feature point matching matrix and the TPS transformation result. The quality assessment submodule includes a consistency check unit, an accuracy verification unit and a visual effect evaluation unit. The consistency check unit randomly selects a set of verification points and calculates the reprojection error. The accuracy verification unit uses an independent control point set to verify the fusion accuracy and calculates the position accuracy index, which includes planar accuracy and elevation accuracy. The visual effect evaluation unit uses the Structural Similarity Index (SSIM) to evaluate the superposition effect. The SSIM formula is as follows: ， Where SSIM(x, y) represents the similarity index of the two images to be compared; (2μ x μ y +c1)(2σ xy +c2) represents the molecular part, namely the combined factor of brightness, contrast, and structural similarity; c1 and c2 represent constants; μ x μ represents the average pixel value of image patch x. y This represents the average pixel value of image patch y; This indicates that the molecules are being normalized. Represents the variance of image patch x; σ represents the variance of the image y; xy It represents the covariance between image patches x and y; it integrates geometric accuracy and visual quality through multi-dimensional quality assessment.

5. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 4, characterized in that, The dynamic update module includes a data stream submodule, a change detection and difference localization submodule, and a data interpolation extrapolation submodule. The data stream submodule includes a data acquisition unit, a parsing unit, and an associated output unit; The data acquisition unit obtains GPS coordinates via the MQTT protocol, periodically acquires updated production data from the database, periodically acquires radar point cloud data via FTP, and obtains a data fusion model through the aforementioned coordinate fusion module. The analysis unit converts GPS coordinates into a unified coordinate system used by the data fusion model, filters and thins the geological radar point cloud, and extracts information on changes in the mining face. The associated output unit spatially associates the device location with the vector elements in the data fusion model to determine the road segment where the device is located and outputs the parsed standardized data and the associated results. The change detection and differential location submodule performs multi-dimensional processing based on the standardized data and correlation results output above, including detection of changes in the mining face, detection of changes in equipment location, and analysis of changes in production output. The processing result is a list of change areas. The data interpolation and extrapolation submodule performs interpolation and extrapolation based on the list of changed areas and historical production data. The process includes Kriging interpolation, temporal extrapolation and spatial propagation simulation. The result is the production status of the entire area after interpolation and extrapolation.

6. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 5, characterized in that, The dynamic update module also includes a layer generation and update submodule and an early warning signal submodule; The layer generation and update submodule constructs a dynamic layer based on the interpolated extrapolated production status of the entire area and uses an incremental update mechanism to update the model blocks corresponding to the changed areas. The warning signal submodule is based on an updated model and sets up a multi-level warning mechanism, which is divided into different levels according to the severity and adopts different notification methods, including information notification and audio notification.

7. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 6, characterized in that, The visualization module includes a data loading submodule and a scene graph construction and rendering submodule; The data loading submodule prepares data based on dynamic layers, updated models, and early warning data. Its processing includes data conversion and analysis, audio signal processing, and model optimization. The processing results are a 3D scene map, a set of LOD models, and a spatial audio object library. The scene graph construction and rendering submodule is based on the aforementioned 3D scene graph, LOD model set, and spatial audio object library to construct and render the scene. Its processing includes scene graph construction → LOD rendering → spatial audio rendering → dynamic data rendering. The processing result is the rendered 3D scene and real-time audio data.

8. The open-pit mine production planning visualization system based on orthophoto and DXF vector overlay as described in claim 7, characterized in that, The visualization module also includes a 3D output submodule; The 3D output submodule, based on the rendered 3D scene and real-time audio data, uses a multi-view output mode to materialize the rendered 3D scene data and present it on the system page.

Images