Automatic extraction method and system of step lines in open-pit mines based on mine segmentation model

Abstract

The present invention provides an automatic extraction method and system for step lines in an open-pit mine area based on a mine segmentation model, the method comprising: obtaining control point deployment and performing control point measurement, performing route planning to capture images in a target area; reconstructing an open-pit mine area tilt model based on the acquired images; generating a digital normal model based on the open-pit mine area tilt model, interactively annotating step areas on a segmentation data set of the digital normal model, and obtaining a segmentation data set of the digital normal model; using the segmentation data set of the digital normal model as training data, training a mine area step segmentation large model network, and obtaining a mine area step segmentation large model; using the mine area step segmentation large model to obtain segmentation results and step line buffer information, dividing the step line into a step slope top line and a step slope bottom line, and thereby extracting and classifying the step line. The present invention is to improve the intelligent level of step monitoring in an open-pit mine area, ensure safe mining in a mine area, and realize fully automatic extraction of step lines in a mine area.

Description

Technical Field

[0001] The present invention relates to the fields of photogrammetry, three-dimensional reconstruction, and artificial intelligence, and in particular to a method and system for automatically extracting step lines in an open-pit mine based on a mine segmentation model. Background Art

[0002] Mine safety is of irreplaceable importance to national economic construction and strategic resource security. However, open-pit mine steps are affected by multiple factors such as tectonic stress, water, and vibration, and are prone to geological disasters such as landslides, collapses, and mudslides, posing a serious threat to equipment safety and personnel safety. In order to effectively identify potential risks, regulatory agencies need to continuously monitor the dynamic changes of step topography and rock mass structural surfaces.

[0003] Although the traditional geodetic geometric monitoring method is accurate, due to its high labor intensity, high risk and terrain restrictions, manual step measurement has gradually been replaced by non-contact measurement. With the rapid advancement of modern surveying and mapping technology, a variety of new step monitoring methods such as satellite navigation and positioning system (GNSS) deformation monitoring, radar differential interferometry (D-InSAR), close-range and unmanned aerial vehicle (UAV) aerial photogrammetry, and long-range three-dimensional laser scanning (TLS) have been applied in the mining field. The development of these technologies has transformed step monitoring from traditional point monitoring to surface and even volume monitoring. However, different monitoring methods have different applicability and limitations due to inherent defects.

[0004] Although GNSS monitoring has the characteristics of high precision and fast response, and can realize online and continuous detection, the satellite signal is poor in deep pit environments, which is prone to multi-path effects and affects the monitoring effect. Although D-InSAR technology can realize surface monitoring and has high measurement accuracy, it is difficult to obtain and process data. Although ground-based TLS is fast and accurate, it has blind spots in scanning and is more complicated to process large-scale point cloud data. Oblique photogrammetry technology is recognized for its high cost-effectiveness, short production cycle, simple operation and good visualization effect, but the amount of data it generates is large and the structure is complex, and it also faces data processing problems. Due to the high steps, steep slopes and large drop in large open-pit mines, the above methods have limitations in practical applications.

[0005] In recent years, deep learning methods have made significant progress in the field of image processing, and the emergence of large image segmentation models has provided new possibilities for image-based step extraction. Although these models show high accuracy in conventional segmentation tasks, their adaptability is still insufficient and their segmentation capabilities are limited for highly professional downstream segmentation tasks, such as mining step extraction. Due to the particularity of the geological topography of the mining area, the color difference between the steps and the road surface is small, which increases the difficulty of measuring the step terrain without the assistance of three-dimensional information. Therefore, it is urgent to develop a fully automatic step detection method for large-scale open-pit mines to improve the level of intelligence in mining monitoring and ensure the safety and efficiency of mining. Summary of the invention

[0006] In view of the technical problems existing in the prior art, the present invention provides an automatic extraction method and system for step lines in open-pit mines based on a mine segmentation model, so as to improve the intelligence level of step monitoring in open-pit mines, ensure safe mining in mines, and realize fully automatic extraction of step lines in mines.

[0007] According to a first aspect of the present invention, the present invention provides an automatic extraction method of step lines in an open-pit mine area based on a mine segmentation model, comprising the following steps:

[0008] Obtain control point deployment and perform control point measurement, perform route planning and capture images within the target area;

[0009] Reconstruction of the open pit mine area tilt model using photogrammetry based on acquired images;

[0010] Generate a digital normal model based on the open pit mine area tilt model, perform interactive step area annotation on the segmented data set of the digital normal model, and obtain the segmented data set of the digital normal model;

[0011] The segmentation data set of the digital normal model is used as training data to train the large model network of the mining area step segmentation to obtain the large model of the mining area step segmentation;

[0012] The segmentation results and step line buffer information are obtained by using the large model of step segmentation in the mining area. The step lines are divided into step slope top lines and step slope bottom lines, and the step lines are extracted and classified based on this.

[0013] Based on the above technical solution, the present invention can also make the following improvements.

[0014] Optionally, obtaining control point deployment and performing control point measurement includes:

[0015] The scope of the measurement area is clarified, the control point positions are selected, and the control point positions are measured to obtain control point data; the control point data includes the distribution and coordinates of the control points, and the image data of the drone, and the image data includes the original image and image coordinates.

[0016] Optionally, the performing route planning to capture images of the target area includes:

[0017] Perform flight mission planning, define modeling areas and boundaries, determine flight altitude and camera inclination;

[0018] A preset number of ground control points are laid out in the mining area for subsequent image correction and accuracy verification;

[0019] Perform flight missions, using drones or aircraft for oblique photography, to capture images of the target area and for each image, record the camera's GPS coordinates.

[0020] Optionally, the reconstructing the open pit mine area tilt model using photogrammetry based on the acquired images includes:

[0021] Extract key feature points from the acquired images, including corner points, edges and textures, and match the same feature points in different images to determine the corresponding relationship between the same feature points in different images;

[0022] According to the matched feature points and camera parameters, the three-dimensional coordinates of each feature point are calculated to generate a sparse three-dimensional point cloud, wherein the sparse three-dimensional point cloud represents the position of the point in the scene, and the image pose of the photo is obtained according to the sparse three-dimensional point cloud;

[0023] Generating a dense three-dimensional point cloud using image matching and interpolation techniques according to the sparse three-dimensional point cloud and image pose;

[0024] Based on the dense 3D point cloud, a surface model is created to represent the 3D shape of the scene using interpolation and 3D mesh generation algorithms;

[0025] The acquired image texture is mapped onto the surface model to obtain a tilted model of the target area.

[0026] Optionally, generating a digital normal model based on the open pit mine area tilt model includes:

[0027] Establish a regular grid network in the target area according to the target resolution;

[0028] Take the grid point of the grid network as the starting point to establish a Rays in the positive direction of the axis ;

[0029] Calculate Rays Intersection with tilted model ;

[0030] By intersection The normal vector of the triangle is The radian value of the angle in the axis direction is used as the pixel value of the grid unit.

[0031] Optionally, the training of the large model network for mining area step segmentation to obtain the large model for mining area step segmentation includes:

[0032] Add feature fusion module based on SAM large model and introduce visual features and high frequency components , add these two features according to a certain weight and input them into the feature fusion module to obtain the fusion feature , the digital normal model image of the mining area is used as training data, and the large model network of the mining area step segmentation is retrained to obtain the large model of the mining area step segmentation.

[0033] Optionally, the visual feature Helps migrate the original data training to the new training task, the high-frequency component The target feature information in the digital normal model data is extracted by fast Fourier transform.

[0034] Optionally, the fusion feature It is expressed as follows:

[0035]

[0036] in, is an activation function, It is a linear layer used to generate different fusion features in each feature fusion module; is an upward projection layer shared among all feature fusion modules to match the dimension of ViT features. Appended to each ViT layer.

[0037] Optionally, the step of using the large mining area step segmentation model to obtain the segmentation result and step line buffer information, and dividing the step line into a step slope top line and a step slope bottom line includes:

[0038] Taking the step line as the center, take a certain length of area to the left and right sides as the left and right buffer zones, and calculate the average height and average normal value of the left and right buffer zones respectively;

[0039] If the buffer zone on one side has a higher average height than the other side and a smaller average normal value, then this step line is the top line of the step slope; conversely, if the buffer zone on one side has a higher average height than the other side but a larger average normal value, then this step line is the bottom line of the step slope.

[0040] According to a second aspect of the present invention, there is provided an automatic extraction system for step lines in an open-pit mine area based on a mine segmentation model, comprising:

[0041] The mining area data acquisition module is used to obtain the control point deployment and control point measurement, and to plan the route and capture the image of the target area;

[0042] A tilt model reconstruction module, which is used to reconstruct the tilt model of the open-pit mine area using photogrammetry based on the acquired images;

[0043] A segmentation data set construction module is used to generate a digital normal model set based on the open pit mine area tilt model, interactively annotate the step area of ​​the segmentation data set of the digital normal model, and obtain the segmentation data set of the digital normal model;

[0044] A mining area step segmentation large model construction module is used to use the segmentation data set of the digital normal model as training data to train the mining area step segmentation large model network to obtain the mining area step segmentation large model;

[0045] The step line extraction and classification module is used to use the large model of mining area step segmentation to obtain segmentation results and step line buffer information, divide the step line into the step slope top line and the step slope bottom line, and use this to extract and classify the step line.

[0046] Technical effects and advantages of the present invention:

[0047] The method and system for automatically extracting step lines in open-pit mines based on the mine segmentation model provided by the present invention can acquire data at high frequency based on the low cost and fast speed of oblique photography data. A digital normal model is proposed, which includes model normal information and significantly reduces the amount of data, and improves the distinction of step structures. A large model for extracting step lines in mining areas is established, and the features of mining area data are integrated through the introduction of a feature fusion module to improve the accuracy of step line extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of a method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of a mining area tilt model provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of creating a digital normal model provided by an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of the original image, digital normal model and annotation result provided by an embodiment of the present invention;

[0052] Figure 5 It is a structural diagram of a mining area step segmentation model provided by an embodiment of the present invention;

[0053] Figure 6It is a schematic diagram of a step structure in a mining area provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In order to improve the intelligence level of step monitoring in open-pit mines, ensure safe mining in mines, and realize fully automatic extraction of step lines in mines, an embodiment of the present invention provides an automatic extraction method and system for step lines in open-pit mines based on a mine segmentation model.

[0056] It is understandable that, based on the defects in the background technology, the embodiment of the present invention proposes an automatic extraction method of step lines in an open-pit mine area based on a mine segmentation model, specifically as follows: Figure 1 As shown, the method comprises the following steps:

[0057] Step S1, obtain control point deployment and perform control point measurement, perform route planning and capture images within the target area;

[0058] In this embodiment, the deployment of control points requires clarifying the scope of the survey area, selecting the location of the control points, and measuring the data of these control points. Route planning needs to ensure sufficient viewing angle and overlap, and use drones to capture images of the target area. The control point data includes the distribution and coordinates of the control points and drone image data, and the drone image data includes the original image and image coordinates.

[0059] In the above implementation scheme, the process of performing route planning to capture images within the target area specifically includes:

[0060] First, the flight mission is planned, the modeling area and boundaries are defined, and the flight altitude and camera inclination are determined;

[0061] Then, a sufficient number of ground control points are laid out in the mining area for subsequent image correction and accuracy verification;

[0062] Finally, the flight mission is carried out, using drones or aircraft for oblique photography to capture images of the target area, and for each image, the camera's location (GPS coordinates) is recorded.

[0063] Step S2, reconstructing the open pit mine area tilt model using photogrammetry based on the acquired images;

[0064] The open pit mine tilt model is as follows Figure 2As shown in the figure, the reconstruction of the tilt model of the open-pit mine area includes image matching, aerotriangulation, dense matching, surface reconstruction and texture mapping;

[0065] ① Image matching: extract key feature points from the image, such as corners, edges or textures, and match the same feature points in different images to determine their correspondence in different images, using feature descriptors and matching algorithms to achieve this;

[0066] ② Aerial triangulation: Based on the matched feature points and camera parameters, the 3D coordinates of each feature point are calculated to generate a sparse 3D point cloud to represent the position of the point in the scene and obtain the pose of the photo;

[0067] ③ Dense matching: Based on the sparse point cloud and image pose, dense 3D point cloud is generated using image matching and interpolation techniques;

[0068] ④Surface reconstruction: Based on the dense point cloud, use interpolation and 3D mesh generation algorithms to create a surface model to represent the 3D shape of the scene;

[0069] ⑤ Texture mapping: Map the collected image texture onto the surface model to obtain the tilted model of the target area.

[0070] Step S3, generating a digital normal model based on the open-pit mine area inclination model, and interactively labeling the step areas of the segmented data set of the digital normal model to obtain the segmented data set of the digital normal model;

[0071] The creation process of the digital normal model (DNM) is as follows: Figure 3 As shown, the generation of a digital normal model based on an open pit mine area tilt model comprises the following steps:

[0072] 1) Establish a regular grid network in the target model area according to the target resolution;

[0073] 2) Use the grid point as the starting point to establish Rays in the positive direction of the axis ;

[0074] 3) Calculate rays Intersection with tilted model ;

[0075] 4) Intersection The normal vector of the triangle is The radian value of the angle in the axis direction is the value of the grid unit (pixel).

[0076] The digital normal model (DNM) generates the following results: Figure 4As shown, finally, the digital normal model (DNM) data is segmented and labeled with Mask, and the AnyLabeling tool is used to perform manual interactive step area labeling based on the segmentation model.

[0077] Step S4, using the segmentation data set of the digital normal model as training data, training the mining area step segmentation large model network to obtain the mining area step segmentation large model;

[0078] The establishment of the feature fusion-based mining area step segmentation model FF-SAM (Feature Fusion SAM). In order to make full use of the segmentation capability of the SAM model and make the SAM segmentation model better applied to the downstream tasks of mining area step extraction, the SAM model is improved and a feature fusion module is added. According to the characteristics of the mining area DNM data, two visual features are introduced. and high frequency components , enhancing the network's ability to segment DNM data.

[0079] It should be noted that SAM stands for ‌ Segment Anything Model is a large image segmentation model that has made a breakthrough in the field of computer vision. It can segment any object in any image without any annotation. The working principle of this model is to complete the rapid segmentation of any target by providing Prompt prompts, which draws on the Prompt strategy in the NLP field. The SAM model contains three core components: image encoder, prompt encoder, and mask decoder. The image encoder is used to output image embedding, the prompt encoder is used to receive prompt information such as points, boxes, and text, and the mask decoder converts the encoded information of these two parts into mask output. This design allows image embedding to be reused in different prompts, and can handle ambiguity and predict multiple masks to handle polysemy.

[0080] The network structure of the feature fusion-based mining area step segmentation model FF-SAM (Feature Fusion SAM) is as follows Figure 5 As shown in the figure, the large-scale model FF-SAM for mining area step segmentation is mainly divided into three parts: image encoder, mask decoder and feature fusion module.

[0081] Image encoder: The image encoder is composed of multiple Vision Transformers (ViT). ViT first divides the input image into 16x16 pixel blocks, which are regarded as image "tokens" similar to words in NLP. Each image block is then converted into a one-dimensional embedding vector through a linear layer (that is, a fully connected layer). This process is similar to word embedding in NLP. Since the Transformer structure itself does not have the ability to process sequence order, it is necessary to add position encoding to each image block embedding vector to provide position information, and finally form the initial image feature vector;

[0082] The initial image feature vector is input into the multi-head self-attention module, and the neighborhood window size of the image block is 14x14. Four equally spaced global self-attention mechanisms are used in it, so that the model can pay attention to the information of all other blocks when processing each image block. This mechanism enables the model to capture the long-range dependencies between image blocks. After the self-attention module and the fully connected layer, normalization is performed to stabilize the training process. Then the position is input into the fully connected feedforward network, which processes the output of each position independently. After the self-attention module and the feedforward network, residual connections are performed, accompanied by layer normalization. This helps avoid the gradient vanishing problem that may occur in deep networks.

[0083] ViT adds a special learning vector, the classification head, in front of the image block sequence for the final classification task. After being processed by the Transformer encoding layer, the output of this classification head is used as the representation of the image. Finally, the output of the classification head is classified through a fully connected layer.

[0084] The parameters of this part use the parameters of the original SAM model and do not change during the training process.

[0085] Mask Decoder: The mask decoder uses the mask decoder part of the original SAM network, which is adapted to image processing tasks by transforming the traditional Transformer decoder. In this process, the network first combines the image feature embeddings produced by the encoder with additional cue information, and uses self-attention and cross-attention mechanisms to update these embeddings to better understand the image content. The feature space is then expanded to the same resolution as the original image through upsampling, and these features are mapped to the classifier through a multi-layer perceptron (MLP). Finally, a dynamic linear classifier calculates the probability of belonging to the foreground for each pixel position in the image, generating the final segmentation mask.

[0086] The parameters of this part are updated during the retraining process.

[0087] Feature fusion module: In order to improve the network's ability to segment DNM images, the two visual features Patch embedding ( ) and high-frequency components (High-frequency Components, ) to make learning adjustments. Helps migrate original data training to new training tasks. The target feature information in the DNM data is extracted through fast Fourier transform. The two features are added according to a certain weight and input into the feature fusion module to obtain the fusion feature. :

[0088]

[0089] in, is an activation function, It is a linear layer used to generate different fused features in each feature fusion module. is an upward projection layer shared among all feature fusion modules to match the dimension of ViT features. Appended to each ViT layer.

[0090] The DNM images of the mining area are used as training data, and the FF-SAM network is retrained to obtain a large model of mining area step segmentation.

[0091] Step S5: Use the large model of mining area step segmentation to obtain segmentation results and step line buffer information, and divide the step line into the step slope top line and the step slope bottom line, so as to extract and classify the step line.

[0092] In this embodiment, the segmentation result and step line can be obtained by using the large model of the mining area step segmentation, and the buffer zone can be obtained according to the step line. After obtaining the segmentation result of the mining area step, the step line can be obtained by extracting the segmentation boundary. The step structure of the mining area is as follows: Figure 6 As shown, the step lines are divided into step top lines and step bottom lines, and the step lines are classified in combination with the DSM data, including classifying the step lines using the model elevation information contained in the DSM data.

[0093] The step line is divided into the step top line and the step bottom line, and specifically includes:

[0094] Taking the step line as the center, take a certain length of area to the left and right sides as the left and right buffer zones, and calculate the average height and average normal value of the left and right buffer zones respectively;

[0095] If the buffer zone on one side has a higher average height than the other side and a smaller average normal value, then this step line is the top line of the step slope; conversely, if the buffer zone on one side has a higher average height than the other side but a larger average normal value, then this step line is the bottom line of the step slope.

[0096] In summary, the embodiment of the present invention provides an automatic extraction method of step lines in open-pit mines based on a mine segmentation model. This method is based on oblique photography data, is low-cost, fast, and can acquire data at high frequencies. A digital normal model DNM is proposed, which includes model normal information and significantly reduces the amount of data, and improves the distinction of step structures. A large model for extracting step lines in mining areas is established, and the features of mining area data are integrated through the introduction of a feature fusion module to improve the accuracy of step line extraction. This ensures safe mining in mining areas and realizes fully automatic extraction of step lines in mining areas.

[0097] According to a second aspect of the present invention, there is provided an automatic extraction system for step lines in an open-pit mine area based on a mine segmentation model, comprising:

[0098] The mining area data acquisition module is used to obtain the control point deployment and control point measurement, and to plan the route and capture the image of the target area;

[0099] A tilt model reconstruction module, which is used to reconstruct the tilt model of the open-pit mine area using photogrammetry based on the acquired images;

[0100] A segmentation data set construction module is used to generate a digital normal model set based on the open pit mine area tilt model, interactively annotate the step area of ​​the segmentation data set of the digital normal model, and obtain the segmentation data set of the digital normal model;

[0101] A mining area step segmentation large model construction module is used to use the segmentation data set of the digital normal model as training data to train the mining area step segmentation large model network to obtain the mining area step segmentation large model;

[0102] The step line extraction and classification module is used to use the large model of mining area step segmentation to obtain segmentation results and step line buffer information, divide the step line into the step slope top line and the step slope bottom line, and use this to extract and classify the step line.

[0103] It can be understood that the automatic extraction system for step lines in open-pit mines based on the mine segmentation model provided by the present invention corresponds to the automatic extraction method for step lines in open-pit mines based on the mine segmentation model provided in the aforementioned embodiments. The relevant technical features of the automatic extraction system for step lines in open-pit mines based on the mine segmentation model can refer to the relevant technical features of the automatic extraction method for step lines in open-pit mines based on the mine segmentation model, which will not be repeated here.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic extraction method of step lines in open-pit mines based on a mine segmentation model, characterized in that: The following steps are involved: Obtain control point deployment and perform control point measurement, perform route planning and capture images within the target area; Reconstruction of the open pit mine area tilt model using photogrammetry based on acquired images; Generate a digital normal model based on the open pit mine area tilt model, perform interactive step area annotation on the segmented data set of the digital normal model, and obtain the segmented data set of the digital normal model; The segmentation data set of the digital normal model is used as training data to train the large model network of the mining area step segmentation to obtain the large model of the mining area step segmentation; including: adding a feature fusion module on the basis of the SAM large model, introducing visual features and high frequency components , add these two features according to a certain weight and input them into the feature fusion module to obtain the fusion feature , the digital normal model image of the mining area is used as training data, and the mining area step segmentation large model network is retrained, that is, the mining area step segmentation large model is obtained; the visual feature Helps migrate the original data training to the new training task, the high-frequency component Extract target feature information from digital normal model data by fast Fourier transform; The segmentation results and step line buffer information are obtained by using the large model of step segmentation in the mining area. The step lines are divided into step slope top lines and step slope bottom lines, and the step lines are extracted and classified based on this.

2. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1 is characterized in that: The obtaining of control point deployment and performing control point measurement includes: The scope of the measurement area is clarified, the control point positions are selected, and the control point positions are measured to obtain control point data; the control point data includes the distribution and coordinates of the control points and the image data of the drone, and the image data includes the original image and image coordinates.

3. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1 is characterized in that: The route planning and capturing of images of the target area includes: Perform flight mission planning, define modeling areas and boundaries, determine flight altitude and camera inclination; A preset number of ground control points are laid out in the mining area for subsequent image correction and accuracy verification; Perform flight missions, using drones or aircraft for oblique photography, to capture images of the target area and for each image, record the camera's GPS coordinates.

4. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1 is characterized in that: The method of reconstructing the tilt model of the open-pit mine area using photogrammetry based on the acquired images includes: Extract key feature points from the acquired images, including corner points, edges and textures, and match the same feature points in different images to determine the corresponding relationship between the same feature points in different images; According to the matched feature points and camera parameters, the three-dimensional coordinates of each feature point are calculated to generate a sparse three-dimensional point cloud, wherein the sparse three-dimensional point cloud represents the position of the point in the scene, and the image pose of the photo is obtained according to the sparse three-dimensional point cloud; Generating a dense three-dimensional point cloud using image matching and interpolation techniques according to the sparse three-dimensional point cloud and image pose; Based on the dense 3D point cloud, a surface model is created to represent the 3D shape of the scene using interpolation and 3D mesh generation algorithms; The acquired image texture is mapped onto the surface model to obtain a tilted model of the target area.

5. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1 is characterized in that: The generating of the digital normal model based on the open pit mine area tilt model comprises: Establish a regular grid network in the target area according to the target resolution; Take the grid point of the grid network as the starting point to establish a Rays in the positive direction of the axis ; Calculate Rays Intersection with tilted model ; By intersection The normal vector of the triangle is The radian value of the angle in the axis direction is used as the pixel value of the grid unit.

6. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1 is characterized in that: The fusion feature It is expressed as follows: in, is an activation function, It is a linear layer used to generate different fusion features in each feature fusion module; is an upward projection layer shared among all feature fusion modules to match the dimension of ViT features. Appended to each ViT layer.

7. The method for automatically extracting step lines in an open-pit mine area based on a mine segmentation model according to claim 1, characterized in that: The step segmentation model of the mining area is used to obtain the segmentation result and the step line buffer information, and the step line is divided into the step slope top line and the step slope bottom line, which includes: Taking the step line as the center, take a certain length of area to the left and right sides as the left and right buffer zones, and calculate the average height and average normal value of the left and right buffer zones respectively; If the buffer zone on one side has a higher average height than the other side and a smaller average normal value, then this step line is the top line of the step slope; conversely, if the buffer zone on one side has a higher average height than the other side but a larger average normal value, then this step line is the bottom line of the step slope.

8. The automatic extraction system of step lines in open-pit mines based on the mine segmentation model is characterized by: include: The mining area data acquisition module is used to obtain the control point deployment and control point measurement, and to plan the route and capture the image of the target area; A tilt model reconstruction module, which is used to reconstruct the tilt model of the open-pit mine area using photogrammetry based on the acquired images; A segmentation data set construction module is used to generate a digital normal model set based on the open pit mine area tilt model, interactively annotate the step area of ​​the segmentation data set of the digital normal model, and obtain the segmentation data set of the digital normal model; The mining area step segmentation large model construction module is used to use the segmentation data set of the digital normal model as training data to train the mining area step segmentation large model network to obtain the mining area step segmentation large model; including: adding a feature fusion module on the basis of the SAM large model, introducing visual features and high frequency components , add these two features according to a certain weight and input them into the feature fusion module to obtain the fusion feature , the digital normal model image of the mining area is used as training data, and the mining area step segmentation large model network is retrained, that is, the mining area step segmentation large model is obtained; the visual feature Helps migrate the original data training to the new training task, the high-frequency component Extract target feature information from digital normal model data by fast Fourier transform; The step line extraction and classification module is used to use the large model of mining area step segmentation to obtain segmentation results and step line buffer information, divide the step line into the step slope top line and the step slope bottom line, and use this to extract and classify the step line.

Images