Methods, devices, equipment and storage media for representing open-pit mine scenes
By acquiring point cloud data of open-pit mines using lidar, correcting pose deviations using ground normal vectors, and constructing histogram descriptors, the accuracy and real-time performance issues of mine scene recognition algorithms under limited computing resources are solved, achieving efficient mine scene recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-23
- Publication Date
- 2026-07-03
AI Technical Summary
Open-pit mine scene recognition algorithms are difficult to run in real time under the limited computing conditions of mining trucks, and commonly used descriptors are difficult to accurately describe the complex three-dimensional shape and irregular geometric features of mines, resulting in low efficiency in localization and mapping.
The raw laser point cloud of the open-pit mine is acquired by lidar, segmented and extracted, and the pose deviation is corrected by ground normal vector and projected onto a two-dimensional plane to construct a histogram matrix descriptor. The similarity is evaluated by the Hungarian matching algorithm to identify similar scenes.
It effectively avoids the impact of height errors caused by ground bumps, and the descriptor has rotation invariance, enabling accurate scene recognition at different times, thus improving the efficiency and accuracy of mine scene recognition.
Smart Images
Figure CN122336339A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, specifically to a method, apparatus, equipment, and storage medium for representing open-pit mine scenes. Background Technology
[0002] With the development of autonomous driving technology, more and more mines are choosing autonomous driving to complete tasks such as transportation, loading, and unloading. Localization and mapping are crucial components of the autonomous driving technology chain, and scene recognition is an important means to accelerate the efficiency of localization and mapping. Especially in mining scenarios, GPS signals are unstable, so descriptor matching and other scene recognition algorithms that do not rely on GPS are needed to perform coarse localization of vehicles, thereby improving the efficiency of accurate localization and map fusion.
[0003] Scene recognition involves matching and recalling locally perceived information within a global environment. However, the global environment often involves massive amounts of data and features. Matching based on specific features (such as points, lines, and surfaces) would consume significant search time. Therefore, scene information is typically represented using descriptors that occupy less space, usually in the form of one-dimensional or two-dimensional matrices. This search and matching method, based on single-frame scenes, greatly improves localization efficiency.
[0004] However, open-pit mines differ from urban environments, and commonly used descriptors struggle to accurately depict their characteristics. For instance, the complex three-dimensional shape of mine formations and the bumpy road surface mean that lidar height information becomes unstable due to changes in vehicle pose. The irregular geometric features of mine formations also result in a lack of effective and stable line and surface features, making feature-point-dependent descriptors difficult to use effectively. Descriptors that partially rely on neural networks lack scene robustness, and their generation and matching processes require significant computational power, making real-time operation difficult under the limited computing power of mining vehicles. Summary of the Invention
[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a method, apparatus, equipment and storage medium for characterizing open-pit mine scenes.
[0006] In a first aspect, embodiments of this application provide a method for characterizing an open-pit mine scene, including: The original laser point cloud of the open-pit mine is obtained by using lidar, and the point cloud space corresponding to the original laser point cloud is segmented and extracted to obtain the mountain point cloud data. The ground is coarsely fitted based on the mountain point cloud data to obtain the ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. After obtaining the boundary values of the two-dimensional mountain projection point cloud and determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, the sampling contour points of the mountain image are determined. A histogram matrix is constructed based on the sampled contour points of the mountain image, and the histogram matrix of all contour points is split into vectors and then concatenated to obtain the descriptor of the mountain projection. The similarity of the descriptors of the mountain projections was evaluated using the Hungarian matching algorithm, and the same open-pit mine scene was identified.
[0007] Optionally, the raw laser point cloud of the open-pit mine is acquired using lidar, and the point cloud space corresponding to the raw laser point cloud is segmented and extracted to obtain mountain point cloud data, including: The raw laser point cloud of the open-pit mine is obtained by using at least one lidar. The point cloud space corresponding to the original laser point cloud is segmented using a concentric cylinder model to obtain several voxels; The feature parameters of the voxels are extracted by principal component analysis and iterative methods, and the point cloud class probability is estimated by adaptive likelihood estimation. The semantic probability of point cloud is estimated by using multiple dimension evaluation functions, and the hyperparameters are adaptively updated based on historical data. The multiple dimension evaluation functions include at least one of the following: verticality evaluation function, height evaluation function, and flatness evaluation function. When the inclination of the lowest plane in a voxel exceeds the verticality requirement, the point cloud of the voxel is classified into the mountain category, and mountain point cloud data is obtained.
[0008] Optionally, a coarse fit is performed on the ground based on the mountain point cloud data to obtain the ground normal vector, including: Based on the mountain point cloud data, the largest plane in the scene is fitted as the ground using the RANSAC algorithm, and the ground normal vector is obtained.
[0009] Optionally, after obtaining the boundary values of the two-dimensional mountain projection point cloud and determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, the sampling contour points of the mountain image are determined, including: Obtain the boundary values of the two-dimensional mountain projection point cloud in the four directions of up, down, left, and right, and use the boundary values in the four directions as the image boundary; The relative distance between each two-dimensional point in the two-dimensional mountain projection point cloud and the image boundary is determined, and the relative distance is used as the pixel coordinates of the two-dimensional point cloud in the image; Erosion, dilation, and Gaussian blur operations are applied to the image to obtain a smoothed mountain image; The smoothed mountain image is subjected to Canny edge extraction to obtain the outer contour; The outer contour is sampled to obtain a fixed number of sampled contour points.
[0010] Optionally, a histogram matrix is constructed based on the sampled contour points of the mountain image, and the histogram matrix of all contour points is split into vectors and then concatenated to obtain a descriptor for the mountain projection, including: Traverse each point of the sampled contour points of the mountain image, take the traversed point as the target point, determine the extreme logarithmic coordinates of the remaining points relative to the target point, and correspond the extreme logarithmic coordinates to the pixel values at different positions in the histogram matrix; Determine the centroid positions of all mountain points within a preset range near the target point, and use the direction from the target point to the centroid as the reference direction of the histogram matrix, then split the histogram matrix of all contour points into vectors; The split vectors are concatenated to obtain a shape matrix; The shape matrix is used as a descriptor for the mountain projection.
[0011] Optionally, the similarity of the descriptors of the mountain projection is evaluated using the Hungarian matching algorithm to identify identical open-pit mine scenes, including: Iterate through the descriptors of any two mountain projections to obtain the shape matrix, where each row of the shape matrix represents the distribution of the mountain from different contour point perspectives; The similarity between contour points in each row of matrices of different shapes is evaluated by using the Hungarian matching algorithm to obtain the similarity between matrices of different shapes. Based on the similarity between the different shape matrices, the same open-pit mine scene is identified.
[0012] Optionally, the similarity of contour points in each row of different shape matrices is evaluated using the Hungarian matching algorithm to obtain the similarity between different shape matrices, including: The optimal correspondence between contour points in each row of different shape matrices is determined by the Hungarian matching algorithm, resulting in several point pairs. The degree of difference between two points in the point pair is determined by a similarity calculation formula; The sum of the differences between all point pairs is used as the similarity between matrices of different shapes.
[0013] Secondly, embodiments of this application provide an open-pit mine scene characterization device, comprising: The mountain point cloud extraction module is used to acquire the original laser point cloud of the open-pit mine through lidar, and to segment and extract the point cloud space corresponding to the original laser point cloud to obtain mountain point cloud data. The two-dimensional point cloud projection module is used to coarsely fit the ground based on the mountain point cloud data to obtain the ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. The two-dimensional image processing module is used to obtain the boundary values of the two-dimensional mountain projection point cloud, and after determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, determine the sampling contour points of the mountain image. The descriptor generation module is used to construct a histogram matrix based on the sampled contour points of the mountain image, and then split the histogram matrix of all contour points into vectors and then concatenate them to obtain the descriptor of the mountain projection. The descriptor matching module is used to evaluate the similarity of the descriptors of the mountain projection using the Hungarian matching algorithm, and to identify the same open-pit mine scene.
[0014] Thirdly, embodiments of this application provide an open-pit mine scene characterization device, comprising: a processor and a memory, wherein the memory stores executable program instructions, and when the processor invokes the program instructions in the memory, the processor is used to: Perform the steps of the open-pit mine scene characterization method as described in any one of the first aspects.
[0015] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a program, which, when executed, implements the steps of the open-pit mine scene characterization method as described in any one of the first aspects.
[0016] Compared with the prior art, the present invention has the following beneficial effects: This application acquires the original laser point cloud of an open-pit mine using lidar, and segments and extracts the corresponding point cloud space to obtain mountain point cloud data. Based on the mountain point cloud data, a coarse fit is performed on the ground to obtain a ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. The boundary values of the two-dimensional mountain projection point cloud are obtained, and the pixel coordinates of each two-dimensional point cloud in the image are determined based on the relative distance between each two-dimensional point cloud and the boundary, thus determining the sampling contour points of the mountain image. A histogram matrix is constructed based on the sampling contour points of the mountain image, and the histogram matrix of all contour points is split into vectors and then concatenated to obtain the descriptor of the mountain projection. The similarity of the mountain projection descriptor is evaluated using a Hungarian matching algorithm to identify similar open-pit mine scenes. This avoids the influence of height errors caused by ground bumps, and the projected mountain point cloud contains information about the shape and trend of the mountain, avoiding the use of geometric features to characterize the scene. In addition, the statistical histograms are based on the centroid direction of each point, which makes the descriptors rotationally invariant and can be used to identify scenes of mountain observation at different times. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A flowchart illustrating a method for characterizing an open-pit mine scene provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the principle of an open-pit mine scene characterization method provided in an embodiment of this application. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] It should be noted that when a component is said to be "fixed" to another component, it can be directly on the other component or it can be in a middle component. When a component is said to be "connected" to another component, it can be directly connected to the other component or it may be in a middle component.
[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0021] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0022] The technical solutions of the present invention and how they solve the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0023] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0024] Existing scene recognition algorithms for open-pit mines are easily affected by observed changes in the height of the mountain, leading to inaccurate descriptors. Furthermore, due to the irregular mining operations of excavators, the mountain exhibits weak geometric characteristics, making it difficult to construct descriptors based on geometric features in the scene. In addition, most descriptors lack rotation invariance, making it impossible to establish connections between descriptors when the mining truck is in different positions observing the scene.
[0025] To address the aforementioned issues, this application aims to provide a method for representing open-pit mine scenes. This method uses mountain point clouds obtained from lidar observations as a foundation. It employs a voxelization method with concentric cylindrical models for vertical partitioning to comprehensively characterize the mountain point cloud features, and extracts voxel feature parameters using principal component analysis iterative methods. An adaptive likelihood estimation method is designed to extract the mountain point cloud, and ground correction techniques are used to mitigate the impact of mine car bumps on the point cloud quality, transforming it into a mountain projection point cloud, which is then further used to generate a two-dimensional image. The outer contour of the mountain projection is obtained through a series of image operations, and random sampling is performed. For each sampled contour point, its centroid direction is calculated, and the distribution of other contour points is statistically analyzed based on this direction to form a histogram matrix. Expanding and concatenating the histogram matrices of all sampled contour points yields the descriptor of the mountain in the current frame. For descriptors from different frames, Hungarian matching is used to obtain a similarity score between them.
[0026] The above method effectively avoids the impact of height errors caused by ground bumps, and the projected mountain point cloud contains information about the shape and trend of the mountain, thus avoiding the use of geometric features to characterize the scene. Furthermore, by using the centroid direction of each point as a reference in the statistical histogram, the descriptor in this embodiment possesses rotation invariance, enabling scene recognition of mountain observations at different times.
[0027] For example, Figure 1 This is a flowchart illustrating a method for characterizing an open-pit mine scene, as provided in an embodiment of this application. Figure 1 As shown, the method in this embodiment may include the following steps: Step S101: Obtain the original laser point cloud of the open-pit mine using lidar, and segment and extract the point cloud space corresponding to the original laser point cloud to obtain the mountain point cloud data.
[0028] In this embodiment, it is applied to the field of autonomous driving in open-pit mines, such as... Figure 2As shown, the process first acquires the original 3D mountain point cloud (i.e., the original laser point cloud collected from the open-pit mine using a vehicle-mounted LiDAR). Then, a 2D point cloud projection module extracts ground normal vectors, segments the mountain point cloud, and projects it along the normal to obtain a 2D mountain projection point cloud. Further, an image processing module generates a mountain image, which is then smoothed using erosion dilation and Gaussian blurring. Canny edge extraction and contour point sampling are then used to obtain sparse contour points. Further, a descriptor generation module calculates the extreme logarithmic coordinates and centroid positions of the contour points and generates a histogram matrix. This histogram matrix is then vector-split and concatenated to obtain a shape matrix (i.e., the descriptor of the mountain projection). Finally, a descriptor matching module calculates the difference between two points in different shape matrices, uses a Hungarian matching algorithm to find the optimal point pair, and uses the sum of the differences between all point pairs as the similarity between the two shape matrices. Therefore, the similarity between two shape matrices can be used to determine whether they represent the same scene.
[0029] For example, the original laser point cloud of an open-pit mine is acquired using at least one lidar; the point cloud space corresponding to the original laser point cloud is segmented using a concentric cylinder model to obtain several voxels; the feature parameters of the voxels are extracted using principal component analysis and iterative methods, and the probability of the point cloud category is estimated using an adaptive likelihood estimation method; the semantic probability of the point cloud is estimated using multiple dimensional evaluation functions, and the hyperparameters are adaptively updated based on historical data, wherein the multiple dimensional evaluation functions include at least one of: verticality evaluation function, height evaluation function, and flatness evaluation function; when the tilt of the lowest plane in the voxel exceeds the verticality requirement, the point cloud of the voxel is classified into the mountain category to obtain mountain point cloud data.
[0030] In this embodiment, the original laser point cloud is first divided into multiple sub-regions using a voxel segmentation based on a non-uniform width concentric cylinder model. Furthermore, by adding vertical segmentation within voxels, the mountains within a single sector can be represented by a combination of multiple voxel features, preserving more original information. Based on this, feature parameters are estimated for the voxel point cloud using the biplane assumption. To improve the robustness of point cloud type identification, an adaptive likelihood estimation method is used to estimate the probability of point cloud categories. The semantic probability of the point cloud is estimated using multi-dimensional evaluation functions such as verticality, height, and flatness. Hyperparameters are adaptively updated based on historical data. When the tilt of the lowest plane in a voxel exceeds the verticality requirement, the point cloud in that region is classified as a mountain. The algorithm is then applied to all original point clouds to obtain the mountain point cloud. .
[0031] Step S102: Based on the mountain point cloud data, perform coarse fitting on the ground to obtain the ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, project the mountain point cloud data onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud.
[0032] For example, the largest plane in the scene can be fitted as the ground using the RANSAC algorithm based on the mountain point cloud data, and the ground normal vector can be obtained.
[0033] In this embodiment, for the mountain point cloud data, the RANSAC algorithm is first used to fit the largest plane in the scene to obtain the ground normal. Since the slope of the road surface near the mine cart does not change significantly between two adjacent point cloud frames, it is assumed that the first frame... Frame and the Frames have the same global ground normal vector, that is This corrects for changes in vehicle pose caused by the bumps of the mine car, and the point cloud is aligned with the corrected ground normal vector. Two-dimensional point clouds can be obtained by projection. .
[0034] Step S103: Obtain the boundary values of the two-dimensional mountain projection point cloud, and determine the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, and then determine the sampling contour points of the mountain image.
[0035] For example, the boundary values of the two-dimensional mountain projection point cloud in the four directions of up, down, left, and right are obtained, and the boundary values in the four directions are used as the image boundary; the relative distance between each two-dimensional point in the two-dimensional mountain projection point cloud and the image boundary is determined, and the relative distance is used as the pixel coordinates of the two-dimensional point cloud in the image; erosion, dilation, and Gaussian blur operations are performed on the image to obtain a smoothed mountain image; Canny edge extraction is performed on the smoothed mountain image to obtain the outer contour; the outer contour is sampled to obtain a fixed number of sampled contour points.
[0036] In this embodiment, the two-dimensional point cloud is traversed to obtain the four boundary values: top, bottom, left, and right. Using the bottom left corner as the origin, calculate the local coordinates of all points in this coordinate system. The corresponding pixel coordinates in the image are used to obtain the transformed two-dimensional image. The image was subjected to operations such as erosion dilation and Gaussian blur to remove point cloud noise, and the Canny algorithm was used to extract the mountain contours. To reduce the memory footprint of the descriptor and improve the running speed, the mountain contour points were downsampled to obtain sparse contour points.
[0037] Step S104: Construct a histogram matrix based on the sampled contour points of the mountain image, and then split the histogram matrix of all contour points into vectors and then concatenate them to obtain the descriptor of the mountain projection.
[0038] For example, each point of the sampled contour points of the mountain image is traversed, and the traversed point is taken as the target point. The extreme logarithmic coordinates of the remaining points relative to the target point are determined, and the extreme logarithmic coordinates are mapped to the pixel values at different positions in the histogram matrix. The centroid positions of all mountain points within a preset range near the target point are determined, and the direction from the target point to the centroid is taken as the reference direction of the histogram matrix. The histogram matrix of all contour points is split into vectors. The split vectors are concatenated to obtain a shape matrix. The shape matrix is used as the descriptor of the mountain projection.
[0039] In this embodiment, the distribution of all other contour points is analyzed for each contour point as a feature, and the output is a two-dimensional shape matrix of the scene, i.e., a descriptor. First, the centroid of the mountain points near the target point is calculated, with the line connecting the centroid and the target point as the reference axis. Second, the extreme logarithmic coordinates of the remaining points relative to the target point (relative to the reference axis) are calculated and normalized to obtain the number of points falling in different regions near and far from the target point, and a histogram matrix is generated. The more points there are, the larger the value at the corresponding position in the matrix. Perform the above operation on all sampled contour points, and decompose and reconstruct each histogram matrix into histogram vectors. By concatenating the histogram vectors of different points, the final two-dimensional shape matrix of the scene is obtained. .
[0040] Step S105: The similarity of the descriptors of the mountain projection is evaluated by the Hungarian matching algorithm to identify the same open-pit mine scene.
[0041] For example, the shape matrix corresponding to any two descriptors of the mountain projection is obtained by traversal, and each row of the shape matrix represents the distribution of the mountain from different contour point perspectives; the similarity of the contour points in each row of different shape matrices is evaluated by the Hungarian matching algorithm to obtain the similarity between different shape matrices; based on the similarity between the different shape matrices, the same open-pit mine scene is identified.
[0042] Specifically, the optimal correspondence between contour points in each row of different shape matrices is determined by the Hungarian matching algorithm, resulting in several point pairs; the difference between two points in the point pair is determined by the similarity calculation formula; and the sum of the differences of all point pairs is taken as the similarity between different shape matrices.
[0043] In this embodiment, the shape matrix Composed of features from different contour points, we first compare the feature differences between different points: ,in and Let S represent the element values at the k-th row of two points in matrices of different shapes, where K is the total number of rows, and S... i Let be the similarity of the i-th point pair. By iterating through all point pairs and using the Hungarian matching algorithm, the optimal point pair association between two shape matrices can be obtained. And the corresponding difference scores, so by summing the difference scores of all point pairs in the optimal matching case, we can obtain the similarity of the final shape matrix. .
[0044] In this embodiment, the original laser point cloud of the open-pit mine is acquired using LiDAR, and the corresponding point cloud space is segmented and extracted to obtain mountain point cloud data. The ground is coarsely fitted based on the mountain point cloud data to obtain a ground normal vector. After correcting pose deviations caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. The boundary values of the two-dimensional mountain projection point cloud are obtained, and the pixel coordinates of each two-dimensional point cloud in the image are determined based on the relative distance between each two-dimensional point cloud and the boundary, thus determining the sampling contour points of the mountain image. A histogram matrix is constructed based on the sampling contour points of the mountain image, and the histogram matrix of all contour points is split into vectors and then concatenated to obtain the descriptor of the mountain projection. The similarity of the mountain projection descriptor is evaluated using the Hungarian matching algorithm to identify similar open-pit mine scenes. This avoids the influence of height errors caused by ground bumps, and the projected mountain point cloud contains information about the shape and trend of the mountain, avoiding the use of geometric features to characterize the scene. In addition, the statistical histograms are based on the centroid direction of each point, which makes the descriptors rotationally invariant and can be used to identify scenes of mountain observation at different times.
[0045] This application embodiment also provides an open-pit mine scene characterization device, which may include: a mountain point cloud extraction module, used to acquire the original laser point cloud of the open-pit mine through lidar, and to segment and extract the point cloud space corresponding to the original laser point cloud to obtain mountain point cloud data; a two-dimensional point cloud projection module, used to perform coarse fitting on the ground based on the mountain point cloud data to obtain the ground normal vector, and after correcting the pose deviation caused by road bumps through the ground normal vector, to project the mountain point cloud data onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud; A two-dimensional image processing module is used to obtain the boundary values of the two-dimensional mountain projection point cloud, and determine the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, and then determine the sampling contour points of the mountain image; a descriptor generation module is used to construct a histogram matrix based on the sampling contour points of the mountain image, and then split the histogram matrix of all contour points into vectors and then concatenate them to obtain the descriptor of the mountain projection; a descriptor matching module is used to evaluate the similarity of the descriptors of the mountain projection using the Hungarian matching algorithm to identify the same open-pit mine scene.
[0046] For example, the mountain point cloud extraction module is specifically used for: acquiring the original laser point cloud of an open-pit mine using at least one lidar; segmenting the point cloud space corresponding to the original laser point cloud using a concentric cylinder model to obtain several voxels; extracting the feature parameters of the voxels using principal component analysis and iterative methods, and estimating the point cloud category probability using an adaptive likelihood estimation method; estimating the semantic probability of the point cloud using multiple-dimensional evaluation functions, and adaptively updating the hyperparameters based on historical data, wherein the multiple-dimensional evaluation functions include at least one of: verticality evaluation function, height evaluation function, and flatness evaluation function; when the tilt of the lowest plane in the voxel exceeds the verticality requirement, classifying the point cloud of the voxel into a mountain category to obtain mountain point cloud data.
[0047] For example, the two-dimensional point cloud projection module is specifically used to: fit the largest plane in the scene as the ground using the RANSAC algorithm based on the mountain point cloud data, and obtain the ground normal vector.
[0048] For example, the two-dimensional image processing module is specifically used for: obtaining the boundary values of the two-dimensional mountain projection point cloud in the four directions of up, down, left, and right, and using the boundary values in the four directions as the image boundary; determining the relative distance between each two-dimensional point in the two-dimensional mountain projection point cloud and the image boundary, and using the relative distance as the pixel coordinates of the two-dimensional point cloud in the image; performing erosion, dilation, and Gaussian blur operations on the image to obtain a smoothed mountain image; performing Canny edge extraction on the smoothed mountain image to obtain the outer contour; and sampling the outer contour to obtain a fixed number of sampled contour points.
[0049] For example, the descriptor generation module is specifically used for: traversing each point of the sampled contour points of the mountain image, taking the traversed point as the target point, determining the extreme logarithmic coordinates of the remaining points relative to the target point, and corresponding the extreme logarithmic coordinates to the pixel values at different positions in the histogram matrix; determining the centroid positions of all mountain points within a preset range near the target point, taking the direction from the target point to the centroid as the reference direction of the histogram matrix, and splitting the histogram matrix of all contour points into vectors; concatenating the split vectors to obtain a shape matrix; and using the shape matrix as the descriptor of the mountain projection.
[0050] For example, the descriptor matching module is specifically used to: traverse and obtain the shape matrix corresponding to any two descriptors of the mountain projection, where each row of the shape matrix represents the distribution of the mountain from different contour point perspectives; evaluate the similarity of the contour points in each row of different shape matrices using the Hungarian matching algorithm to obtain the similarity between different shape matrices; and identify the same open-pit mine scene based on the similarity between the different shape matrices.
[0051] Specifically, the similarity between contour points in each row of different shape matrices is evaluated using the Hungarian matching algorithm to obtain the similarity between different shape matrices. This includes: determining the optimal correspondence between contour points in each row of different shape matrices using the Hungarian matching algorithm to obtain several point pairs; determining the difference between two points in the point pair using the similarity calculation formula; and using the sum of the differences of all point pairs as the similarity between different shape matrices.
[0052] Furthermore, this application embodiment also provides an open-pit mine scene characterization device, including: a processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to: execute the steps of the open-pit mine scene characterization method in the above method embodiment.
[0053] It should be noted that those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "platform."
[0054] Furthermore, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions. When at least one processor of a user device executes these computer-executable instructions, the user device performs the various possible methods described above. The computer-readable medium includes a computer storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transfer of a computer program from one location to another. The storage medium can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in the user device. Alternatively, the processor and storage medium can exist as discrete components in a communication device.
[0055] This application also provides a program product including a computer program stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the server to implement any of the methods described in the embodiments of the present invention.
[0056] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0057] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A method for representing an open-pit mine scene, characterized in that, include: The original laser point cloud of the open-pit mine is obtained by using lidar, and the point cloud space corresponding to the original laser point cloud is segmented and extracted to obtain the mountain point cloud data. The ground is coarsely fitted based on the mountain point cloud data to obtain the ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. After obtaining the boundary values of the two-dimensional mountain projection point cloud and determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, the sampling contour points of the mountain image are determined. A histogram matrix is constructed based on the sampled contour points of the mountain image, and the histogram matrix of all contour points is split into vectors and then concatenated to obtain the descriptor of the mountain projection. The similarity of the descriptors of the mountain projections was evaluated using the Hungarian matching algorithm, and the same open-pit mine scene was identified.
2. The method for representing open-pit mine scenes according to claim 1, characterized in that, The raw laser point cloud of the open-pit mine is acquired using lidar, and the corresponding point cloud space is segmented and extracted to obtain the mountain point cloud data, including: The raw laser point cloud of the open-pit mine is obtained by using at least one lidar. The point cloud space corresponding to the original laser point cloud is segmented using a concentric cylinder model to obtain several voxels; The feature parameters of the voxels are extracted by principal component analysis and iterative methods, and the point cloud class probability is estimated by adaptive likelihood estimation. The semantic probability of point cloud is estimated by using multiple dimension evaluation functions, and the hyperparameters are adaptively updated based on historical data. The multiple dimension evaluation functions include at least one of the following: verticality evaluation function, height evaluation function, and flatness evaluation function. When the inclination of the lowest plane in a voxel exceeds the verticality requirement, the point cloud of the voxel is classified into the mountain category, and mountain point cloud data is obtained.
3. The method for representing open-pit mine scenes according to claim 1, characterized in that, Based on the mountain point cloud data, a coarse fit is performed on the ground to obtain the ground normal vector, including: Based on the mountain point cloud data, the largest plane in the scene is fitted as the ground using the RANSAC algorithm, and the ground normal vector is obtained.
4. The method for representing open-pit mine scenes according to claim 1, characterized in that, After obtaining the boundary values of the two-dimensional mountain projection point cloud and determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, the sampling contour points of the mountain image are determined, including: Obtain the boundary values of the two-dimensional mountain projection point cloud in the four directions of up, down, left, and right, and use the boundary values in the four directions as the image boundary; The relative distance between each two-dimensional point in the two-dimensional mountain projection point cloud and the image boundary is determined, and the relative distance is used as the pixel coordinates of the two-dimensional point cloud in the image; Erosion, dilation, and Gaussian blur operations are applied to the image to obtain a smoothed mountain image; The smoothed mountain image is subjected to Canny edge extraction to obtain the outer contour; The outer contour is sampled to obtain a fixed number of sampled contour points.
5. The method for representing an open-pit mine scene according to claim 1, characterized in that, A histogram matrix is constructed based on the sampled contour points of the mountain image. The histogram matrix of all contour points is then split into vectors and concatenated to obtain the descriptor of the mountain projection, including: Traverse each point of the sampled contour points of the mountain image, take the traversed point as the target point, determine the extreme logarithmic coordinates of the remaining points relative to the target point, and correspond the extreme logarithmic coordinates to the pixel values at different positions in the histogram matrix; Determine the centroid positions of all mountain points within a preset range near the target point, and use the direction from the target point to the centroid as the reference direction of the histogram matrix, then split the histogram matrix of all contour points into vectors; The split vectors are concatenated to obtain a shape matrix; The shape matrix is used as a descriptor for the mountain projection.
6. The method for representing an open-pit mine scene according to claim 5, characterized in that, The similarity of the descriptors of the mountain projections was evaluated using the Hungarian matching algorithm, identifying identical open-pit mine scenes, including: Iterate through the descriptors of any two mountain projections to obtain the shape matrix, where each row of the shape matrix represents the distribution of the mountain from different contour point perspectives; The similarity between contour points in each row of matrices of different shapes is evaluated by using the Hungarian matching algorithm to obtain the similarity between matrices of different shapes. Based on the similarity between the different shape matrices, the same open-pit mine scene is identified.
7. The method for representing an open-pit mine scene according to claim 6, characterized in that, The similarity between contour points in each row of matrices of different shapes is evaluated using the Hungarian matching algorithm, yielding the similarity between matrices of different shapes, including: The optimal correspondence between contour points in each row of different shape matrices is determined by the Hungarian matching algorithm, resulting in several point pairs. The degree of difference between two points in the point pair is determined by a similarity calculation formula; The sum of the differences between all point pairs is used as the similarity between matrices of different shapes.
8. A device for representing an open-pit mine scene, characterized in that, include: The mountain point cloud extraction module is used to acquire the original laser point cloud of the open-pit mine through lidar, and to segment and extract the point cloud space corresponding to the original laser point cloud to obtain mountain point cloud data. The two-dimensional point cloud projection module is used to coarsely fit the ground based on the mountain point cloud data to obtain the ground normal vector. After correcting the pose deviation caused by road bumps using the ground normal vector, the mountain point cloud data is projected onto a two-dimensional plane along the ground normal vector to obtain a two-dimensional mountain projection point cloud. The two-dimensional image processing module is used to obtain the boundary values of the two-dimensional mountain projection point cloud, and after determining the pixel coordinates of the two-dimensional point cloud in the image based on the relative distance between each two-dimensional point cloud and the boundary, determine the sampling contour points of the mountain image. The descriptor generation module is used to construct a histogram matrix based on the sampled contour points of the mountain image, and then split the histogram matrix of all contour points into vectors and then concatenate them to obtain the descriptor of the mountain projection. The descriptor matching module is used to evaluate the similarity of the descriptors of the mountain projection using the Hungarian matching algorithm, and to identify the same open-pit mine scene.
9. An open-pit mine scene representation device, characterized in that, include: A processor and a memory, wherein the memory stores executable program instructions, and when the processor invokes the program instructions in the memory, the processor is used to: The steps of performing the open-pit mine scene characterization method according to any one of claims 1 to 7.
10. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, it implements the steps of the open-pit mine scene representation method according to any one of claims 1 to 7.