Ore stockpile volume measurement method based on binocular stereoscopic vision

Abstract

The invention relates to the technical field of ore stockpile volume measurement, in particular to an ore stockpile volume measurement method based on binocular stereoscopic vision, which comprises the following steps of: acquiring left and right views of an ore stockpile by using a binocular camera, and constructing a data set of stereoscopic matching data; the method comprises the following steps: firstly, correcting collected left and right views by adopting an improved multi-scale adaptive Bouguet stereo correction algorithm to eliminate lens distortion and align row pixels of the left and right views to obtain a stereo correction image pair; carrying out graying processing on the stereo correction image pair by adopting a weighted average method; and finally, carrying out contrast-limited adaptive histogram equalization on the image subjected to graying processing by the weighted average method so as to highlight the edge and texture information of the ore stockpile. An MS-BGNet stereo matching model based on fusion transfer learning is constructed; inputting ore stockpiling left and right views to the model to generate a disparity map; and based on the generated disparity map, calculating the volume of the ore pile by using a triangular prism infinitesimal splitting method.

Description

Technical Field

[0001] This invention relates to the field of ore stockpile volume measurement technology, specifically to a method for measuring ore stockpile volume based on binocular stereo vision. Background Technology

[0002] In the operation and management of mining enterprises, the volume measurement of ore stockpiles is a key link in cost accounting, benefit assessment and production planning. Its measurement accuracy and efficiency directly affect the economic benefits and management level of the enterprise.

[0003] Traditional methods for measuring stockpile volume primarily employ contact-based approaches (such as manual measurement with a measuring tape). These methods are not only time-consuming, labor-intensive, and costly, but also pose significant safety hazards in harsh working environments such as high dust levels and high temperatures. Furthermore, they are prone to large human measurement errors. Therefore, non-contact measurement methods have emerged and become the mainstream trend. Among these, vision-based measurement methods are widely used due to their high safety and remote operation capabilities. Vision measurement methods mainly include monocular, binocular, and multi-view solutions: Monocular vision methods can only acquire two-dimensional image information and cannot directly obtain depth data. When measuring large stockpiles, it is difficult to cover the entire object at once, leading to significant measurement errors. While multi-view vision methods can improve accuracy, they are expensive in hardware and involve complex multi-camera calibration, synchronization, and data fusion issues, significantly increasing implementation difficulty and cost. Binocular vision methods achieve a good balance between cost, accuracy, and technological maturity, and are considered an ideal solution for achieving automated stockpile volume measurement.

[0004] However, existing binocular stereo vision methods still face challenges when applied to the specific scenario of ore stockpiles. The surface texture of ore stockpiles is weak and features are sparse, and the ambient lighting is complex, resulting in disparity maps that often contain numerous holes and distortions, severely impacting the integrity of subsequent 3D reconstruction and the accuracy of volume calculation. Therefore, researching a binocular vision measurement method capable of accurately generating disparity maps and achieving high-precision volume calculation based on them has significant practical implications and application value. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for measuring the volume of ore stockpiles based on binocular stereo vision.

[0006] To achieve the above objectives, the following technical solution is provided:

[0007] A method for measuring the volume of ore stockpiles based on binocular stereo vision, characterized by comprising the following steps:

[0008] Step 1: Use a binocular camera to collect left and right views of the ore stockpile and construct a dataset of stereo matching data, including 300 pairs of stereo matching data for large-particle ore stockpile and 300 pairs of stereo matching data for small-particle ore stockpile.

[0009] Step 2: First, the left and right views acquired in Step 1 are corrected using the improved multi-scale adaptive Bouguet stereo correction algorithm to eliminate lens distortion and align the row pixels of the left and right views to obtain stereo-corrected image pairs. Then, the stereo-corrected image pairs are processed into grayscale using a weighted average method. Finally, the images processed into grayscale using the weighted average method are subjected to adaptive histogram equalization with limited contrast to highlight the edges and texture information of the ore pile.

[0010] Step 3: Construct the MS-BGNet stereo matching model based on fusion transfer learning; The training phase of the MS-BGNet stereo matching model adopts a two-stage training strategy: First, pre-training is performed using an open-source large-scale stereo matching dataset. Then, the stereo matching network with the obtained pre-trained weights is fine-tuned using a self-built ore stockpile dataset. After fine-tuning, the left and right views of the ore stockpile obtained in Step 2 are input into the model to generate a disparity map.

[0011] Step 4: Based on the disparity map generated in Step 3, calculate the ore stockpile volume using the triangular prism micro-element decomposition method.

[0012] Preferably, in step two, the improved multi-scale adaptive Bouguet stereo correction algorithm includes the following steps:

[0013] Step 1: After completing the basic Bouguet stereo correction, in order to achieve a balance between large-scale distortion and local details, a multi-resolution fusion correction module is introduced to decompose the original image at multiple scales, applying stereo correction at multiple resolution levels. The formula for its pyramid construction process can be expressed as:

[0014] , ;

[0015] In the formula, I l (k) I r (k) These are the left and right eye images at the k-th level of the pyramid, respectively. PyrDown() is the pyramid downsampling operation. l (k-1) I r (k-1) These are the left and right images of the pyramid on the next level, respectively, where k is the pyramid level number.

[0016] Step 2: After completing local corrections at each scale, the correction results at different resolutions are fused to obtain a final corrected image that possesses both global consistency and local detail preservation. The fusion process can be described as follows:

[0017] ;

[0018] ;

[0019] In the formula, This is the final fused and corrected image. PyrUp() is the pyramid upsampling operation. k Let be the weight of the k-th layer scale in the final fusion;

[0020] Step 3: After completing multi-scale correction and fusion, an adaptive ROI module is introduced. By analyzing the effective regions of the left and right corrected images, the optimal cropping window is automatically calculated. First, the non-zero pixel regions of the left and right images are detected, and the intersection is calculated as the effective ROI. The formula is as follows:

[0021] ;

[0022] In the formula, (x,y) are the pixel coordinates in the image, I l (x,y), I r (x, y) represent the pixel intensities at position (x, y) in the left and right corrected images, respectively, and τ is the pixel intensity threshold. The logical symbol "AND";

[0023] Step 4: Based on the effective ROI region, calculate the cropping boundary that maximizes the preservation of the field of view, ensuring consistency of the field of view between the left and right images, while minimizing the invalid black area. The formula is:

[0024] ;

[0025] In the formula, x min y min This represents the coordinates of the top-left corner of the final cropped region (ROI), x. max y max This represents the coordinates of the top right corner of the final cropped region (ROI). , This represents the coordinates of the top-left corner of the valid area of ​​the left image. , This represents the coordinates of the bottom right corner of the valid area of ​​the left image. , This represents the coordinates of the top-left corner of the valid area of ​​the right image. , This represents the coordinates of the lower right corner of the valid area of ​​the right image.

[0026] Finally, through multi-scale and adaptive ROI cropping, high-quality stereo-corrected image pairs with maximized field of view are obtained.

[0027] Preferably, in step two, the weighted average method for grayscale conversion involves performing an appropriate weighted average operation on the pixel values ​​of each RGB channel, transforming the original RGB three-channel image containing rich color information into a single-channel image with only one-dimensional grayscale information. The formula is as follows:

[0028] ;

[0029] In the formula, Gyay represents the gray value; i and j represent the number of rows and columns, respectively, and (i, j) represents the pixel coordinates in the two-dimensional image; R represents the red channel; G represents the green channel; B represents the blue channel; and the three weights 0.299, 0.578 and 0.114 correspond to the 4:2:0 YUV format luma values.

[0030] Preferably, in step two, the adaptive histogram equalization processing for limiting contrast specifically includes the following steps:

[0031] Step 1, Image segmentation and local histogram calculation: Divide the grayscale image into non-overlapping rectangular context regions and calculate the grayscale histogram for each region;

[0032] Step 2, Contrast Limiting and Histogram Cropping: Set a cropping threshold and crop each local histogram to limit its amplitude. The formula is:

[0033] ;

[0034] In the formula, T clip This represents the total number of pixels within the context region. N is the preset clipping factor. pixels N represents the number of gray levels. bins This indicates the number of gray levels in the histogram.

[0035] Will exceed T clip The number of pixels is evenly redistributed across all gray levels;

[0036] Step 3, Local Histogram Equalization: Based on the clipped and redistributed local histograms, calculate the cumulative distribution function for each context region, and apply the function to perform histogram equalization transformation on the region.

[0037] Step 4, Bilinear Interpolation Synthesis: To avoid block artifacts, the final gray value of any pixel in the image is obtained by bilinear interpolation through the transformation functions of its four adjacent context regions.

[0038] Preferably, in step three, the MS-BGNet stereo matching model based on fusion transfer learning includes a feature extraction module, a cost volume construction and aggregation module, a cost volume upsampling module based on bilateral grids, a multi-scale feature-guided cost volume enhancement module, and a residual disparity refinement module; wherein,

[0039] The feature extraction module is used to extract multi-scale features of the input stereo image pair using a ResNet-like architecture, and outputs feature pyramids containing three resolution levels: 1 / 2, 1 / 4, and 1 / 8.

[0040] The cost volume construction and aggregation module is used to construct grouped associated cost volumes based on the multi-scale features, wherein the inter-group correlation is calculated using the following formula:

[0041] ;

[0042] In the formula, C g N represents the matching cost of the g-th group. g G represents the total number of feature channels, and G represents the number of groups. and are the feature vectors of the left and right images in the g-th group, respectively;

[0043] The cost volume upsampling module based on bilateral grids is used to upsample the low-resolution cost volume to a high resolution through a slicing operation, wherein the slicing operation is defined as follows:

[0044] ;

[0045] In the formula, For a bilateral network, This refers to the ratio of the width or height of the grid dimension to the high-resolution cost volume dimension. The ratio of the gray levels of the grid to the gray levels of the guide map G;

[0046] The multi-scale feature-guided cost volume enhancement module is used to fuse 1 / 2, 1 / 4, and 1 / 8 multi-scale features from the feature extraction module, and enhances the cost volume through a spatial-parallax dual attention mechanism, wherein the spatial attention weight is calculated as follows:

[0047] ;

[0048] In the formula, W is the sigmoid function. S For convolution weights, Indicates feature splicing, This means concatenating feature maps at three scales (1 / 2, 1 / 4, and 1 / 8) along the channel dimension.

[0049] The residual disparity refinement module is used to regress the final disparity map from the enhanced high-resolution cost volume using the soft argmin function, and its calculation is defined as follows:

[0050] ;

[0051] In the formula, D pred (x,y) represents the final predicted disparity value, and d is the disparity depth, D max For the maximum parallax depth, C enhanced (x,y,d) represents the enhanced cost volume;

[0052] And optimization is performed using a smooth L1 loss function, which is defined as:

[0053] .

[0054] Preferably, in step three, the two-stage training strategy is as follows: First, the MS-BGNet stereo matching model is pre-trained using the open-source large-scale stereo matching dataset Scene Flow and the KITTI dataset to learn general stereo matching prior knowledge; then, the stereo matching network with the obtained pre-trained weights is fine-tuned using a self-built ore pile dataset to optimize the model's ability to perceive the surface texture and geometric features of the ore pile, thereby improving its matching accuracy and robustness in specific application scenarios.

[0055] Preferably, in step four, the specific steps for calculating the ore stockpile volume using the triangular prism micro-element decomposition method based on the disparity map generated in step three include the following:

[0056] Step 1: Based on the disparity map generated by the MS-BGNet stereo matching model, the two-dimensional image points are mapped to three-dimensional space through coordinate transformation, and each pixel... and its disparity value Can be converted to three-dimensional coordinates, disparity values The calculation formula is:

[0057] ;

[0058] In the formula, x l and x r These are the distances from the imaging points in the left and right imaging planes to the left edge of the plane, respectively.

[0059] Three-dimensional coordinates (X) w Y w Z w The calculation formula is:

[0060] ;

[0061] In the formula, b is the baseline of the binocular camera, f is the focal length of the camera, d is the parallax value, (x,y) is the coordinate in the imaging plane coordinate system, (u,v) is the coordinate in the pixel coordinate system, and α and β are the focal distances of the pixels in the x-axis and y-axis directions of the imaging plane.

[0062] Step 2: Construct triangular prism micro-elements based on the three-dimensional coordinates obtained above. Each triangular prism micro-element consists of three-dimensional spatial points corresponding to three adjacent pixels. Each pixel corresponds to an upper vertex located on the surface of the material and a lower vertex located on the reference plane, thus forming an irregular triangular prism with six vertices.

[0063] Step 3: For each triangular prism element, divide it into six irregular tetrahedrons; calculate the center point G of the triangular prism and connect the center point G to the six vertices of the triangular prism to form the six tetrahedrons; wherein, the three-dimensional coordinates of the center point G are calculated using the following formula:

[0064]

[0065] Among them, (X) n1 , Y n1 Z n1 ), (X n2 , Y n2 Z n2 ), (X n3 , Y n3 Z n3 Let (X) be the three-dimensional coordinates of the three vertices of the lower base of the triangular prism. m1 , Y m1 Z m1 ), (X m2 , Y m2 Z m2 ), (X m3 , Y m3 Z m3 ) represents the three-dimensional coordinates of the three vertices of the upper base of the triangular prism;

[0066] Step 4: Calculate the volume of each tetrahedron using the determinant method. For a tetrahedron with four vertices P1(x1,y1,z1),

[0067] The volume V of the tetrahedron formed by P2(x2, y2, z2), P3(x3, y3, z3), and P4(x4, y4, z4) is calculated using the following formula:

[0068] ;

[0069] Step 5: Add the volumes of the six tetrahedrons together to obtain the volume of a single triangular prism element;

[0070] Step 6: Sum the volumes of all triangular prism elements to obtain the total volume of the ore stockpile.

[0071] The beneficial effects of this invention are as follows:

[0072] 1. The improved multi-scale adaptive Bouguet stereo correction algorithm used in this invention has the advantage of maximizing the preservation of the effective field of view while aligning pixel rows through multi-resolution fusion and adaptive ROI cropping, providing a high-quality, distortion-free image foundation for stereo matching.

[0073] 2. To address the challenges of weak surface texture and sparse features in ore stockpiles, traditional matching algorithms generate disparity maps with severe holes and distortion. Therefore, this invention constructs the MS-BGNet stereo matching model based on fusion transfer learning. This model utilizes multi-scale features and an attention mechanism to generate high-quality disparity maps with rich detail and accurate matching. However, even with accurate 3D point clouds, traditional volumetric micro-element models struggle to accurately capture the complex surface undulations of the stockpile. Therefore, a triangular prism micro-element decomposition method is proposed. By decomposing each micro-element into six tetrahedrons for fine integration, the approximation error in volume calculation is significantly reduced compared to traditional methods. Attached Figure Description

[0074] Figure 1 This is a flowchart of the ore stockpile volume measurement based on binocular stereo vision provided by the present invention;

[0075] Figure 2 This is a structural diagram of the MS-BGNet model provided by the present invention;

[0076] Figure 3 This is a schematic diagram of the dissection of the irregular triangular prism provided by the present invention. Detailed Implementation

[0077] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0078] Example 1

[0079] A method for measuring the volume of ore stockpiles based on binocular stereo vision. Figure 1 As shown, it includes the following steps:

[0080] Step 1: Use a binocular camera to collect left and right views of the ore stockpile and construct a dataset of stereo matching data, including 300 pairs of stereo matching data for large-particle ore stockpile and 300 pairs of stereo matching data for small-particle ore stockpile.

[0081] Step 2: First, the improved multi-scale adaptive Bouguet stereo correction algorithm is used to correct the left and right views of the acquired images to eliminate lens distortion and align the row pixels of the left and right views. Then, a weighted average method is used for grayscale processing and adaptive histogram equalization with limited contrast to highlight the edge and texture information of the ore pile.

[0082] Step 3: Construct an MS-BGNet stereo matching model based on fusion transfer learning. First, pre-train it using an open-source large-scale stereo matching dataset. Then, fine-tune the stereo matching network with the obtained pre-trained weights using a self-built ore stockpile dataset. After fine-tuning, input the left and right views of the ore stockpile into the model to generate a disparity map.

[0083] Step 4: Based on the disparity map, calculate the volume of the ore stockpile using the triangular prism micro-element decomposition method. Divide the three-dimensional space into dense triangular prism micro-elements, and decompose each triangular prism micro-element into six irregular tetrahedrons. Calculate the volume of each tetrahedron separately and then sum them to obtain the volume of the triangular prism micro-element. Finally, sum all the micro-elements to obtain the total volume.

[0084] In step 2, stereo correction employs an improved multi-scale adaptive Bouguet stereo correction algorithm. After completing the basic Bouguet correction, to achieve a balance between large-scale distortion and local details, a multi-resolution fusion correction module is introduced to decompose the original image at multiple scales, applying stereo correction at multiple resolution levels. The formula for its pyramid construction process can be expressed as:

[0085] ,

[0086] In the formula, I l (k) I r (k) These are the left and right eye images at the k-th level of the pyramid, respectively. PyrDown() is the pyramid downsampling operation. l (k-1) I r (k-1) These are the left and right images of the pyramid on the next level, respectively, where k is the pyramid level number.

[0087] The correction transformation parameters are calculated separately at different scales, and the multi-scale correction results are fused through pyramid reconstruction to obtain a corrected image with better detail preservation. The formula is as follows:

[0088] ;

[0089] ;

[0090] In the formula, This is the final fused and corrected image. PyrUp() is the pyramid upsampling operation. k denoted as the weight of the k-th layer scale in the final fusion.

[0091] Furthermore, an adaptive ROI module is introduced, which automatically calculates the optimal cropping window by analyzing the effective regions of the left and right corrected images. First, the non-zero pixel regions of the left and right images are detected, and their intersection is calculated as the effective ROI. The formula is as follows:

[0092]

[0093] In the formula, (x,y) are the pixel coordinates in the image, I l (x,y), I r (x, y) represent the pixel intensities at position (x, y) in the left and right corrected images, respectively, and τ is the pixel intensity threshold. It is the logical symbol "AND".

[0094] Based on the effective ROI region, the cropping boundary that maximizes the preservation of the field of view is calculated to ensure the consistency of the field of view between the left and right images, while minimizing the invalid black area. The formula is as follows:

[0095] ;

[0096] In the formula, x min y min This represents the coordinates of the top-left corner of the final cropped region (ROI), x. max y max This represents the coordinates of the top right corner of the final cropped region (ROI). , This represents the coordinates of the top-left corner of the valid area of ​​the left image. , This represents the coordinates of the bottom right corner of the valid area of ​​the left image. , This represents the coordinates of the top-left corner of the valid area of ​​the right image. , This represents the coordinates of the lower right corner of the valid area of ​​the right image.

[0097] Finally, through multi-scale and adaptive ROI cropping, high-quality stereo-corrected image pairs with maximized field of view are obtained.

[0098] The stereo-corrected image pairs are then converted to grayscale using a weighted average method. By appropriately weighting and averaging the pixel values ​​of each RGB channel, the original RGB three-channel image containing rich color information is transformed into a single-channel image with only one-dimensional grayscale information. The formula is as follows:

[0099]

[0100] In the formula, Gyay represents the gray value; i and j represent the number of rows and columns, respectively, and (i, j) represents the pixel coordinates in the two-dimensional image; R represents the red channel; G represents the green channel; B represents the blue channel; and the three weights 0.299, 0.578 and 0.114 correspond to the 4:2:0 YUV format luma values.

[0101] The image after grayscale conversion using the weighted averaging method is subjected to contrast-limited adaptive histogram equalization. The specific process includes:

[0102] (1) Image segmentation and local histogram calculation: The grayscale image is divided into non-overlapping rectangular context regions, and a grayscale histogram is calculated for each region;

[0103] (2) Contrast Limitation and Histogram Crop: Set a cropping threshold and crop each local histogram to limit its amplitude. The formula is:

[0104]

[0105] In the formula, T clip This represents the total number of pixels within the context region. N is the preset clipping factor. pixels N represents the number of gray levels. bins This indicates the number of gray levels in the histogram.

[0106] Will exceed T clip The number of pixels is evenly redistributed across all gray levels;

[0107] (3) Local histogram equalization: Based on the clipped and redistributed local histogram, calculate the cumulative distribution function of each context region and apply the function to perform histogram equalization transformation on the region;

[0108] (4) Bilinear interpolation synthesis: To avoid block artifacts, the final gray value of any pixel in the image is obtained by bilinear interpolation through the transformation functions of its four adjacent context regions.

[0109] In step 3, the MS-BGNet stereo matching model based on fusion transfer learning sequentially includes the following five processing modules, such as... Figure 2As shown: (1) Feature extraction module, which is used to extract multi-scale features of input stereo image pairs using an architecture similar to ResNet, and outputs feature pyramids with three resolution levels of 1 / 2, 1 / 4 and 1 / 8;

[0110] (2) Cost volume construction and aggregation module, used to construct grouped associated cost volumes based on the multi-scale features, wherein the inter-group correlation is calculated using the following formula:

[0111]

[0112] In the formula, C g N represents the matching cost of the g-th group. g G represents the total number of feature channels, and G represents the number of groups. and are the feature vectors of the left and right images in the g-th group, respectively;

[0113] (3) A cost volume upsampling module based on a bilateral grid, used to upsample the low-resolution cost volume to a high resolution through a slicing operation, wherein the slicing operation is defined as:

[0114]

[0115] In the formula, B represents a two-sided network. This refers to the ratio of the width or height of the grid dimension to the high-resolution cost volume dimension. is the ratio of the gray level of the grid to the gray level of the guide map G.

[0116] (4) A multi-scale feature-guided cost volume enhancement module is used to fuse 1 / 2, 1 / 4, and 1 / 8 multi-scale features from the feature extraction module and enhance the cost volume through a spatial-disparity dual attention mechanism, wherein the spatial attention weight is calculated as follows:

[0117]

[0118] in W is the sigmoid function. S For convolution weights, Indicates feature splicing, This means concatenating feature maps at three scales (1 / 2, 1 / 4, and 1 / 8) along the channel dimension.

[0119] (5) Residual disparity refinement module, used to regress the final disparity map from the enhanced high-resolution cost volume using the soft argmin function, its calculation definition is:

[0120]

[0121] In the formula, D pred(x,y) represents the final predicted disparity value, and d is the disparity depth, D max For the maximum parallax depth, C enhanced (x,y,d) represents the enhanced cost volume.

[0122] And optimization is performed using a smooth L1 loss function, which is defined as:

[0123]

[0124] The model training phase employs a two-stage training strategy. First, the MS-BGNet model is pre-trained using the open-source large-scale stereo matching dataset SceneFlow and the KITTI dataset to learn general stereo matching prior knowledge. Then, the stereo matching network with the obtained pre-trained weights is fine-tuned using a self-built stereo matching dataset with ore stockpile as the target to optimize the model's ability to perceive the surface texture and geometric features of ore stockpile, thereby improving its matching accuracy and robustness in specific application scenarios.

[0125] The left and right views of the ore stockpile after the stereo correction and preprocessing in step 2 are input into the finely tuned MS-BGNet model. Through the forward propagation calculation of the model, a high-quality disparity map is directly output.

[0126] In step 4, the disparity map generated based on the MS-BGNet stereo matching algorithm is used to map two-dimensional image points to three-dimensional space through coordinate transformation, with each pixel... and its disparity value Can be converted to 3D world coordinates, parallax value The calculation formula is:

[0127]

[0128] In the formula, x l and x r denoted as , where is the distance from the imaging point in the left and right imaging planes to the left edge of the plane, respectively.

[0129] Three-dimensional coordinates (X) w Y w Z w The calculation formula is:

[0130]

[0131] In the formula, b is the baseline of the binocular camera, f is the focal length of the camera, d is the parallax value, (x,y) is the coordinate in the imaging plane coordinate system, (u,v) is the coordinate in the pixel coordinate system, and α and β are the focal distances of the pixels in the x-axis and y-axis directions of the imaging plane.

[0132] Based on the point cloud data obtained above, a triangular prism micro-element is constructed. Each triangular prism micro-element consists of three-dimensional spatial points corresponding to three adjacent pixels. Each pixel corresponds to an upper vertex located on the surface of the material and a lower vertex located on the reference plane, thus forming an irregular triangular prism with six vertices.

[0133] For each triangular prism element, it is divided into six irregular tetrahedrons, as shown in the diagram below. Figure 3 As shown; by calculating the center point G of the triangular prism and connecting this center point G to the six vertices of the triangular prism, the six tetrahedrons are formed; wherein, the three-dimensional coordinates of the center point G are calculated by the following formula:

[0134]

[0135] Among them, (X) n1 , Y n1 Z n1 ), (X n2 , Y n2 Z n2 ), (X n3 , Y n3 Z n3 Let (X) be the three-dimensional coordinates of the three vertices of the lower base of the triangular prism. m1 , Y m1 Z m1 ), (X m2 , Y m2 Z m2 ), (X m3 , Y m3 Z m3 ) represents the three-dimensional coordinates of the three vertices of the upper base of the triangular prism.

[0136] Calculate the volume of each tetrahedron using the determinant method. For a tetrahedron formed by four vertices P1(x1, y1, z1), P2(x2, y2, z2), P3(x3, y3, z3), and P4(x4, y4, z4), the volume V is calculated using the following formula:

[0137]

[0138] The volume of a single triangular prism element is obtained by adding the volumes of the six tetrahedrons together.

[0139] The total volume of the ore stockpile is obtained by summing the volumes of all the triangular prism elements.

[0140] Example 2

[0141] Field measurements were conducted at the mine. The disparity maps generated by the SGBM, BM, and PM algorithms and the MS-BGNet algorithm of this invention were tested using the Alpha Shapes algorithm, the direct integration method, and the triangular prism micro-element decomposition volume measurement method proposed in this invention. The results were compared with the actual volume. The volume measurement results are shown in Table 1, and the disparity map generation time of each stereo matching algorithm is shown in Table 2.

[0142] Table 1. Volume Measurement Results

[0143]

[0144] Table 2 Disparity map generation time for each stereo matching algorithm

[0145]

[0146] Experimental results show that, based on the MS-BGNet stereo matching algorithm proposed in this invention, the disparity map generated and the volume of the ore stockpile calculated using various methods have the lowest relative error rate in each group of experiments. Furthermore, when the MS-BGNet stereo matching algorithm and the triangular prism micro-element decomposition volume measurement method proposed in this invention are combined, the volume measurement effect is the best, with the lowest relative error rate of 3.53%, and the disparity map generation time is short. This demonstrates that the method proposed in this invention meets the requirements of field measurement in terms of real-time performance and accuracy, indicating the effectiveness of the proposed method.

[0147] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for measuring the volume of ore stockpiles based on binocular stereo vision, characterized in that, Includes the following steps: Step 1: Use a binocular camera to collect left and right views of the ore stockpile and construct a dataset of stereo matching data, including 300 pairs of stereo matching data for large-particle ore stockpile and 300 pairs of stereo matching data for small-particle ore stockpile. Step 2: First, the left and right views acquired in Step 1 are corrected using the improved multi-scale adaptive Bouguet stereo correction algorithm to eliminate lens distortion and align the row pixels of the left and right views to obtain stereo-corrected image pairs. Then, the stereo-corrected image pairs are processed into grayscale using a weighted average method. Finally, the images processed into grayscale using the weighted average method are subjected to adaptive histogram equalization with limited contrast to highlight the edges and texture information of the ore pile. Step 3: Construct the MS-BGNet stereo matching model based on fusion transfer learning; The training phase of the MS-BGNet stereo matching model adopts a two-stage training strategy: First, pre-training is performed using an open-source large-scale stereo matching dataset. Then, the stereo matching network with the obtained pre-trained weights is fine-tuned using a self-built ore stockpile dataset. After fine-tuning, the left and right views of the ore stockpile obtained in Step 2 are input into the model to generate a disparity map. Step 4: Based on the disparity map generated in Step 3, calculate the ore stockpile volume using the triangular prism micro-element decomposition method.

2. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 1, characterized in that: In step two, the improved multi-scale adaptive Bouguet stereo correction algorithm includes the following steps: Step 1: After completing the basic Bouguet stereo correction, in order to achieve a balance between large-scale distortion and local details, a multi-resolution fusion correction module is introduced to decompose the original image at multiple scales, applying stereo correction at multiple resolution levels to construct a pyramid. The pyramid construction process can be represented as follows: ， ； In the formula, I l (k) I r (k) These are the left and right eye images at the k-th level of the pyramid, respectively. PyrDown() is the pyramid downsampling operation. I l (k-1) I r (k-1) These are the left and right images of the pyramid on the next level, respectively, where k is the pyramid level number. Step 2: After completing local corrections at each scale, the correction results at different resolutions are fused to obtain a final corrected image that possesses both global consistency and local detail preservation. The fusion process can be described as follows: ; ； In the formula, This is the final fused and corrected image. PyrUp() is the pyramid upsampling operation. k Let be the weight of the k-th layer scale in the final fusion; Step 3: After completing multi-scale correction and fusion, an adaptive ROI module is introduced. By analyzing the effective regions of the left and right corrected images, the optimal cropping window is automatically calculated. First, the non-zero pixel regions of the left and right images are detected, and the intersection is calculated as the effective ROI. The formula is as follows: ； In the formula, (x,y) are the pixel coordinates in the image, I l (x,y), I r (x, y) represent the pixel intensities at position (x, y) in the left and right corrected images, respectively, and τ is the pixel intensity threshold. The logical symbol "AND"; Step 4: Based on the effective ROI region, calculate the cropping boundary that maximizes the preservation of the field of view, ensuring consistency of the field of view between the left and right images, while minimizing the invalid black area. The formula is: ； In the formula, x min y min This represents the coordinates of the top-left corner of the final cropped region (ROI), x. max y max This represents the coordinates of the top right corner of the final cropped region (ROI). , This represents the coordinates of the top-left corner of the valid area of ​​the left image. , This represents the coordinates of the bottom right corner of the valid area of ​​the left image. , This represents the coordinates of the top-left corner of the valid area of ​​the right image. , This represents the coordinates of the lower right corner of the valid area of ​​the right image; Step 5: Finally, through multi-scale and adaptive ROI cropping, high-quality stereo-corrected image pairs with maximized field of view are obtained.

3. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 2, characterized in that: In step two, the weighted average method for grayscale conversion involves performing an appropriate weighted average operation on the pixel values ​​of each RGB channel, transforming the original RGB three-channel image containing rich color information into a single-channel image with only one-dimensional grayscale information. The formula is as follows: ； In the formula, Gyay represents the grayscale value; Where i and j represent the number of rows and columns, respectively, and (i, j) represents the pixel coordinates in the two-dimensional image; R G represents the red channel; G represents the green channel; B represents the blue channel. The three weights, 0.299, 0.578, and 0.114, correspond to 4:2:0 YUV format luma values, respectively.

4. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 3, characterized in that: In step two, the adaptive histogram equalization process for limiting contrast specifically includes the following steps: Step 1, Image segmentation and local histogram calculation: Divide the grayscale image into non-overlapping rectangular context regions and calculate the grayscale histogram for each region; Step 2, Contrast Limiting and Histogram Cropping: Set a cropping threshold and crop each local histogram to limit its amplitude. The formula is: ； In the formula, T clip This represents the total number of pixels within the context region. N is the preset clipping factor. pixels N represents the number of gray levels. bins This indicates the number of gray levels in the histogram. Will exceed The number of pixels is evenly redistributed across all gray levels; Step 3, Local Histogram Equalization: Based on the clipped and redistributed local histograms, calculate the cumulative distribution function for each context region, and apply the function to perform histogram equalization transformation on the region. Step 4, Bilinear Interpolation Synthesis: To avoid block artifacts, the final gray value of any pixel in the image is obtained by bilinear interpolation through the transformation functions of its four adjacent context regions.

5. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 4, characterized in that: In step three, the MS-BGNet stereo matching model based on fusion transfer learning includes a feature extraction module, a cost volume construction and aggregation module, a cost volume upsampling module based on bilateral grids, a multi-scale feature-guided cost volume enhancement module, and a residual disparity refinement module; wherein: The feature extraction module is used to extract multi-scale features of the input stereo image pair using a ResNet-like architecture, and outputs feature pyramids containing three resolution levels: 1 / 2, 1 / 4, and 1 / 8. The cost volume construction and aggregation module is used to construct grouped associated cost volumes based on the multi-scale features, wherein the inter-group correlation is calculated using the following formula: ； In the formula, C g N represents the matching cost of the g-th group. g G represents the total number of feature channels and the number of groups. and are the feature vectors of the left and right images in the g-th group, respectively; The cost volume upsampling module based on bilateral grids is used to upsample the low-resolution cost volume to a high resolution through a slicing operation, the mathematical definition of which is: ； In the formula, B represents a two-sided network. This refers to the ratio of the width or height of the grid dimension to the high-resolution cost volume dimension. The ratio of the gray levels of the grid to the gray levels of the guide map G; The multi-scale feature-guided cost volume enhancement module is used to fuse 1 / 2, 1 / 4, and 1 / 8 multi-scale features from the feature extraction module, and enhances the cost volume through a spatial-parallax dual attention mechanism, wherein the spatial attention weight is calculated as follows: ； In the formula, W is the sigmoid function. S For convolution weights, Indicates feature splicing, This means concatenating feature maps at three scales (1 / 2, 1 / 4, and 1 / 8) along the channel dimension. The residual disparity refinement module is used to regress the final disparity map from the enhanced high-resolution cost volume using the soft argmin function, and its calculation is defined as follows: ； In the formula, D pred (x,y) represents the final predicted disparity value, and d is the disparity depth, D max For the maximum parallax depth, C enhanced (x,y,d) represents the enhanced cost volume; And optimization is performed using a smooth L1 loss function, which is defined as: 。 6. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 5, characterized in that: In step three, the two-stage training strategy is as follows: First, the MS-BGNet stereo matching model is pre-trained using the open-source large-scale stereo matching dataset SceneFlow and the KITTI dataset, so that it learns general stereo matching prior knowledge. Subsequently, the stereo matching network with pre-trained weights is fine-tuned using a self-built ore stockpile dataset to optimize the model's ability to perceive the surface texture and geometric features of the ore stockpile, thereby improving its matching accuracy and robustness in specific application scenarios.

7. The method for measuring the volume of ore stockpiles based on binocular stereo vision according to claim 6, characterized in that: In step four, the specific steps for calculating the ore stockpile volume using the triangular prism infinitesimal element decomposition method based on the disparity map generated in step three include the following: Step 1: Based on the disparity map generated by the MS-BGNet stereo matching model, the two-dimensional image points are mapped to three-dimensional space through coordinate transformation, and each pixel... and its disparity value Can be converted to three-dimensional coordinates, disparity values The calculation formula is: ； In the formula, x l and x r These are the distances from the imaging points in the left and right imaging planes to the left edge of the plane, respectively. Three-dimensional coordinates (X) w Y w Z w The calculation formula is: ； In the formula, b is the baseline of the binocular camera, f is the focal length of the camera, d is the parallax value, (x,y) is the coordinate in the imaging plane coordinate system, (u,v) is the coordinate in the pixel coordinate system, and α and β are the focal distances of the pixels in the x-axis and y-axis directions of the imaging plane. Step 2: Construct triangular prism micro-elements based on the three-dimensional coordinates obtained above. Each triangular prism micro-element consists of three-dimensional spatial points corresponding to three adjacent pixels. Each pixel corresponds to an upper vertex located on the surface of the material and a lower vertex located on the reference plane, thus forming an irregular triangular prism with six vertices. Step 3: For each triangular prism element, divide it into six irregular tetrahedrons; calculate the center point G of the triangular prism and connect the center point G to the six vertices of the triangular prism to form the six tetrahedrons; wherein, the three-dimensional coordinates of the center point G are calculated using the following formula:

8. Among them, (X n1 Y n1 Z n1 ), (X n2 Y n2 Z n2 ), (X n3 Y n3 Z n3 ( ) represents the three-dimensional coordinates of the three vertices of the lower base of the triangular prism. (X m1 Y m1 Z m1 ), (X m2 Y m2 Z m2 ), (X m3 Y m3 Z m3 ) represents the three-dimensional coordinates of the three vertices of the upper base of the triangular prism; Step 4: Calculate the volume of each tetrahedron using the determinant method. For a tetrahedron formed by four vertices P1(x1, y1, z1), P2(x2, y2, z2), P3(x3, y3, z3), and P4(x4, y4, z4), the volume V is calculated using the following formula: ； Step 5: Add the volumes of the six tetrahedrons together to obtain the volume of a single triangular prism element; Step 6: Sum the volumes of all triangular prism elements to obtain the total volume of the ore stockpile.

Images