Method for calculating gravity center of material pile in mine truck based on visual data

Abstract

The invention belongs to the technical field of mining machinery intelligence and automation, and discloses a method for calculating the gravity center of a material pile in a mine truck based on visual data. Material pile images and mine card pose data are collected through a convergent binocular vision system and a GNSS / IMU system; obtaining an effective material pile image through median filtering, image segmentation and spatial constraint screening; a Canny algorithm and a line-by-line scanning method are adopted to detect material pile vertexes; building a material pile three-dimensional model based on binocular vision coordinate conversion and repose angle modeling; a Monte Carlo sampling method is adopted to generate sampling points in a car hopper space range, effective sampling points are screened through a material pile three-dimensional model equation, and the barycentric coordinates of a material pile are calculated based on the effective sampling points. The blank of special detection of the gravity center of the material pile in the prior art is filled, the method has the advantages of small error, less time consumption, low cost and the like, the state of the material pile can be sensed in real time, data support is provided for dynamic adjustment of an unloading point, and the unbalance loading overturning risk of the mine truck is reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent and automated mining machinery technology, and in particular to a method for calculating the center of gravity of the material pile inside a mining truck based on visual data. Background Technology

[0002] In open-pit mining operations, mining trucks are core transportation equipment, and the safety and efficiency of their loading operations directly affect the overall production benefits of the mine. The problem of uneven loading of materials in the truck bed is a common technical issue in loading operations: when the center of gravity of the material pile deviates too much from the central axis of the truck bed, it leads to an imbalance of forces during the truck's movement, increasing the risk of overturning when turning or climbing slopes. Simultaneously, uneven loading also exacerbates tire wear, chassis deformation, and other equipment damage, reducing the service life of the mining truck and increasing mine operating costs.

[0003] Currently, the unloading of material from electric shovels into mine truck buckets in open-pit mines relies on the operator's experience. The operator visually observes the shape of the material pile to determine if there is uneven loading and manually adjusts the unloading point. This method suffers from high subjectivity, low accuracy, and poor consistency. Furthermore, with the development of intelligent mining, manual operation is no longer sufficient to meet the demands of automated operations. Therefore, it is essential to detect the center of gravity of the material pile inside the mine truck bucket to ensure the normal unloading process. Summary of the Invention

[0004] To address the aforementioned issues, a method for calculating the center of gravity of a stockpile within a mining truck based on visual data is provided, enabling accurate perception of the stockpile's three-dimensional information and real-time calculation of its center of gravity coordinates.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for calculating the center of gravity of a stockpile inside a mining truck based on visual data, comprising the following steps:

[0006] Step 1: Acquire images of the material pile inside the mining truck bucket using a convergent binocular vision system installed on the slewing body of the electric shovel; obtain the position and orientation data of the mining truck relative to the electric shovel using a positioning and orientation system installed on the slewing body and the body of the mining truck; the position and orientation data includes the three-dimensional position coordinates and attitude angles of each corner point of the mining truck bucket relative to the electric shovel.

[0007] Step 2: Perform median filtering for noise reduction, HSV color space conversion, and Otsu threshold segmentation on the acquired material pile images in sequence, and remove background interference by combining the space constraints of the truck bed.

[0008] Step 3: For the image processed in Step 2, the Canny edge detection algorithm is used to extract the outline of the material pile, and the pixel coordinates of the vertices of the material pile in the left and right camera images are located by line-by-line scanning.

[0009] Step 4: Based on the calibration parameters of the binocular camera, perform coordinate transformation, calculate the three-dimensional coordinates of the top of the material pile, and construct a simplified conical model of the material pile by combining the material repose angle and the geometric constraint equation of the bucket; the geometric constraint equation of the bucket uses the pose data as the spatial position reference.

[0010] Step 5: Use the Monte Carlo sampling method to generate random points within the constraints of the truck bed, filter valid sampling points through the material pile model equation, and calculate the coordinates of the center of gravity of the material pile.

[0011] The camera optical axis of the convergent binocular vision system is installed on the side of the electric shovel, and the angle between the two cameras is a preset angle. The baseline distance meets the accuracy requirements of long baseline measurement. The camera moves synchronously with the rotating body of the electric shovel, and there is no need for repeated calibration. The positioning antenna of the positioning and attitude determination system is installed on the rotating body of the electric shovel and the body of the mining truck.

[0012] The original image is filtered using a median filtering algorithm, with the filter template size adaptively selected based on the image noise intensity. The original RGB image is converted to the HSV color space. The truck bed region and the background are segmented based on the grayscale difference of the saturation channel, and the material pile region and the truck bed surface are segmented based on the grayscale difference of the luminance channel. Based on the positional relationship between the truck bed and the material pile, a truck bed region constraint model in pixel coordinates is established. In the X direction, the left and right boundaries of the material pile region are restricted from exceeding the left and right boundaries of the truck bed region, and in the Y direction, the material pile region is restricted to be above the truck bed region. Invalid pixels outside the truck bed region are removed, and only the valid image region containing the material pile is retained.

[0013] The Canny edge detection algorithm is used to extract edges from the effective material pile image. The edge detection effect is optimized by adaptive threshold adjustment to preserve the continuous edge features of the material pile outline. The horizontal line-by-line scanning is performed from the bottom to the top of the effective image area, and the coordinates of the intersection point of each scan line and the edge of the material pile are recorded.

[0014] When the intersection of a scan line and the edge of the material pile degenerates into a single pixel, the pixel is determined to be the vertex of the material pile, and its pixel coordinates in the left and right field images are recorded simultaneously.

[0015] The Zhang Zhengyou calibration method was used to pre-calibrate the binocular camera, obtain the camera's intrinsic and extrinsic parameters and distortion coefficients, and establish a mathematical model for camera imaging. Based on the distortion coefficients obtained from the calibration, the pixel coordinates in the left and right field-of-view images were distorted to eliminate the influence of lens optical distortion on coordinate measurement. Based on the triangulation principle of binocular vision, the disparity of the material pile vertex in the left and right field-of-view images was calculated. Combined with the camera calibration parameters and the reprojection matrix, the pixel coordinates were converted into three-dimensional coordinates in the camera coordinate system. Based on the mine truck pose data obtained from the GNSS / IMU system, a rigid transformation matrix between the camera coordinate system and the mine truck coordinate system was constructed. The three-dimensional coordinates of the material pile vertex were transformed to the mine truck coordinate system through coordinate transformation.

[0016] Based on the angle of repose characteristics of bulk materials, the material pile is abstracted into a conical geometric model. The apex of the material pile is the three-dimensional coordinate point obtained in step 4. The angle between the generatrix of the cone and the horizontal plane is equal to the angle of repose of the material. The angle of repose of the material is pre-determined through a preset standard test method, or a preset angle of repose parameter range is matched based on the material type on site. According to the length, width, height and internal contour structure of the mining truck bucket, the spatial constraint equation of the truck bucket in the mining truck coordinate system is established to clarify the boundary range of the truck bucket in the X, Y and Z directions. The conical geometric model equation and the truck bucket spatial constraint equation are solved simultaneously to obtain the three-dimensional model of the material pile within the constraint range of the truck bucket. When the conical part exceeds the boundary of the truck bucket in the Z direction, the material pile model inside the truck bucket is a truncated irregular conical structure.

[0017] Within the truck bed space in the mining coordinate system, a preset number of uniformly distributed random sampling points are generated. The number of sampling points is adaptively adjusted according to the accuracy requirements of the centroid calculation. The coordinates of each random sampling point are substituted into the three-dimensional model equation of the stockpile. The intersection of the cone equation constructed based on the stockpile vertex and the angle of repose and the spatial constraint equation of the truck bed, that is, the set of spatial points that simultaneously satisfy the "basic cone equation" and the "truck bed X / Y / Z three-dimensional boundary constraints", constitutes the three-dimensional model equation of the stockpile within the constraint range of the truck bed. Sampling points that satisfy the constraint conditions of the model equation are determined as valid sampling points. When the stockpile is completely within the truck bed boundary, the mean of the coordinates of all valid sampling points is calculated to obtain the centroid coordinates of the stockpile. When the stockpile vertex exceeds the truck bed boundary, the volumes of the complete cone part outside the truck bed and the irregular cone part with truncated tops inside the truck bed are calculated respectively. Based on the volume ratio of each part and its respective centroid coordinates, the total centroid coordinates of the stockpile are calculated by weighted summation.

[0018] The beneficial effects of this invention are as follows: This invention abandons the redundant calculation logic of global stereo matching and adopts a lightweight technical path of "material pile edge detection - vertex localization - repose angle modeling": First, the outline of the material pile is extracted by Canny edge detection (only the pixels in the edge area of ​​the material pile are processed, rather than the pixels of the whole image), then the vertex of the material pile is quickly located by line-by-line scanning, and finally the material pile model is directly reconstructed by combining the material repose angle and the vehicle bucket constraint - without the need to build a complete point cloud, which greatly reduces the amount of data processing. The total time for material pile modeling and centroid calculation can be shortened to less than 2.5 seconds. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention.

[0020] Figure 2 This is a schematic diagram of the installation of the binocular system according to the method of the present invention.

[0021] Figure 3 The images shown are: (a) a schematic diagram of image segmentation of a mining truck bucket; and (b) a schematic diagram of image segmentation of the candidate material pile area.

[0022] Figure 4 The images show the results of the pile vertex detection; (a) shows the results of the pile vertex detection in the left image; and (b) shows the results of the pile vertex detection in the right image.

[0023] Figure 5 This is a schematic diagram of the truck bed coordinate system.

[0024] Figure 6 This is a schematic diagram of the three-dimensional reconstruction of the material pile and the calculation of its center of gravity coordinates. Detailed Implementation

[0025] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to examples and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0026] (1) Images of the material pile inside the mining truck bucket are collected by a convergent binocular vision system installed on the slewing body of the electric shovel, and the position and attitude data of the mining truck relative to the electric shovel are obtained by a GNSS / IMU positioning and attitude determination system installed on the slewing body of the electric shovel and the body of the mining truck.

[0027] (2) The collected material pile images are subjected to median filtering for noise reduction, HSV color space conversion and Otsu threshold segmentation in sequence, and background interference is removed by combining the space constraints of the truck bed.

[0028] (3) The outline of the material pile is extracted by the Canny edge detection algorithm, and the pixel coordinates of the top of the material pile in the left and right camera images are located by the line-by-line scanning method.

[0029] (4) Based on the calibration parameters of the binocular camera, coordinate transformation is performed to calculate the three-dimensional coordinates of the top of the material pile. Combined with the material repose angle and the geometric constraint equation of the truck hopper, a simplified model of the material pile cone is constructed.

[0030] (5) Use the Monte Carlo sampling method to generate random points within the constraints of the truck bed, filter the effective sampling points through the material pile model equation, and calculate the coordinates of the center of gravity of the material pile.

[0031] Furthermore, the optical axis of the convergent binocular vision system is installed on the side of the electric shovel, with the two cameras at a preset angle. The baseline distance meets the accuracy requirements for long baseline measurement. The cameras move synchronously with the rotating body of the electric shovel, eliminating the need for repeated calibration. The positioning antenna of the GNSS / IMU positioning and attitude determination system is installed on the rotating body of the electric shovel and the body of the mining truck.

[0032] Furthermore, median filtering is used to filter the original images acquired by the stereo camera, and the size of the filter template is adaptively selected according to the image noise intensity.

[0033] Furthermore, the original RGB image is converted to the HSV color space. The grayscale difference of the saturation channel is used to segment the truck bed area from the background, and the grayscale difference of the luminance channel is used to segment the material pile area from the truck bed surface.

[0034] Furthermore, based on the preset geometric contour parameters of the mining truck bucket, a truck bucket area constraint model in the image pixel coordinate system is established, invalid pixels outside the truck bucket area are eliminated, and only the effective image area containing the material pile is retained.

[0035] Furthermore, the Canny edge detection algorithm is used to extract edges from the effective material pile image. The edge detection effect is optimized by adaptive threshold adjustment, while preserving the continuous edge features of the material pile outline.

[0036] Furthermore, a horizontal line-by-line scan is performed from the bottom to the top of the effective image area, and the coordinates of the intersection point of each scan line with the edge of the material pile are recorded.

[0037] Furthermore, when the intersection of a certain scan line and the edge of the material pile degenerates into a single pixel, the pixel is determined to be the vertex of the material pile, and its pixel coordinates in the left and right field images are recorded simultaneously.

[0038] Furthermore, the Zhang Zhengyou calibration method was used to pre-calibrate the binocular camera, obtain the camera's intrinsic and extrinsic parameters and distortion coefficients, and establish a mathematical model for camera imaging.

[0039] Furthermore, based on the calibrated distortion coefficients, distortion correction is performed on the pixel coordinates in the left and right field-of-view images to eliminate the influence of lens optical distortion on coordinate measurement.

[0040] Furthermore, based on the triangulation principle of binocular vision, the disparity of the material pile vertex in the left and right field-of-view images is calculated. Combined with camera calibration parameters and reprojection matrix, the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system.

[0041] Furthermore, based on the pose data of the mining truck acquired by the GNSS / IMU system, a rigid transformation matrix is ​​constructed between the camera coordinate system and the mining truck body coordinate system. The three-dimensional coordinates of the top of the material pile are transformed to the mining truck coordinate system through coordinate transformation.

[0042] Furthermore, based on the angle of repose characteristics of bulk materials, the material pile is abstracted into a conical geometric model, with the vertex of the material pile being the three-dimensional coordinate point obtained in the previous step, and the angle between the generatrix of the cone and the horizontal plane being equal to the angle of repose of the material.

[0043] Furthermore, the angle of repose of the material is pre-determined using a preset standard test method, or a preset angle of repose parameter range is matched based on the type of material on site; according to the length, width, height and internal contour structure of the mining truck bucket, a spatial constraint equation for the truck bucket in the mining truck coordinate system is established to clarify the boundary range of the truck bucket in the X, Y and Z directions.

[0044] Furthermore, the geometric model equations of the cone and the spatial constraint equations of the bucket are solved simultaneously to obtain the three-dimensional model of the material pile within the constraint range of the bucket; when the cone part exceeds the boundary of the bucket in the Z direction, the material pile model inside the bucket is a truncated irregular cone structure.

[0045] Furthermore, within the truck bed space of the mining truck coordinate system, a preset number of uniformly distributed random sampling points are generated, and the number of sampling points is adaptively adjusted according to the accuracy requirements of the centroid calculation.

[0046] Furthermore, the coordinates of each random sampling point are substituted into the three-dimensional model equation of the material pile, and the sampling points that satisfy the constraints of the model equation are determined to be valid sampling points; when the material pile is completely located within the boundary of the truck hopper, the mean of the coordinates of all valid sampling points is calculated to obtain the coordinates of the centroid of the material pile.

[0047] Furthermore, when the apex of the material pile extends beyond the boundary of the truck hopper, the volumes of the complete conical part outside the truck hopper and the truncated conical part inside the truck hopper are calculated separately. Based on the volume ratio of each part and its respective center of gravity coordinates, the total center of gravity coordinates of the material pile are calculated by weighted summation.

[0048] Reference Figure 1 As shown, the present invention provides a method for calculating the center of gravity of a material pile in a mining truck hopper based on visual data, comprising the following steps:

[0049] Select an industrial-grade binocular camera with a resolution of at least 1920×1080 and a frame rate of at least 30FPS to ensure clear and real-time image acquisition. Secure the camera to the side of the shovel's rotating body, adjusting the angle between the optical axes of the two cameras to 15°-30° to ensure coverage of the entire loading area of ​​the mining truck's bucket.

[0050] GNSS positioning and directional antennas are installed on both sides of the electric shovel's loading platform, and IMU sensors are installed in the cab. GNSS positioning and directional antennas are also installed on both sides of the mining truck's bucket, and IMU sensors are installed in the cab to ensure stable sensor signal reception.

[0051] The Zhang Zhengyou calibration method was used to calibrate the stereo camera. The camera intrinsic matrix, extrinsic matrix and distortion coefficients were calculated using the camera calibration component in Matlab and stored in the system for coordinate transformation.

[0052] Record the structural parameters such as the length, width, and height of the electric shovel and the mining truck bucket in order to establish the spatial constraint equations for the bucket.

[0053] When the electric shovel begins loading, it triggers the binocular camera and GNSS / IMU system to work synchronously. The binocular camera acquires real-time left and right field-of-view images of the material pile inside the mining truck's bucket, while the GNSS / IMU system acquires real-time position and attitude data of the mining truck. The two types of data are aligned in time and space through timestamp synchronization technology.

[0054] A Python environment was set up in VS Code. A 3×3 median filter template was used to suppress noise and remove dust interference from the acquired left and right field-of-view images. The RGB images were converted to the HSV color space, and the saturation and luminance channels were extracted. Otsu's thresholding was used to segment the truck bed region and the material pile region, respectively. Based on preset truck bed structure parameters, the truck bed region was delineated in the image, and background interference outside the region was removed to obtain the effective material pile region image.

[0055] An adaptive threshold Canny algorithm is used to extract edges from the effective material pile image, obtaining continuous material pile edge contours. The effective image is scanned line by line from bottom to top, and the coordinates of the intersection points of each scan line and the material pile edge are recorded. When the intersection point of the scan line and the material pile edge degenerates into a single pixel, this point is determined as the vertex of the material pile, and its pixel coordinates in the left and right field-of-view images are recorded.

[0056] Based on preset camera distortion coefficients, distortion correction is performed on the vertex pixel coordinates. The disparity of the vertex in the left and right field-of-view images is calculated, and combined with camera calibration parameters and the reprojection matrix, the pixel coordinates are converted to 3D coordinates in the camera coordinate system. Based on the mine truck pose data acquired by the GNSS / IMU system, a rigid transformation matrix is ​​constructed to transform the 3D coordinates in the camera coordinate system to the mine truck coordinate system, obtaining the final 3D coordinates of the pile vertex. .

[0057] Based on the preset angle of repose Establish the equations for the cone-shaped material pile model: .according to Figure 5 The coordinate system shown is used as follows. Substituting the preset space constraint equations for the truck bed, the constraint equation in the Y direction is: The constraint in the X direction changes with the Z direction. hour:

[0058]

[0059] when hour:

[0060]

[0061] The three-dimensional model of the material pile within the constraint range of the truck hopper is obtained by solving the simultaneous equations, and it is determined whether the vertices of the material pile exceed the boundary of the truck hopper.

[0062] A predetermined number of random sampling points are generated within the space of the truck bed. These points are then substituted into the three-dimensional model equation of the material pile to obtain valid sampling points. If the material pile does not extend beyond the truck side, the centroid coordinate R is [value missing].

[0063]

[0064] In the formula For all points falling within the range of the stockpile equation coordinates and, For all points falling within the range of the stockpile equation coordinates and, Let z be the sum of the z-coordinates of all points falling within the range of the stockpile equation.

[0065] If the material pile exceeds the side of the truck, calculate the volume and center of gravity of the inner and outer parts of the truck bed separately, and obtain the coordinates of the total center of gravity by weighted summation:

[0066]

[0067] In the formula for The volume of the material pile exceeding the side of the car for:

[0068]

[0069] The volume of material pile inside the truck bed for:

[0070]

[0071] Center of gravity of material stack inside the truck bed for:

[0072] .

[0073] Figure 2 The diagram shows the camera installation location. In the initial unloading state, the mining truck is stopped on the side of the electric shovel. The binocular vision system is installed on the rotating body of the electric shovel to capture images of the mining truck carrying the material pile. Before the electric shovel performs the unloading action, the center of gravity coordinates of the material in the mining truck bucket are calculated, and the unloading is guided based on this.

[0074] Figure 3 In Figures (a) and (b), the RGB image is converted to the HSV color space, the saturation and luminance channels are extracted, and the hopper region and candidate material pile region are obtained after Otsu thresholding. The complete material pile region can be extracted from the image based on the positional relationship between the material pile and the hopper, which serves as the basis for subsequent edge detection.

[0075] Figure 4 In the left and right images, (a) and (b) show the results (pixel coordinates) of the pile vertex detection. Based on the camera calibration parameters, the three-dimensional coordinates of the pile vertex in the camera coordinate system can be calculated. This is then used as the basis for subsequent pile reconstruction.

[0076] Figure 5 This is a schematic diagram of the coordinate system for a mining truck bucket. The origin is the front left corner of the bucket, the X direction is the front of the bucket, the Y direction is the right side of the bucket, and the Z direction points to the ground. Based on different bucket dimensions, the corresponding bucket constraint equations can be calculated. Intersecting these equations with the equation of the material pile cone yields the complete equation of the irregular cone of the material pile.

[0077] Figure 6 This is a schematic diagram of three-dimensional reconstruction of a material pile given the dimensions of the truck bucket, the coordinates of the pile apex, and the angle of repose. The three-dimensional coordinates of the current center of gravity of the material pile in the truck bucket coordinate system can be calculated by Monte Carlo sampling.

Claims

1. A method for calculating the center of gravity of a material pile in a mine car based on visual data, characterized by, The method comprises the following steps: Step 1: collecting the image of the material pile in the truck body of the mine truck through a converging binocular vision system installed on the slewing body of the electric shovel, and obtaining the pose data of the mine truck relative to the electric shovel through a positioning and pose system installed on the slewing body of the electric shovel and the body of the mine truck; the pose data comprises three-dimensional position coordinates and attitude angles of each corner point of the truck body of the mine truck relative to the electric shovel; Step 2: sequentially performing median filter denoising, HSV color space conversion and Otsu threshold segmentation on the collected material pile image, and removing background interference in combination with the truck body space constraint; Step 3: extracting the material pile profile by using a Canny edge detection algorithm on the image processed in step 2, and positioning the pixel coordinates of the material pile vertex in the left and right camera images through the line-by-line scanning method; Step 4: performing coordinate conversion based on the binocular camera calibration parameters, calculating the three-dimensional coordinates of the material pile vertex, and constructing a material pile cone simplified model in combination with the material repose angle and the truck body geometric constraint equation; the truck body geometric constraint equation takes the pose data as the spatial position reference; Step 5: generating random points in the truck body constraint range by using the Monte Carlo sampling method, screening effective sampling points by using the material pile model equation, and calculating the coordinates of the material pile barycenter.

2. The method for calculating the center of gravity of the material pile in the mine car based on visual data according to claim 1, characterized in that, The camera optical axes of the converging binocular vision system are installed on the side of the electric shovel, the included angle between the two cameras is a preset angle, the baseline distance meets the long-baseline measurement accuracy requirement, the cameras move synchronously with the slewing body of the electric shovel, and repeated calibration is not needed; the positioning antennas of the positioning and pose system are installed on the slewing body of the electric shovel and the body of the mine truck.

3. The method for calculating the center of gravity of the material pile in the mine car based on visual data according to claim 1, characterized in that, The original image is filtered by using the median filter algorithm, and the filter template size is adaptively selected according to the image noise intensity; the original image in the RGB format is converted to the HSV color space, the truck body area and the environment background are segmented according to the gray difference in the saturation channel, and the material pile area and the truck body surface are segmented according to the gray difference in the brightness channel; based on the position relationship between the truck body of the mine truck and the material pile, a truck body area constraint model under the pixel coordinate system is established, the left and right boundaries of the material pile area are limited to not exceed the left and right boundaries of the truck body area in the X direction, the material pile area is limited to be above the truck body area in the Y direction, invalid pixel points outside the truck body area are removed, and an effective image area containing only the material pile is reserved.

4. The method for calculating the center of gravity of the material pile in the mine car based on visual data according to claim 1, characterized in that, The Canny edge detection algorithm is used to extract the edges of the effective material pile image, the edge detection effect is optimized by adaptively adjusting the threshold, and the continuous edge features of the material pile profile are reserved; the horizontal line-by-line scanning is performed from the bottom to the top of the effective image area, and the intersection coordinates of each scanning line and the material pile edge are recorded; When the intersection of a certain scanning line and the material pile edge degenerates into a single pixel point, the pixel point is determined as the material pile vertex, and the pixel coordinates of the pixel point in the left and right field images are recorded synchronously.

5. The method for calculating the center of gravity of a material pile in a mine car based on visual data according to claim 1, characterized in that, The Zhang Zhengyou calibration method is used to pre-calibrate the binocular camera to obtain the camera intrinsic parameters, extrinsic parameters and distortion coefficients, and to establish a mathematical model of camera imaging. Based on the distortion coefficients obtained by calibration, the pixel coordinates in the left and right field of view images are corrected to eliminate the influence of lens optical distortion on coordinate measurement. Based on the triangulation principle of binocular vision, the disparity of the pile vertex in the left and right field of view images is calculated, and the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system combined with the camera calibration parameters and the re-projection matrix. Based on the pose data of the mine truck obtained by the GNSS / IMU system, a rigid transformation matrix between the camera coordinate system and the mine truck coordinate system is constructed, and the three-dimensional coordinates of the pile vertex are converted to the mine truck coordinate system through coordinate transformation.

6. The method for calculating the center of gravity of a material pile in a mine car based on visual data according to claim 1, characterized in that, Based on the repose angle characteristics of bulk materials, the pile is abstracted as a conical geometric model, and the pile vertex is the three-dimensional coordinate point obtained in step 4. The angle between the generatrix of the conical body and the horizontal plane is equal to the repose angle of the material. The repose angle of the material is determined in advance by a pre-set standard test method, or the repose angle parameter range is matched based on the type of the material on site. According to the length, width, height and internal contour structure of the mine truck bucket, a spatial constraint equation of the bucket in the mine truck coordinate system is established to clearly define the boundary range of the bucket in X, Y and Z directions. The conical geometric model equation and the bucket spatial constraint equation are solved simultaneously to obtain the three-dimensional model of the pile within the bucket constraint range. When the conical part exceeds the Z-direction bucket boundary, the pile model in the bucket is an irregular truncated cone structure.

7. The method for calculating the center of gravity of a material pile in a mine car based on visual data according to claim 1, characterized in that, In the spatial range of the bucket in the mine truck coordinate system, a preset number of uniformly distributed random sampling points are generated, and the number of sampling points is adaptively adjusted according to the accuracy requirement of the center of gravity calculation. The coordinates of each random sampling point are substituted into the three-dimensional model equation of the pile, and the intersection of the conical equation constructed based on the pile vertex and the repose angle and the bucket spatial constraint equation is obtained, i.e. the space point set that satisfies the "basic conical equation" and the "bucket X / Y / Z three-direction boundary constraint" at the same time, which constitutes the three-dimensional model equation of the pile within the bucket constraint range, and the sampling points that satisfy the model equation constraint condition are determined as effective sampling points. When the pile is completely located within the bucket boundary, the mean value of the coordinates of all effective sampling points is calculated to obtain the center of gravity coordinates of the pile. When the pile vertex exceeds the bucket boundary, the volumes of the complete conical body outside the bucket and the irregular truncated conical body inside the bucket are calculated respectively, and the total center of gravity coordinates of the pile are calculated by weighted summation based on the volume proportion and the respective center of gravity coordinates of each part.

Images