Method for Detecting Wear of Liner Plate of Mine Crusher

Abstract

The present invention relates to a method for detecting the wear of a lining plate of a mine crusher, belonging to the technical field of mine machinery crushing. The method uses binocular vision to obtain three-dimensional information of the lining plate surface in regions. When obtaining the information, a digital speckle pattern is projected onto the lining plate surface. While supplementing light in the case of insufficient light in the crushing chamber, the texture information of the lining plate surface is increased. The speckle pattern provides a higher parallax estimation when obtaining the depth map of the binocular images, and the point cloud information of the lining plate surface with higher accuracy is obtained. The present invention introduces the information of an inertial measurement unit to provide a reference for the initial stitching of the point cloud of the lining plate surface after regional division. After the initial stitching, a secondary stitching with higher accuracy is carried out, which improves the data acquisition efficiency while ensuring the acquisition accuracy. The present invention aligns the lining plate point cloud through an improved normal transformation algorithm, finds the nearest points of each point of the worn point cloud in the point cloud obtained before wear, calculates the distance between the two points in the unit normal direction, obtains a higher accuracy of the wear amount of the lining plate, and obtains the wear distribution of the lining plate.

Description

Technical Field

[0001] The present invention relates to the technical field of mine machinery crushers, and particularly relates to a method for detecting the wear of a lining plate of a mine crusher. Background Art

[0002] A mine crusher refers to equipment for crushing raw ores such as iron and copper, and its comprehensive performance determines the mining efficiency of the entire mine. In recent years, with the serious depletion of various metal and non-metal minerals globally, in order to maintain or increase the production of various metals and non-metals, the amount of raw ore that needs to be processed through the crushing and grinding production process each year has increased significantly. Crushers mostly use impact, extrusion, and impact methods. By the action of the crusher lining plate and the material, the material is changed from large particles to small particles, while causing wear of the lining plate. The wear of the lining plate will cause a change in the cavity shape of the crushing chamber, thereby affecting the material crushing effect. As the wear amount increases, the lining plate needs to be replaced regularly.

[0003] A binocular camera can obtain two images of the same scene from different angles. Binocular stereo vision technology matches and detects two image points of the same target point on the two images. According to the imaging model of the binocular camera and the matching corresponding points, the three-dimensional coordinate information of this point can be calculated. However, there are still problems when using binocular stereo vision technology in the crushing chamber environment of a mine crusher, such as the simple texture of the lining plate, the inability to obtain the complete three-dimensional information of the lining plate in one acquisition, and the inaccurate acquisition information.

[0004] For a long time, there has been a lack of a rapid and effective detection method for the wear of the lining plate of a mine crusher. The replacement cycle of the lining plate is often planned based on indirect parameters such as product particle size, power, or the experience of the lining plate usage time, resulting in under-wear or over-wear situations often occurring during the replacement of the lining plate. The replacement of the lining plate needs to be judged according to the wear condition of the lining plate. Efficiently obtaining accurate lining plate wear data or wear distribution can provide a reference for enterprises to organize production and maintenance, and avoid major economic losses caused by under-wear or over-wear during the replacement of the lining plate. In recent years, lining plate wear detection methods such as ultrasonic measurement, embedded bolt sensors, and laser three-dimensional scanning measurement have problems such as low accuracy, low acquisition efficiency, or high acquisition cost. For the wear detection of the lining plate of a mine crusher, there is an urgent need to develop a detection method with high precision, low cost, and high efficiency. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting the wear of the lining plate of a mine crusher, which solves the problems faced by the application of binocular stereo vision technology in measuring the wear of the lining plate, such as the single texture of the lining plate surface, the complex operation of three-dimensional data stitching, and data noise interference. The method of the present invention projects a digital speckle pattern on the surface of the lining plate, obtains the three-dimensional point cloud of the lining plate surface in the crushing cavity of the crusher in regions, filters and smooths the point cloud, introduces the information of the inertial measurement unit for point cloud stitching, compares the point clouds before and after wear to obtain the wear amount and wear distribution of the lining plate, and improves the accuracy and efficiency of the lining plate wear detection.

[0006] The above object of the present invention is achieved by the following technical solutions:

[0007] A method for detecting the wear of the lining plate of a mine crusher includes the following steps:

[0008] Step S1: Obtain binocular images of the target area using the device for detecting the wear of the lining plate of a mine crusher. Specifically:

[0009] Step S101: Divide the detection area and determine the acquisition position.

[0010] The division of the detection area and the determination of the acquisition position are specifically realized as follows: Determine the acquisition location Y according to the detected lining plate area X and the acquisition environment.

[0011] X = {x1, x2, x3, …, x i}

[0012] Y = {y1, y2, y3, …, y j}

[0013] Divide the detected lining plate area X into rectangular target areas x size_a *x siez_b with a size of x1, …, x i . Select the acquisition location y i according to the position of the target area x j . Multiple target areas x i correspond to one acquisition location y j .

[0014] Step S102: Obtain IMU data.

[0015] The IMU data is the data output by the three single-axis accelerometers, three single-axis gyroscopes, and three single-axis magnetometers of the inertial measurement unit, which can calculate the pose change of the IMU and further obtain the pose change of the binocular camera fixed to the IMU.

[0016] The IMU data comes from an inertial measurement unit fixedly connected to a binocular camera in the lining wear detection device. The lining wear detection device includes a binocular camera, an inertial measurement unit (IMU) fixedly connected to the binocular camera, a projection device capable of projecting patterns, a computer as a data processing system, and a power supply capable of supplying power to the binocular camera, IMU, projection device, and data processing system simultaneously.

[0017] Start recording IMU data at the acquisition location y1, and continuously record IMU data when transferring the binocular camera to all subsequent acquisition locations y i . At the last acquisition location y end After the binocular camera acquires images, make the binocular camera return to the acquisition location y1 again and record IMU data. As the last acquisition, continuously record IMU data until this location, and then the acquisition of IMU data is completed.

[0018] Step S103: Use the projection device to project a digital speckle pattern onto the target area on the lining.

[0019] The digital speckle pattern is composed of random black and white pixels and is a rectangle with a size of T size_a *T siez_b . At each acquisition location y j Project the digital speckle pattern onto multiple target areas x i on the corresponding lining. The digital speckle pattern covers all target areas x j corresponding to this acquisition location y i .

[0020] Step S104: Use the binocular camera to obtain binocular images of the target area.

[0021] While projecting onto the lining at the acquisition location y j , use the binocular camera to collect images of multiple target areas x j on the lining corresponding to this acquisition location y i . Transmit the collected binocular images to the data processing system.

[0022] Step S2: Obtain the three-dimensional point cloud of the lining surface. Specifically:

[0023] Step S201: Adopt a stereo matching method based on Census matching cost calculation to obtain the point cloud of each target area.

[0024] In the data transmission system, for the binocular image groups (a, b) j at each acquisition location y j adopt an improved stereo matching method for calculating the matching cost to obtain the disparity map P d. The corresponding pixel points of the same binocular image group are located in the same row. Based on the left image a in the image group j find the right image b j and the corresponding matching point pairs in the same row.

[0025] Use the Census matching cost calculation method to calculate the matching cost C(p i ) of the pixel point p i in the image. The value of the matching cost is obtained by calculating the Hamming distance between it and the corresponding pixel point (p i , b j ) in the same group of images.

[0026] C(p i ) = H[C(p i , a j ), C(p i , b j )]

[0027] C(p i , a j ) represents the Census transform coding of the neighborhood of the pixel point (p j , a i ) in the left image a j . C(p i , b j ) represents the Census transform coding of the neighborhood of the pixel point (p j , b i ) in the right image b j . H[,] represents calculating the Hamming distance between the two codings. Taking the pixel point p i as the origin to establish a coordinate system, and taking the coordinate points within the coordinate window with the origin as the center and a size of 7*7 in the coordinate system as the preset coordinate points, the gray values (x, y) of the pixel points corresponding to the preset coordinate points are obtained Gv . Compare each gray value with the gray value of the pixel point p i . If the former is smaller, the coding position is 0; if the former is larger, the coding position is 1. After comparing the pixel point p i with all the preset coordinate points, the Census transform coding of this pixel point is generated.

[0028] Traverse the pixel points (p j , a i ) in the left image a j to generate the Census transform coding C(p i , a j ) of the pixel points. Find all the pixel points in the same row in the right image b j in the same group, denoted as the pixel points (p i , a j) point set, generate respective Census transform encodings for all pixel points in this point set. Calculate the Hamming distance between the pixel point (p i ,a j ) and the encodings of each pixel point in its point set. The pixel point corresponding to the minimum Hamming distance is the pixel point (p i ,a j )'s matching pixel point in the right image b j . Calculate the depth information for the corresponding matching pixel points of the same group of images in combination with the camera parameters, and obtain the three-dimensional coordinates of all points in the left image in the camera coordinate system, that is, obtain the point cloud.

[0029] Step S202: Filter the point cloud of each target area to obtain the three-dimensional point cloud on the surface of the liner area.

[0030] For each point P cloud_point_i in the point cloud of each target area, apply a three-dimensional KD tree to search for the K nearest neighboring points around it as the search point set of this point. Calculate the Euclidean distance between P cloud_point_i and the points in the search point set, and calculate the average value d cloud_point_i of the Euclidean distances between P pi and all points in the search point set;

[0031] d th_i = d pi + s

[0032] where d th_i is the density threshold of the said P cloud_point_i ; d pi is the average value of the Euclidean distances between the said P cloud_point_i and all points in its search point set; s is the standard deviation of the average value of the Euclidean distances between the said P cloud_point_i and its K nearest neighboring points;

[0033] Judge whether P cloud_point_i meets the filtering condition, and filter out the points that meet the filtering condition. The filtering condition is: count the number of points in its search point set whose Euclidean distance from the said P cloud_point_i is less than d th_i , judge whether this value is less than 0.84K, and judge whether the average value d cloud_point_i of the Euclidean distances between the said P pi and its search point set is greater than the density threshold.

[0034] Step S203: Complete point cloud stitching in combination with IMU data.

[0035] Obtain the relative pose Pose k of the binocular camera at the acquisition location y k-1 through the IMU data. kk-1 .

[0036]

[0037] Where R0 is a 3x3 rigid body rotation matrix and t0 is a 3x1 rigid body translation matrix, both of which are calculated from IMU data.

[0038] The relative pose of the binocular camera at the last acquisition and the previous acquisition is Pose jj-1 . The point cloud of the target area at the last acquisition is matched with the point cloud of the target area at the first acquisition, and the relative pose Pose of the camera is calculated and obtained 1j .

[0039] Pose 21 Pose 32 …Pose jj-1 Pose 1j = Pose error

[0040] In the formula, Pose error is the value obtained by multiplying all relative pose matrices. After introducing the relative pose Pose 1j , all poses form a loop, and the theoretical value of multiplying all relative pose matrices should be the identity matrix. The Pose error matrix obtained by multiplying all relative pose matrices is subtracted from the theoretical value identity matrix, and all relative pose matrices are adjusted and optimized with the minimum of this difference as the objective function.

[0041] The point cloud coordinates of the target area at all acquisition locations Y are transformed to the camera coordinate system of the acquisition location y1 through the optimized relative pose matrix, and the initial point cloud stitching is completed.

[0042]

[0043] In the formula, x j , y i , z j are the three-dimensional coordinates of the point in the point cloud obtained at the acquisition location y j in the camera coordinate system of this location; x j-1 , y i-1 , z j-1 are the three-dimensional coordinates of the point in the point cloud obtained at the acquisition location y j in the camera coordinate system of the previous acquisition location y j-1 ; R1 is the optimized rigid body rotation matrix; t1 is the optimized rigid body translation matrix.

[0044] For the three-dimensional point clouds PC j and PC j-1, the sparse matching algorithm is used to perform the second stitching of the point cloud, and the relative pose matrix is optimized again. The point cloud of the target area at all acquisition locations is transformed to the camera coordinate system of the acquisition location y1 through the relatively pose matrix after the secondary optimization, and the point cloud stitching is completed.

[0045] Step S204: Post-process the point cloud on the surface of the lining plate

[0046] Apply the voxel grid filtering algorithm to sample the lining plate point cloud. The point cloud space is evenly divided into multiple cubes of size a*b*c. For the point cloud within the cube, calculate its centroid, and approximately replace the point cloud within the cube with the centroid point.

[0047] The moving least squares algorithm is used to resample and smooth the sampled point cloud, and the octree structure of the input point cloud is constructed using the nearest neighbor search method.

[0048] Step S3: Calculate the wear amount of the lining plate, specifically:

[0049] Step S301: Import the surface point cloud obtained when the new lining plate is installed.

[0050] Use the lining plate wear detection device to obtain the original surface three-dimensional point cloud P of the un-worn lining plate after installation cloud_0 , and this point cloud can also be obtained by extracting the surface of the three-dimensional digital model during the lining plate design.

[0051] Step S302: Perform point cloud alignment to unify the two point cloud coordinates.

[0052] Respectively, for the original surface three-dimensional point cloud P cloud_0 and the processed worn lining plate three-dimensional point cloud P obtained cloud_1 estimate the surface normal of each point, that is, estimate the plane normal of this point from the point set composed of k nearest points of this point. Further, unitize the plane normal as the unit surface normal of this point.

[0053] Use the improved normal transformation algorithm to calculate the pose transformation matrix of the aligned point cloud P cloud_0 and the point cloud P cloud_1 The improvement to the normal transformation algorithm is to use the three-dimensional coordinates of each point in the point cloud and the unit surface normal vector of this point as the matching point set, that is, for the point sets X p and Y p perform alignment.

[0054] X p ={x1,x2,…,x Nx}

[0055] Y p ={y1,y2,…,y Ny}

[0056] xNx = [x, y, z, x N , y N , z N T

[0057] y Nx = [x, y, z, x N , y N , z N T

[0058] where the point set X p is the point set in each voxel grid of size A * B * C obtained by uniformly dividing the point cloud P cloud_0 ; the point set Y p is all the points in the point cloud P cloud_1 ; Nx and Ny are the numbers of points in the divided point sets; x, y, z are the three-dimensional coordinates of the points; x N , y N , z N are the values of the unit surface normal vector of the point.

[0059] The improvement to the normal transformation algorithm is that the residual function is expressed as

[0060] f(T) = ‖(Ry i + t) - μ‖

[0061]

[0062] t = [t normal_0 , 0, 0, 0] T

[0063] where R normal_0 , t normal_0 are the parameters to be solved; R normal_0 is a 3x3 rotation transformation matrix; t normal_0 is a 3x1 translation transformation matrix; μ is the normal distribution parameter of each voxel grid.

[0064] Calculate the Jacobian matrix of the residual function with respect to the parameters to be solved, and iteratively optimize to obtain the parameters to be solved R normal_1 , t normal_1 , and transform the three-dimensional point cloud P cloud_1 of the worn liner to the three-dimensional point cloud P cloud_0 of the original surface of the liner in the coordinate system to complete point cloud alignment.

[0065] P cloud_2 = R normal_1 P cloud_1 + t normal_1

[0066] where P cloud_2 ​​It is the point cloud of the worn liner in the coordinate system of the surface of the unworn liner.

[0067] Step S303: Calculate the wear value by obtaining the corresponding points of the two point clouds.

[0068] Construct the original surface three-dimensional point cloud point cloud P of the unworn liner cloud_0 of the octree structure Octree. Search in Octree for the point P cloud_2 in the middle point P cloud_2_point of the Euclidean distance closest point P cloud_0_point . Taking P cloud_2_point as the starting point and P cloud_0_point as the end point to construct the vector d 02 , project this vector onto the unit surface normal direction P cloud_2_point of the point P cloud_2_point_normal , and obtain the wear value Wear at this point.

[0069] Wear = d 02 ·P cloud_2_point_normal

[0070] Traverse all points in P cloud_2 , calculate the wear value Wear of this point, and obtain the wear distribution of the worn liner.

[0071] The beneficial effects of the present invention are as follows: The present invention uses binocular vision to obtain the three-dimensional information of the liner surface in regions. When obtaining, a digital speckle pattern is projected on the liner surface, improving the texture information of the liner surface while supplementing light in the case of insufficient light in the crushing cavity. The speckle pattern provides a higher parallax estimate when obtaining the depth map from the binocular images, resulting in more accurate point cloud information of the liner surface. The present invention introduces the information of the inertial measurement unit to provide a reference for the initial stitching of the point cloud of the liner surface after regional division. After the initial stitching, a higher-precision secondary stitching is performed, improving the data acquisition efficiency while ensuring the acquisition accuracy. The present invention aligns the liner point cloud through an improved normal transformation algorithm, finds the closest points of the points in the worn point cloud in the point cloud obtained from the unworn one, calculates the distance between the two points in the unit normal direction, obtains a higher-precision liner wear amount, and obtains the wear distribution of the liner. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The schematic examples of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0073] Figure 1 It is the flow chart of the method for detecting the wear of the liner of the mining crusher of the present invention;

[0074] Figure 2 It is the flow chart of the steps for obtaining the binocular images of the target area by the method for detecting the wear of the liner of the mining crusher of the present invention.

[0075] Figure 3 It is the flow chart of the steps to obtain the three-dimensional point cloud of the surface of the lining plate for the method of detecting the wear of the lining plate of the mine crusher of the present invention;

[0076] Figure 4 It is the flow chart of the steps to calculate the wear amount of the lining plate for the method of detecting the wear of the lining plate of the crusher of the present invention. Specific embodiments

[0077] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0078] See Figures 1 to 4 As shown, the method for detecting the wear of the lining plate of the mine crusher of the present invention can obtain a higher-precision wear amount of the lining plate and obtain the wear distribution of the lining plate. A method for detecting the wear of the lining plate of a mine crusher provided by the present invention specifically includes the following steps:

[0079] Step S1: Use the device for detecting the wear of the lining plate of the mine crusher to obtain the binocular image of the target area.

[0080] See Figure 2 As shown, the specific implementation steps for obtaining the binocular image of the target area are as follows:

[0081] Step S101: Divide the detection area and determine the acquisition position.

[0082] The division of the detection area and the determination of the acquisition position are specifically implemented as follows: Determine the acquisition location Y according to the detected lining plate area X and the acquisition environment.

[0083] X = {x1, x2, x3, …, x i}

[0084] Y = {y1, y2, y3, …, y j}

[0085] Divide the detected lining plate area X into rectangular target areas x1, …, x size_a *x siez_b of size x i . Select the acquisition location y i according to the position of the target area x j , and multiple target areas x i correspond to one acquisition location yj 。

[0086] In this embodiment, image acquisition is performed on the crusher liner with an area of 2.5 * 3.4 m, and the rectangular target area X = {x1, x2, x3, …, x 30} is evenly divided, and the area of the region x size_a *x siez_b is 0.6 * 0.6 m. The acquisition locations are divided into Y = {y1, y2, y3, …, y 12} according to the position of the liner in the crusher.

[0087] Step S102: Obtain IMU data.

[0088] The IMU data is the data output by the three single-axis accelerometers, three single-axis gyroscopes, and three single-axis magnetometers of the inertial measurement unit, which can calculate the pose change of the IMU and further obtain the pose change of the binocular camera fixed to the IMU.

[0089] The IMU data comes from the inertial measurement unit fixedly connected to the binocular camera in the liner wear detection device. The liner wear detection device includes a binocular camera, an inertial measurement unit (IMU) fixedly connected to the binocular camera, a projection device, a computer as a data processing system, and a power supply that can supply power to the binocular camera, IMU, projection device, and data processing system simultaneously.

[0090] Start recording IMU data at the acquisition location y1, and continuously record IMU data when the binocular camera is transferred to all subsequent acquisition locations y i . After the binocular camera acquires images at the last acquisition location y end , make the binocular camera return to the acquisition location y1 again and record IMU data as the last acquisition, and continuously record IMU data until this location, then the acquisition of IMU data is completed.

[0091] Step S103: Use the projection device to project a digital speckle pattern onto the target area on the liner.

[0092] The digital speckle pattern is composed of random black and white pixels and is a rectangle with a size of T size_a *T siez_b . Project the digital speckle pattern onto multiple target areas x j on the corresponding liner at each acquisition location y i . The digital speckle pattern covers all target areas x j corresponding to this acquisition location y i .

[0093] In this embodiment, when the projection distance is 1980 mm, the projection pattern size T size_a*T siez_b It is 1219mm * 914mm. The resolution of the projection device is 1920 * 1080, and the light output is 4200 lumens.

[0094] Step S104: Use a binocular camera to obtain binocular images of the target area.

[0095] At the acquisition location y j While projecting onto the liner, use a binocular camera to j perform image acquisition on multiple target areas x on the liner corresponding to the acquisition location y i . Transmit the acquired binocular images to the data processing system.

[0096] In this embodiment, a total of 13 groups and 26 binocular images are acquired.

[0097] Step S2: Obtain the three-dimensional point cloud of the liner surface.

[0098] See Figure 3 As shown, the specific implementation steps for obtaining the three-dimensional point cloud of the liner surface are as follows:

[0099] Step S201: Use a stereo matching method based on Census matching cost calculation to obtain the point cloud of each target area.

[0100] In the data transmission system, for each group of binocular images (a, b) at each acquisition location y j use an improved stereo matching method for calculating the matching cost to obtain the disparity map P of each target area j . The corresponding pixel points of the same group of binocular images are located in the same row. Based on the left image a in the image group d find the corresponding matching point pairs in the same row of the right image b j j

[0101] Use the Census matching cost calculation method to calculate the matching cost C(p i ) of the pixel point p in the image i . The value of the matching cost is obtained by calculating the Hamming distance between it and the corresponding pixel point (p i , b j ) in the same group of images.

[0102] C(p i ) = H[C(p i , a j ), C(p i , b j )]

[0103] C(p i , a j ) represents the left image a jCensus transform coding of the neighborhood of pixel point p i , a j ) in the left image, and C(p i , b j ) represents the Census transform coding of the neighborhood of pixel point (p j , b i ) in the right image b j . H[,] represents calculating the Hamming distance between the two codings. Taking pixel point p i as the origin to establish a coordinate system, the coordinate points within a 7*7 coordinate window centered at the origin in the coordinate system are taken as the preset coordinate points, and the gray values (x, y) Gv of the pixel points corresponding to the preset coordinate points are obtained. Comparing each gray value with the gray value of pixel point p i , if the former is smaller, the coding position is 0, and if the former is larger, the coding position is 1. After comparing this pixel point p i with all the preset coordinate points, the Census transform coding of this pixel point is generated.

[0104] Traverse the pixel points (p j , a i ) in the left image a j to generate the Census transform coding C(p i , a j ) of the pixel points. Find all the pixel points in the same row of the right image b j in the same group, denoted as the point set of pixel point (p i , a j ). Generate the respective Census transform codings for all the pixel points in this point set. Calculate the Hamming distance between the coding of pixel point (p i , a j ) and the codings of each pixel point in its point set respectively. The pixel point corresponding to the minimum Hamming distance is the matching pixel point of pixel point (p i , a j ) in the right image b j . Combining the camera parameters, calculate the depth information for the corresponding matching pixel points of the same group of images, and obtain the three-dimensional coordinates of all the points in the left image in the camera coordinates, that is, obtain the point cloud.

[0105] Step S202: Filter the point cloud of each target area to obtain the three-dimensional point cloud on the surface of the lining board area.

[0106] For each point P cloud_point_i in the point cloud of each target area, apply a three-dimensional KD tree to search for the K nearest neighboring points around it as the search point set of this point. Calculate the Euclidean distance between P cloud_ppint_i and the points in the search point set, and calculate the average value d cloud_point_i of the Euclidean distances between P pi and all the points in the search point set;

[0107] d th_i = d pi + s

[0108] where d th_i is the density threshold of the said P cloud_point_i ; d pi is the average Euclidean distance between the said P cloud_point_i and all points in its search point set; s is the standard deviation of the average Euclidean distance between the said P cloud_point_i and the average Euclidean distances of its respective K nearest points;

[0109] Determine whether P cloud_point_i meets the filtering condition, and filter out the points that meet the filtering condition; the filtering condition is: count the number of points in the search point set of this P cloud_point_i whose Euclidean distance is less than d th_i , determine whether this value is less than 0.84K, and determine whether the average value d cloud_point_i of the Euclidean distance between P pi and its search point set is greater than the density threshold;

[0110] In this embodiment, the number of neighboring points K = 45.

[0111] Step S203: Complete point cloud stitching in combination with IMU data.

[0112] Obtain the relative pose Pose k of the binocular camera at the acquisition location y k-1 through the IMU data; kk-1 ;

[0113]

[0114] where R0 is a 3x3 rigid body rotation matrix and t0 is a 3x1 rigid body translation matrix, both calculated from the IMU data;

[0115] The relative pose of the binocular camera at the last acquisition and the previous acquisition is Pose jj-1 ; Match the point cloud of the target area collected at the last time with the point cloud of the target area collected at the first time, and calculate to obtain the relative pose Pose 1j of the camera;

[0116] Pose 21 Pose 32 …Pose jj-1 Pose 1j = Pose error

[0117] In the formula, Pose erroris the value obtained by multiplying all relative pose matrices; add the relative pose Pose 1j After that, all poses form a loop, and the theoretical value of multiplying all relative pose matrices should be the identity matrix; the Pose error matrix obtained by multiplying all relative pose matrices is subtracted from the theoretical identity matrix, and all relative pose matrices are adjusted and optimized with the minimum of this difference as the objective function;

[0118] The point cloud coordinates of the target area at all collection locations Y are transformed to the camera coordinate system of the collection location y1 through the optimized relative pose matrix to complete the initial point cloud stitching;

[0119]

[0120] where x j 、y i 、z j are the three-dimensional coordinates of the point in the point cloud obtained at the collection location y j in the camera coordinate system of this location; x j-1 、y i-1 、z j-1 are the three-dimensional coordinates of the point in the point cloud obtained at the collection location y j in the camera coordinate system of the previous collection location y j-1 ; R1 is the optimized rigid body rotation matrix; t1 is the optimized rigid body translation matrix.

[0121] For the three-dimensional point clouds PC j and PC j-1 of two adjacent collection locations after the initial stitching is completed, use the sparse matching algorithm to perform the second stitching on the point cloud and optimize the relative pose matrix again. The point clouds of the target areas at all collection locations are transformed to the camera coordinate system of the collection location y1 through the second optimized relative pose matrix to complete the point cloud stitching.

[0122] Step S204: Post-process the point cloud on the liner surface

[0123] Apply the voxel grid filtering algorithm to sample the liner point cloud. The point cloud space is evenly divided into multiple cubes of size a*b*c, and for the point cloud within the cube, calculate its centroid and approximately replace the point cloud within the cube with this centroid point.

[0124] Use the moving least squares algorithm to resample and smooth the sampled point cloud, and the neighborhood search method uses the octree structure constructed for the input point cloud.

[0125] In this embodiment, the size a*b*c of the small cube divided in the point cloud space is 0.5*0.5*0.5 mm. In the moving least squares algorithm, set the parameter to use a second-order polynomial for fitting, set the fitting search method to octree, and the value of the number k of neighboring points searched is 6.

[0126] Step S3: Calculate the wear amount of the liner.

[0127] See Figure 4 As shown, the specific implementation steps for calculating the wear amount of the liner are as follows:

[0128] Step S301: Import the surface point cloud obtained when the new liner is installed.

[0129] Use the liner wear detection device to obtain the original surface three-dimensional point cloud P of the un-worn liner after installation cloud_0 , and this point cloud can also be obtained by extracting from the surface of the three-dimensional digital model during the liner design.

[0130] Step S302: Perform point cloud alignment to unify the coordinates of the two point clouds.

[0131] Respectively estimate the surface normal of each point for the original surface three-dimensional point cloud P cloud_0 and the processed worn liner three-dimensional point cloud P cloud_1 , that is, estimate the plane normal of this point from the point set composed of k nearest points of this point. Further, unitize this plane normal as the unit surface normal of this point.

[0132] Use the improved normal transformation algorithm to calculate the pose transformation matrix of the aligned point cloud P cloud_0 and the point cloud P cloud_1 . The improvement to the normal transformation algorithm is to use both the three-dimensional coordinates of each point in the point cloud and the unit surface normal vector of this point as the matching point set, that is, align the point sets X p and Y p .

[0133] X = {x1, x2,..., x Nx}

[0134] Y = {y1, y2,..., y Ny}

[0135] x Nx = [x, y, z, x N , y N , z N T

[0136] y Nx = [x, y, z, x N , y N , z N T

[0137] Among them, the point set X p is the point set in each voxel grid of size A * B * C obtained by uniformly dividing the point cloud P cloud_0 ; the point set Yp is the point cloud P cloud_1 For all points Nx and Ny in it, where Nx and Ny are the numbers of points in the divided point sets; x, y, z are the three-dimensional coordinates of the points; x N , y N , z N is the value of the unit surface normal vector of the point.

[0138] The improvement to the normal transformation algorithm is that the residual function is expressed as

[0139] f(T) = ‖(Ry i + t) - μ‖

[0140]

[0141] t = [t normal_0 , 0, 0, 0] T

[0142] where R normal_0 , t normal_0 are the parameters to be solved; R normal_0 is a 3x3 rotation transformation matrix; t normal_0 is a 3x1 translation transformation matrix; μ is the normal distribution parameter of each voxel grid.

[0143] Calculate the Jacobian matrix of the residual function with respect to the parameters to be solved, and iteratively optimize to obtain the parameters to be solved R normal_1 , t normal_1 , and transform the three-dimensional point cloud P cloud_1 of the worn liner to the three-dimensional point cloud P cloud_0 of the original surface of the liner in the coordinate system to complete the point cloud alignment.

[0144] P cloud_2 = R normal_1 P cloud_1 + t normal_1

[0145] where P cloud_2 is the point cloud of the worn liner in the coordinate system of the surface of the unworn liner.

[0146] In this embodiment, the size A*B*C of the voxel grid obtained by uniformly dividing the point cloud P cloud_0 is 0.2*0.2*0.2 m.

[0147] Step S303: Calculate the corresponding points of the two point clouds to obtain the wear value.

[0148] Construct the octree structure Octree of the three-dimensional point cloud P cloud_0 of the original surface of the unworn liner. Search for the point P cloud_2 in Octree that is the closest to the point P cloud_2_point in terms of Euclidean distance.cloud_0_point Starting from P cloud_2_point as the starting point and P cloud_0_point as the ending point, construct a vector d 02 . Project this vector onto the unit surface normal direction P cloud_2_point of point P cloud_2_point_normal to obtain the wear value Wear at this point.

[0149] Wear = d 02 ·P cloud_2_point_normal

[0150] Traverse all points in P cloud_2 to calculate the wear value Wear of each point and obtain the wear distribution of the worn liner.

[0151] The above are only the preferred examples of the present invention and are not used to limit the present invention. For those skilled in the art, various modifications and changes can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made to the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting the wear of crusher liners for mining, characterized in that: It includes the following steps: Step S1: Obtain a binocular image of the target area using a lining wear detection device. Specifically: Step S101: Divide the detection area and determine the acquisition position: Determine the acquisition location Y according to the detected lining area X and the acquisition environment; X = {x1, x2, x3, …, x i} Y = {y1, y2, y3, …, y j} Divide the detected liner area X into rectangular target areas x1, …, x of size x size_a *x siez_b ; Select the acquisition location y according to the position of the target area x i ; Multiple target areas x i correspond to one acquisition location y j ; i j ;​ Step S102: Obtain IMU data: The IMU is an inertial measurement unit; the IMU data is the data output by the accelerometers of three single axes, the gyroscopes of three single axes, and the magnetometers of three single axes of the inertial measurement unit, which can calculate the pose change of the IMU and further obtain the pose change of the binocular camera fixed to the IMU; The IMU data comes from the inertial measurement unit fixedly connected to the binocular camera in the lining wear detection device; the lining wear detection device includes a binocular camera, an IMU fixedly connected to the binocular camera, a projection device that can project patterns, a computer as a data processing system, and a power supply that can supply power to the binocular camera, IMU, projection device, and data processing system simultaneously; Start recording IMU data at the acquisition location y1 and continue recording IMU data while transferring the binocular camera to all subsequent acquisition locations y i ; At the last acquisition location y end After the binocular camera captures images, make the binocular camera return to the acquisition location y1 again and record IMU data. As the last acquisition, continue recording IMU data until this location and then complete the acquisition of IMU data; Step S103: Use the projection device to project a digital speckle pattern onto the target area on the lining; The digital speckle pattern consists of random black and white pixels and is a rectangle of size T size_a *T siez_b At each acquisition location y j Project the digital speckle pattern onto a plurality of target regions x on the corresponding backing plate i The digital speckle pattern covers all target regions x corresponding to the acquisition location y j ; i ; Step S104: Use the binocular camera to obtain a binocular image of the target area; At the collection location y j While projecting onto the lining plate, use a binocular camera to capture the collection location y j Multiple target areas x on the corresponding lining plate i Perform image acquisition; transmit the acquired binocular images to the data processing system; Step S2: Obtain the three-dimensional point cloud of the lining surface; Step S3: Calculate the lining wear amount.

2. The method for detecting the wear of the lining plate of the mine crusher according to claim 1, characterized in that: The obtaining of the three-dimensional point cloud of the lining surface in Step S2 is specifically: Step S201: Use a stereo matching method based on Census matching cost calculation to obtain the point cloud of each target area; In a data transmission system, for each acquisition location y j of the binocular image group (a, b) j An improved stereo matching method for calculating matching cost is adopted to obtain the disparity map P of each target area d ; The corresponding pixel points of the same binocular image group are located in the same row; Based on the left image a in the image group j Find the corresponding matching point pairs in the right image b j in the same row; Calculate the matching cost C(p) of the pixel point p in the image by using the Census matching cost calculation method i of the matching cost C(p i ), and the value of the matching cost is obtained by calculating the Hamming distance between it and the corresponding pixel point (p i , b j ) in the same group of images; C(p i ) = H[C(p i , a j ), C(p i , b j )] C(p i , a j ) represents the Census transform coding of the neighborhood of the pixel point (p j , a i ) in the left image a j ), C(p i , b j ) represents the Census transform coding of the neighborhood of the pixel point (p j , b i ) in the right image b j ), H[,] represents calculating the Hamming distance between the two codings; taking the pixel point p i as the origin to establish a coordinate system, and taking the coordinate points within a 7*7 coordinate window centered at the origin in the coordinate system as the preset coordinate points, obtaining the gray values (x, y) of the pixel points corresponding to the preset coordinate points Gv ; comparing each gray value with the gray value of the pixel point p i , if the former is smaller, the coding position is 0, if the former is larger, the coding position is 1; after comparing the pixel point p i with all the preset coordinate points, generating the Census transform coding of this pixel point; Traverse the left image a j in the pixel point (p i , a j ), generate the Census transform code C(p i , a j ), find all the pixel points in the same row of the right image b j in the same group, denoted as the point set of the pixel point (p i , a j ), generate their respective Census transform codes for all the pixel points in this point set; calculate the Hamming distance between the code of the pixel point (p i , a j ) and the codes of each pixel point in its point set, and the pixel point corresponding to the minimum Hamming distance is the matching pixel point of the pixel point (p i , a j ) in the right image b j ; combine the camera parameters to calculate the depth information of the corresponding matching pixel points of the same group of images, and obtain the three-dimensional coordinates of all points in the left image in the camera coordinates, that is, obtain the point cloud; Step S202: Filter the point cloud of each target area to obtain the three-dimensional point cloud of the lining area surface; For each point P in the point cloud of each target region cloud_point_i Apply a 3D KD-tree to search for the K nearest neighboring points as the search point set for this point; calculate P cloud_point_i The Euclidean distance from the points in the search point set, calculate P cloud_point_i The average value d of the Euclidean distances from P to all the points in the search point set pi ; d th_i = d pi + s where d th_i is the density threshold of the P cloud_point_i ; d pi is the average of the Euclidean distances between the P cloud_point_i and all points in its search point set; s is the standard deviation of the average of the Euclidean distances between the P cloud_point_i and the average of the Euclidean distances to their respective K nearest points; Determine P cloud_point_i whether it meets the filtering condition, and filter out the points that meet the filtering condition; the filtering condition is: count the P cloud_point_i whose Euclidean distance from its search point set is less than d th_i the number of points, whether it is less than 0.84K, and determine P cloud_point_i the average value d of its Euclidean distance from the search point set pi whether it is greater than the density threshold; Step S203: Complete point cloud stitching in combination with IMU data: Obtain the acquisition location y through IMU data k For y k-1 the relative pose Pose of the binocular camera kk-1 ; Where R0 is a 3x3 rigid body rotation matrix and t0 is a 3x1 rigid body translation matrix, both of which are calculated from the IMU data; The relative pose of the binocular camera at the last acquisition and the previous acquisition is Pose jj-1 ; Match the point cloud of the target area collected at the last time with the point cloud of the target area collected at the first time, and calculate the relative pose Pose of the camera 1j ; Pose 21 Pose 32 …Pose jj-1 Pose 1j =Pose error where Pose error is the value obtained by multiplying all relative pose matrices; after adding the relative pose Pose 1j , all poses form a loop, and the theoretical value of multiplying all relative pose matrices should be the identity matrix; subtract the Pose error matrix obtained by multiplying all relative pose matrices from the theoretical value identity matrix, and adjust and optimize all relative pose matrices with the minimum of this difference as the objective function; Transform the point cloud coordinates of the target area at all acquisition locations Y to the camera coordinate system of the acquisition location y1 through the optimized relative pose matrix to complete the initial point cloud stitching; where x j , y i , z j are the three-dimensional coordinates of the point in the point cloud obtained at the acquisition location y j in the camera coordinate system of that location; x j-1 , y i-1 , z j-1 are the three-dimensional coordinates of the point in the point cloud obtained at the acquisition location y j in the camera coordinate system of the previous acquisition location y j-1 ; R1 is the optimized rigid body rotation matrix; t1 is the optimized rigid body translation matrix; For the three-dimensional point cloud PC of two adjacent acquisition locations that have completed the initial splicing j and PC j-1 , use the sparse matching algorithm to perform the second splicing of the point cloud and optimize the relative pose matrix; transform the point cloud of the target area of all acquisition locations to the camera coordinate system of the acquisition location y1 through the optimized relative pose matrix to complete the point cloud splicing; Step S204: Post-process the point cloud of the lining surface: Sample the point cloud of the lining using a voxel grid filtering algorithm; evenly divide the point cloud space into multiple cubes of size a*b*c, and for the point cloud within the cube, calculate its centroid and approximate the point cloud within the cube with the centroid point; Use the moving least squares algorithm to resample and smooth the sampled point cloud, and the nearest neighbor search method uses the construction of an octree structure of the input point cloud.

3. The method for detecting the wear of the lining plate of the mine crusher according to claim 1, characterized in that: The calculation of the lining wear amount in Step S3 is specifically: Step S301: Import the surface point cloud obtained when the new lining is installed; Obtain the original surface three-dimensional point cloud P of the liner after installation and before wear using the liner wear detection device cloud_0 , and this point cloud can also be obtained by extracting the surface of the three-dimensional digital model during the liner design; Step S302: Perform point cloud alignment to unify the two point cloud coordinates: Respectively for the original surface three-dimensional point cloud P cloud_0 and the obtained processed worn lining three-dimensional point cloud P cloud_1 Estimate the surface normal of each point, that is, estimate the plane normal of the point from the point set composed of k nearest points of the point; further, unitize the plane normal as the unit surface normal of the point; Calculate the pose transformation matrix of the aligned point cloud P using an improved normal transformation algorithm cloud_0 and the point cloud P cloud_1 ; The improvement to the normal transformation algorithm is to use the three-dimensional coordinates of each point in the point cloud and the unit surface normal vector of the point as the matching point set, that is, align the point sets X′ P and Y P ′ X′ P = {x′1, x′2, …, x′ Nx} Y′ P = {y′1, y′2, …, y′ Ny} x′ Nx = [x′, y′, z′, x′ N , y′ N , z′ N T ​ y′ Ny = [x′, y′, z′, x′ N , y′ N , z′ N T ​ Among them, the point set X' P is the point set in each voxel grid of size A*B*C obtained by evenly dividing the point cloud P cloud_0 ; the point set Y P ' is all the points in the point cloud P cloud_1 ; Nx and Ny are the numbers of points in the divided point sets; x', y', z' are the three-dimensional coordinates of the points; x' N , y' N , z' N are the values of the unit surface normal vector of the point The improvement of the normal transformation algorithm is that the residual function is expressed as f(T) = ‖(Ry i ′ + t) - μ‖ t = [t normal_0 , 0, 0, 0] T where y i ′ is the midpoint set Y P ′ point; R normal_0 , t normal_0 are parameters to be determined; R normal_0 is a 3x3 rotation transformation matrix; t normal_0 is a 3x1 translation transformation matrix; μ is the normal distribution parameter of each voxel grid; Calculate the Jacobian matrix of the residual function with respect to the parameters to be solved, and iteratively optimize to obtain the parameters R normal_1 , t normal_1 , and transform the three-dimensional point cloud P of the worn liner cloud_1 to the three-dimensional point cloud P of the original surface of the liner cloud_0 in the coordinate system to complete the point cloud alignment; P cloud_2 = R normal_1 P cloud_1 + t normal_1 Among them, P cloud_2 is the point cloud of the worn liner in the coordinate system of the surface of the unworn liner; Step S303: Calculate the corresponding points of the two point clouds to obtain the wear value: Construct the original surface three-dimensional point cloud Point Cloud P of the unworn liner cloud_0 of the octree structure Octree; Search in Octree for the point P cloud_2 in the middle point P cloud_2_point of the closest Euclidean distance point P cloud_0_point ; Taking P cloud_2_point as the starting point and P cloud_0_point as the ending point to construct the vector d 02 , project this vector onto the unit surface normal direction P cloud_2_point of the point P cloud_2_point_normal , and obtain the wear value Wear at this point; Wear=d 02 ·P cloud_2_point_normal Traverse all points in P cloud_2 Calculate the wear value Wear of each point and obtain the wear distribution of the worn liner.

Images