Mine steel wire rope space vibration displacement measurement method and device based on depth camera

Abstract

The embodiment of the invention provides a mine steel wire rope space vibration displacement measurement method and device based on a depth camera. The method comprises the steps of obtaining a depth image of a steel wire rope; pixel points in the depth image are converted into initial point cloud data; performing point cloud simplification on the segmented point cloud data to obtain three-dimensional point cloud data of the steel wire rope; calculating the three-dimensional point cloud data to obtain a center point coordinate when the steel wire rope is in a static state; and calculating the space vibration displacement of the central point of the steel wire rope in the static reference coordinate system. Therefore, the problem of inaccurate measurement caused by lack of depth information is solved, and relatively high measurement precision and stability can be kept in a complex mine environment; the shape difference between the steel wire rope and the background can be effectively highlighted, high-precision steel wire rope point cloud segmentation is realized, and the method is suitable for steel wire rope monitoring in a multi-interference environment; the point cloud data volume can be effectively reduced, meanwhile, key geometric features are kept, and simplified and accurate input data are provided for steel wire rope vibration analysis.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of machine vision and mine safety monitoring, and relate to but are not limited to a method and device for measuring the spatial vibration displacement of a mine wire rope based on a depth camera. Background Art

[0002] In the mine hoisting system, the suspended wire rope between the drum and the sheave will vibrate under the influence of external factors and end excitation during normal operation. With the increase of operating speed, hoisting depth and load, the vibration energy of the wire rope will also increase accordingly. The wire rope vibration will cause the lateral vibration of the vertical wire rope through the sheave, which not only affects the safety of the hoisting system, but also aggravates the wear of the wire rope and the sheave, shortens the service life, and in severe cases may even cause the wire rope to detach from the sheave, causing a major accident. Therefore, effective measurement and control of wire rope vibration is crucial for the safe and stable operation of the mine hoisting system.

[0003] At present, displacement sensors and two-dimensional cameras are widely used at home and abroad to measure the lateral vibration of wire ropes. However, since the wire rope is in a state of continuous motion, the installation and configuration of the sensor are quite difficult. Traditional two-dimensional cameras lack spatial depth information, making it difficult to accurately measure the distance between the wire rope and the camera, and thus unable to accurately capture the spatial vibration of the wire rope. In addition, two-dimensional cameras are also easily affected by changes in lighting, and when measuring multiple wire ropes, it is difficult to achieve effective measurement due to mutual occlusion between the wire ropes. Although laser displacement sensors can provide relatively accurate measurements, their measurement range is limited, and usually require labels to be added to the wire rope, which increases the complexity and uncertainty of the measurement. Summary of the invention

[0004] Based on the problems in the related art, an embodiment of the present invention provides a method and device for measuring the spatial vibration displacement of a mine wire rope based on a depth camera.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides a method for measuring the spatial vibration displacement of a mine wire rope based on a depth camera, the method comprising:

[0007] The depth image of the wire rope is obtained by using a binocular structured light camera;

[0008] Based on the internal and external parameters of the binocular structured light camera, the pixel points in the depth image are converted into corresponding three-dimensional space coordinates to obtain initial point cloud data;

[0009] The initial point cloud data after the straight-through filtering is segmented by a region generation algorithm based on curvature priority to obtain segmented point cloud data;

[0010] The segmented point cloud data is simplified by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope;

[0011] The three-dimensional point cloud data is calculated by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state;

[0012] The spatial vibration displacement of the center point of the wire rope in a static reference coordinate system is calculated by using a dynamic point tracking method; the static reference coordinate system is established with the center point coordinates as the origin.

[0013] An embodiment of the present invention provides a device for measuring the spatial vibration displacement of a mine wire rope based on a depth camera, the device comprising:

[0014] An acquisition module, used for acquiring a depth image of the wire rope through a binocular structured light camera;

[0015] A conversion module, used to convert the pixel points in the depth image into corresponding three-dimensional space coordinates based on the intrinsic parameters and extrinsic parameters of the binocular structured light camera to obtain initial point cloud data;

[0016] A point cloud segmentation module is used to perform point cloud segmentation on the initial point cloud data after the straight-through filtering by using a region generation algorithm based on curvature priority to obtain segmented point cloud data;

[0017] A point cloud simplification module, used to simplify the segmented point cloud data by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope;

[0018] A calculation module, used to calculate the three-dimensional point cloud data by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state;

[0019] The calculation module is also used to calculate the spatial vibration displacement of the center point of the wire rope in a static reference coordinate system using a dynamic point tracking method; the static reference coordinate system is established with the center point coordinates as the origin.

[0020] In some embodiments, the point cloud segmentation module is also used to remove the point cloud data in the initial point cloud data that exceeds the threshold range based on the preset X, Y and Z axis threshold range, so as to obtain the point cloud data after the through filtering; and perform point cloud segmentation on the initial point cloud data after the through filtering through a curvature-first region generation algorithm to obtain the segmented point cloud data.

[0021] In some embodiments, the calculation module is also used to perform principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each segment of the wire rope to obtain the distribution direction of the wire rope and calculate the center point coordinates of each segment of the fitting straight line; the center point coordinates of multiple segments of the fitting straight line are weighted averaged to obtain the center point coordinates of the wire rope when it is in a static state.

[0022] In some embodiments, the calculation module is also used to obtain the three-dimensional coordinates of the initial center point of the wire rope when it is in a static state, and use the three-dimensional coordinates of the initial center point as the initial reference point and the initial position of the tracking target; calculate the three-dimensional coordinates of the corresponding center point of the wire rope in each frame of the depth image, and use the three-dimensional coordinates of the corresponding center point as the current position of the tracking target; use the dynamic point tracking method to calculate the displacement of the current position relative to the initial position, that is, to obtain the spatial vibration displacement.

[0023] In some embodiments, the point cloud segmentation module is also used to calculate the curvature of each point in the point cloud data after the straight-through filtering based on a preset area generation algorithm, and use points with higher curvature and satisfying distance and normal vector constraints as seed points; starting from the seed point, gradually merge the points in the neighborhood of the seed point that satisfy the preset similarity criteria into the same area, and determine that the generation of the area can capture the shape characteristics of the wire rope; use the points newly added to the area as new seed points, and repeat the area generation until all points that satisfy the preset similarity criteria are merged; based on the preset minimum cluster threshold and the preset maximum cluster threshold, eliminate areas that are too small or too large, and determine the qualified areas as the point cloud data after the wire rope is segmented.

[0024] In some embodiments, the point cloud simplification module is also used to determine the optimal initial voxel size using a preset particle swarm optimization algorithm; use a preset K-nearest neighbor algorithm to estimate the local point cloud density of each point in the segmented point cloud data; based on each of the local point cloud densities, divide the segmented point cloud data into areas with different density levels, and adaptively adjust the voxel size based on the density level; calculate the center of mass of each point in each voxel, and form the three-dimensional point cloud data based on the center of mass.

[0025] In some embodiments, before performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the wire rope to obtain the distribution direction of the wire rope, the device also includes: a division module, used to divide the three-dimensional point cloud data into a number of sections with equal distances along the length direction of the wire rope; the determination module is also used to perform principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the wire rope to obtain the main direction vector of the wire rope; based on the main direction vector and the center point of the three-dimensional point cloud data of each section of the wire rope, construct a spatial straight line equation, and use the least squares method to fit to obtain the optimal straight line parameters; based on the optimal straight line parameters, calculate the center point coordinates.

[0026] An embodiment of the present invention provides a depth camera-based mine wire rope spatial vibration displacement measurement device, comprising: a memory for storing executable instructions; a processor for executing the executable instructions stored in the memory to implement the above-mentioned depth camera-based mine wire rope spatial vibration displacement measurement method.

[0027] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned depth camera-based mine wire rope spatial vibration displacement measurement method.

[0028] The method and device for measuring the spatial vibration displacement of a mine wire rope based on a depth camera provided by an embodiment of the present invention adopts a binocular structured light camera to collect the depth data of the wire rope, projects structured light on the surface of the wire rope, and uses the binocular stereo vision principle to capture the speckle pattern image sequence formed on the surface of the wire rope. The embedded vision processor is based on an image stereo matching algorithm and combined with a triangulation method to accurately calculate the depth data of each point on the surface of the wire rope, thereby solving the problem of inaccurate measurement caused by lack of depth information and maintaining high measurement accuracy and stability in a complex mine environment. On the other hand, in the process of point cloud segmentation, a region growing method based on curvature priority is proposed. The algorithm calculates the curvature and normal vector of the point cloud data after pass filtering, which can effectively highlight the shape difference between the wire rope and the background and realize high-precision wire rope point cloud segmentation, which is suitable for wire rope monitoring in multi-interference environments; thirdly, the adaptive voxel grid simplification method based on point cloud density introduced can effectively reduce the amount of point cloud data while maintaining key geometric features, providing simplified and accurate input data for subsequent wire rope vibration analysis; fourthly, the proposed piecewise linear fitting method based on PCA can accurately calculate the coordinates of the center point of the wire rope, providing a reliable basis for subsequent vibration displacement calculations, and effectively improving the accuracy and stability of vibration measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1It is a flow chart of a method for measuring spatial vibration displacement of a mine wire rope based on a depth camera provided by the present invention;

[0030] Figure 2 is a schematic diagram of a wire rope vibration displacement measurement system provided by the present invention;

[0031] Figure 3 It is a schematic diagram of the coordinate system conversion relationship provided by the present invention;

[0032] Figure 4 It is a flow chart of the preprocessing of the wire rope area point cloud data provided by the present invention;

[0033] Figure 5 It is a simplified visualized point cloud schematic diagram of the wire rope point cloud provided by the present invention;

[0034] Figure 6 It is a three-dimensional schematic diagram after the point cloud straight line fitting provided by the present invention;

[0035] Figure 7 It is a calculation flow chart for measuring wire rope vibration displacement provided by the present invention;

[0036] FIG8 is a graph showing the wire rope vibration displacement-time curve obtained by the laser displacement sensor and the camera provided by the present invention;

[0037] Fig. 9 A schematic diagram of the composition structure of a mine wire rope spatial vibration displacement measuring device based on a depth camera provided in an embodiment of the present invention;

[0038] Fig.10 A schematic diagram of the composition structure of a mine wire rope spatial vibration displacement measurement device based on a depth camera provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0040] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as those commonly understood by those skilled in the art to which the embodiments of the present invention pertain. The terms used in the embodiments of the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0041] The following describes an exemplary application of a depth camera-based mine wire rope spatial vibration displacement measuring device according to an embodiment of the present invention. The depth camera-based mine wire rope spatial vibration displacement measuring device provided by the embodiment of the present invention can be implemented as a terminal or a server. In one implementation, the depth camera-based mine wire rope spatial vibration displacement measuring device provided by the embodiment of the present invention can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, and mobile devices; in another implementation, the depth camera-based mine wire rope spatial vibration displacement measuring device provided by the embodiment of the present invention can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present invention. Below, an exemplary application of a depth camera-based mine wire rope spatial vibration displacement measuring device when implemented as a server will be described.

[0042] The embodiment of the present invention provides a method for measuring the spatial vibration displacement of a mine wire rope based on a depth camera, see Figure 1 , Figure 1 is a flow chart of a method for measuring the spatial vibration displacement of a mine wire rope based on a depth camera provided by an embodiment of the present invention, which will be combined with Figure 1 The steps shown are explained.

[0043] Step S110, acquiring a depth image of the wire rope through a binocular structured light camera.

[0044] In some embodiments, the depth image of the wire rope is collected by a binocular structured light camera, which can be placed near the drum in the winch room, and the camera optical axis is perpendicular to the central axis of the wire rope.

[0045] In some embodiments, the depth image of the wire rope mainly reflects the depth information of the wire rope surface in three-dimensional space. Each pixel of the depth image contains not only color or grayscale information, but also the depth information of the point relative to a reference plane in three-dimensional space. This image can intuitively show the details of the wire rope surface such as unevenness and wear marks, and provide basic data for subsequent image processing and analysis.

[0046] Step S120: Based on the intrinsic parameters and extrinsic parameters of the binocular structured light camera, the pixel points in the depth image are converted into corresponding three-dimensional space coordinates to obtain initial point cloud data.

[0047] In some embodiments, the internal parameters of the binocular structured light camera are key parameters in the binocular camera imaging process, which describe the internal characteristics and imaging laws of the camera. These internal parameters may include: focal length, optical center, distortion parameters. The external parameters of the binocular structured light camera describe the spatial geometric position relationship between the binocular cameras, that is, the rotation and translation relationship between the two camera imaging planes. These external parameters may include: rotation matrix, translation vector.

[0048] In some embodiments, each pixel point in the collected depth image is mapped into three-dimensional space, and the pixel points in the depth image are converted into corresponding three-dimensional space coordinates using the intrinsic and extrinsic parameters of the camera to generate point cloud position information including each point.

[0049] Step S130 , performing point cloud segmentation on the initial point cloud data after the straight-through filtering by using a region generation algorithm based on curvature priority to obtain segmented point cloud data.

[0050] In some embodiments, the initial point cloud data of the wire rope may be subjected to through-filtering to obtain through-filtered point cloud data. Here, the purpose of through-filtering is to remove point cloud data that is not in the region of interest, thereby reducing the amount of calculation for subsequent processing.

[0051] In some embodiments, the point cloud data after the through filtering may be segmented to obtain segmented point cloud data. Here, the purpose of performing point cloud segmentation is to segment the point cloud data into multiple parts to facilitate data processing and analysis.

[0052] In some embodiments, if the surface characteristics of the wire rope (such as color, reflectivity, etc.) are significantly different from the surrounding environment, attribute-based segmentation methods (such as color segmentation, curvature segmentation, etc.) can be used. If the surface characteristics of the wire rope are not significantly different from the surrounding environment, more complex algorithms need to be used, such as methods based on region generation, clustering, or graph theory.

[0053] In the present invention, a region generation algorithm based on curvature priority is proposed, that is, a point cloud segmentation method combining curvature segmentation and region generation. The point cloud data after straight-through filtering of the wire rope is segmented, which can ensure that the generation of the region can accurately capture the shape characteristics of the wire rope, effectively highlight the shape difference between the wire rope and the background, and realize high-precision wire rope point cloud segmentation, which is more suitable for wire rope monitoring in multi-interference environments.

[0054] Step S140, performing point cloud simplification on the segmented point cloud data by using an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope.

[0055] In some embodiments, the segmented point cloud data may be simplified to obtain three-dimensional point cloud data of the wire rope. Here, the purpose of performing point cloud simplification is to reduce the number of points in the point cloud while maintaining its shape and features as much as possible to reduce the amount of calculation and storage requirements.

[0056] In some embodiments, the point cloud simplification method includes at least: voxel gridding, random sampling, and farthest point sampling. In the present invention, an adaptive voxel grid simplification method based on point cloud density is introduced, which can effectively reduce the amount of point cloud data while maintaining key geometric features, providing simplified and accurate input data for subsequent processing of wire rope vibration analysis.

[0057] Step S150, calculating the three-dimensional point cloud data by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state.

[0058] In some embodiments, the three-dimensional point cloud data of the wire rope is projected onto the main direction found by PCA, i.e., the longest eigenvector. The distance from each point to the vector (i.e., the projection length) and the position in this direction are calculated. The projected point cloud data is divided into multiple segments according to the bending degree of the wire rope or the preset segment length. Linear regression or other suitable fitting methods are used for each segment to find the best fitting line. The center point coordinates of each segment of the fitting line are calculated, and the weighted average of the multiple center point coordinates is performed to obtain the center point coordinates when the wire rope is in a static state.

[0059] In this invention, a piecewise linear fitting method based on PCA is proposed to extract the center point of the wire rope. The wire rope is divided into equal segments along the length direction, and the PCA dimension reduction is performed on each point cloud segment to find the main direction vector of the main direction, and the optimal straight line parameters are fitted using the least squares method. This method can accurately calculate the coordinates of the center point of the wire rope, providing a reliable basis for the subsequent vibration displacement calculation, and effectively improving the accuracy and stability of vibration measurement.

[0060] Step S160, using a dynamic point tracking method, calculates the spatial vibration displacement of the center point of the wire rope in a static reference coordinate system, wherein the static reference coordinate system is established with the center point coordinates as the origin.

[0061] In some embodiments, first, it is necessary to determine the coordinates of the center point of the wire rope when it is in a static state, where the static state may refer to the wire rope vibrating less or being in a static state for a period of time. A three-dimensional static reference coordinate system is established with this static center point as the origin. In practical applications, the axis of the static coordinate system can be aligned and oriented with the main direction of the wire rope as needed.

[0062] In some embodiments, for each frame or each time point of the point cloud data, an appropriate algorithm (such as PCA combined with clustering, least squares fitting, etc.) is used to track the center point of the wire rope. This center point will be the "center of mass" or "average position" of all the point cloud data of the wire rope at the current time point.

[0063] In some embodiments, the above step S130 can be implemented by the following steps S131 to S132:

[0064] Step S131 : based on the preset threshold ranges of the X, Y and Z axes, point cloud data exceeding the threshold ranges in the initial point cloud data are removed to obtain the point cloud data after the through filtering.

[0065] In the present invention, according to the set threshold ranges of the X, Y and Z axes, the point cloud data beyond the specified ranges are removed to extract the point cloud within the region of interest where the wire rope is located.

[0066] Step S132, performing point cloud segmentation on the initial point cloud data after the straight-through filtering by using a region generation algorithm based on curvature priority to obtain segmented point cloud data.

[0067] In the present invention, a region growing algorithm based on curvature priority is adopted. By calculating the curvature of the point cloud, points with higher curvature are preferentially selected as seed points for region growing. By using the similarity of normal vector and curvature, points that meet the conditions are gradually gathered, and finally the point cloud data of the wire rope is segmented.

[0068] In some embodiments, the above step S132 may be implemented through steps S1321 to S1324:

[0069] Step S1321, based on a preset region generation algorithm, calculate the curvature of each point in the point cloud data after the straight-through filtering, and use the points with higher curvature and satisfying the distance and normal vector constraints as seed points.

[0070] In the present invention, the curvature of each point is calculated for the point cloud data after through-filtering, and points with higher curvature and satisfying distance and normal vector constraints are preferentially selected as seed points.

[0071] Step S1322, starting from the seed point, gradually merge the points in the neighborhood of the seed point that meet the preset similarity criteria into the same area, and determine whether the generation of the area can capture the shape characteristics of the wire rope.

[0072] In the present invention, starting from the selected seed point, points in its neighborhood that meet the similarity criteria (including features such as normal vector, distance and curvature) are gradually merged into the same region to ensure that the growth of the region can accurately capture the shape characteristics of the wire rope.

[0073] Step S1323: taking the points newly added to the region as new seed points, and repeating the region generation until all points satisfying the preset similarity criterion are merged.

[0074] In the present invention, the points newly added to the region are used as new seed points, and the region growth is repeated until all points that meet the conditions are merged and the region no longer grows.

[0075] Step S1324: based on a preset minimum cluster threshold and a preset maximum cluster threshold, remove areas that are too small or too large, and determine the areas that meet the conditions as the point cloud data after the wire rope segmentation.

[0076] In the present invention, based on the preset thresholds of the minimum cluster and the maximum cluster, the too small or too large area is eliminated, and finally the area that meets the conditions is retained as the point cloud data of the wire rope.

[0077] In some embodiments, the above step S140 may be implemented by steps S141 to S144:

[0078] Step S141, using a preset particle swarm optimization algorithm to determine the optimal initial voxel size.

[0079] In the present invention, a particle swarm optimization algorithm (PSO) is used to search for the optimal initial voxel size, balancing the streamlining efficiency and accuracy.

[0080] Step S142, using a preset K nearest neighbor algorithm to estimate the point cloud local density of each point in the segmented point cloud data.

[0081] In the present invention, the K-nearest neighbor algorithm (K-NN) is used to estimate the local density of each point, providing a basis for adaptive voxel partitioning.

[0082] Step S143, based on the local density of each point cloud, the segmented point cloud data is divided into regions with different density levels, and the voxel size is adaptively adjusted based on the density level.

[0083] In the present invention, the point cloud is divided into low, medium and high density areas according to the point cloud density distribution, and the voxel size is adaptively adjusted according to the density level.

[0084] Step S144 , calculating the centroid of each point in each voxel, and forming the three-dimensional point cloud data based on the centroid.

[0085] In the present invention, the centroid of all points in each voxel is calculated and used as the representative point of the voxel to form a simplified point cloud.

[0086] In some embodiments, the above step S150 can be implemented by the following steps S151 to S152:

[0087] Step S151, performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the steel wire rope to obtain the distribution direction of the steel wire rope and calculate the coordinates of the center point of each section of the fitting straight line.

[0088] In the present invention, the entire wire rope is evenly divided into several small segments, and the three-dimensional point cloud data of each small segment is first subjected to principal component analysis (PCA) dimensionality reduction to find the main direction of the point cloud data distribution, and then the least squares method is used to fit the spatial straight line, and the coordinates of the center point of each small segment of the fitted straight line are calculated.

[0089] Step S152, performing weighted averaging on the center point coordinates of the plurality of segments of the fitted straight line to obtain the center point coordinates of the steel wire rope.

[0090] In the present invention, the center coordinates of each small segment are weighted averaged to calculate the center point coordinates of the wire rope.

[0091] In some embodiments, before executing the above step S150, the method further includes: dividing the three-dimensional point cloud data into a plurality of segments at equal distances along the length direction of the wire rope.

[0092] In the present invention, the steel wire rope is divided into a number of small sections at equal distances along the length direction, and the length of each section is 3-5 times of the diameter of the steel wire rope.

[0093] In some embodiments, the above step S151 can be implemented by the following steps S1511 to S1513:

[0094] Step S1511, performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the steel wire rope to obtain the main direction vector of the steel wire rope.

[0095] In the present invention, PCA dimension reduction is performed on each segment of point cloud data to find the main direction vector representing the main direction of the wire rope in that segment.

[0096] Step S1512, constructing a spatial straight line equation based on the main direction vector and the center point of the three-dimensional point cloud data of each section of the wire rope, and using the least squares method to fit to obtain the optimal straight line parameters.

[0097] In the present invention, the main direction vector and the center point of the point cloud segment are used to construct a space straight line equation, and the least squares method is used to fit the optimal straight line parameters.

[0098] Step S1513, calculating the center point coordinates based on the optimal straight line parameters.

[0099] In the present invention, the coordinates of the center point of each segment of the point cloud are calculated, and the coordinates of the center point of each segment are weighted averaged according to the fitting error of each segment to obtain the coordinates of the center point of the entire wire rope.

[0100] In some embodiments, the above step S160 can be implemented by the following steps S161 to S163:

[0101] Step S161, obtaining the three-dimensional coordinates of the initial center point of the steel wire rope when it is in a static state, and using the three-dimensional coordinates of the initial center point as the initial reference point and the initial position of the tracking target.

[0102] In the present invention, when the wire rope is stationary, the three-dimensional coordinates P0 (x0, y0, z0) of its center point are obtained as the initial reference point and the initial position of the tracking target.

[0103] Step S162, calculating the three-dimensional coordinates of the corresponding center point of the steel wire rope in each frame of the depth image, and using the three-dimensional coordinates of the corresponding center point as the current position of the tracking target.

[0104] In the present invention, for each frame of depth image, the three-dimensional coordinates P of the center point of the wire rope point cloud are calculated. t (x(t), y(t), z(t)) and use it as the current position of the tracking target.

[0105] Step S163, using the dynamic point tracking method, calculating the displacement of the current position relative to the initial position, that is, obtaining the spatial vibration displacement.

[0106] In the present invention, the spatial position change of the center point of the steel wire rope is continuously tracked and its displacement relative to the initial reference point is calculated, thereby obtaining the vibration displacement d(t) of the steel wire rope in the spatial position.

[0107] The present invention adopts a binocular structured light camera to collect depth data of the wire rope. By projecting structured light on the surface of the wire rope and utilizing the principle of binocular stereo vision, a speckle pattern image sequence formed on the surface of the wire rope is captured. The embedded vision processor is based on an image stereo matching algorithm and combined with a triangulation method to accurately calculate the depth data of each point on the surface of the wire rope and upload it to a host computer in real time. Compared with traditional two-dimensional cameras, the present invention can directly obtain three-dimensional information of objects, solves the problem of inaccurate measurement caused by lack of depth information, and can maintain high measurement accuracy and stability in complex mine environments.

[0108] In the process of point cloud segmentation, the present invention proposes a region growing algorithm based on curvature priority. First, the curvature and normal vector of the point cloud data after straight-through filtering are calculated, and points with higher curvature and satisfying certain distance and normal vector constraints are preferentially selected as seed points for region growth. Then, the points in the neighborhood that meet the similarity criteria are gradually merged into the same region through the region growing method, ensuring that the growth of the region can accurately capture the shape characteristics of the wire rope. This method can effectively highlight the shape difference between the wire rope and the background, and realize high-precision wire rope point cloud segmentation, which is particularly suitable for wire rope monitoring in multi-interference environments.

[0109] The present invention introduces an adaptive voxel grid simplification method based on point cloud density. First, the particle swarm optimization algorithm (PSO) is used to search for the optimal initial voxel size to balance the efficiency and accuracy of point cloud simplification. Secondly, the K-nearest neighbor algorithm (K-NN) is used to estimate the local density of each point, and the point cloud is divided into low, medium and high density areas according to the density information, and the voxel size is adaptively adjusted. Finally, the centroid of all points in each voxel is calculated and used as the representative point of the voxel to form a simplified point cloud. This method can effectively reduce the amount of point cloud data while maintaining key geometric features, providing simplified and accurate input data for subsequent processing such as wire rope vibration analysis.

[0110] This paper proposes a piecewise linear fitting method based on PCA to extract the center point of the wire rope. The wire rope is divided into equal-distance segments along the length direction, and each point cloud is subjected to PCA dimensionality reduction to find the main direction vector of the main direction, and the optimal straight line parameters are fitted using the least squares method. This method can accurately calculate the coordinates of the center point of the wire rope, providing a reliable basis for subsequent vibration displacement calculations, and effectively improving the accuracy and stability of vibration measurement.

[0111] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0112] The present invention provides a method for measuring the spatial vibration displacement of a mine hoisting wire rope based on a depth camera, comprising:

[0113] Step 1: Depth image acquisition:

[0114] Step 101: Use a binocular structured light camera to collect a depth image sequence of the wire rope.

[0115] Figure 2 The schematic diagram of the wire rope vibration displacement measurement system provided by the present invention is as follows: Figure 2As shown in the figure. 1 is a binocular structured light camera, 2 is a drum, 3 is an information processing platform, 4 is a network switch, 5 is an upper and lower sheave, and 6 is a main bucket and an auxiliary bucket. The camera is placed at a specific position near the drum in the winch room, and the camera optical axis is kept perpendicular to the central axis of the wire rope. The image data collected by the camera is transmitted to the data processing platform of the ground control center through an Ethernet optical cable and a network switch.

[0116] Step 102: Use a depth camera to continuously collect depth image information of the wire rope between the drum and the sheave. These depth images contain the three-dimensional position information of the wire rope in space and its motion state. The collected data is transmitted to the network switch via an Ethernet optical cable, and the network switch is responsible for efficiently distributing and transmitting the data to the information processing platform of the ground control center.

[0117] Step 2: Deep image point cloud:

[0118] Each pixel point in the acquired depth image is mapped to the three-dimensional space, and the pixel points in the depth image are converted into the corresponding three-dimensional space coordinates using the intrinsic and extrinsic parameters of the camera to generate a point cloud containing the position information of each point.

[0119] In order to convert the collected wire rope depth image into point cloud data, the camera factory calibrated intrinsic and extrinsic parameter matrices are used to map each pixel in the depth image into three-dimensional space to obtain the corresponding three-dimensional coordinates to form the initial point cloud data. The conversion relationship is as follows: Figure 3 shown.

[0120] O w -X w Y w Z w is the world coordinate system, O c -X c Y c Z c is the camera coordinate system, o xy -xy is the image coordinate system, o uv -uv is the pixel coordinate system. Point P is any point in the world coordinate system, and its coordinates in the world coordinate system are P w (X w ,Y w ,Z w ), the coordinate in the camera coordinate system is P c (X c ,Y c ,Z c ). Point P is point P w The coordinates of the imaging point in the image are (x, y) in the image coordinate system and (u, v) in the pixel coordinate system. The conversion from the pixel coordinate system to the world coordinate system can be divided into two steps:

[0121] Step 201: Pixel coordinates (u, v) to camera coordinate system (X c ,Y c ,Z c )Conversion:

[0122]

[0123] Where (u, v) is the pixel coordinate, (c x ,c y ) is the optical center coordinate, (f x ,f y ) is the focal length, Z c is the depth of the point in the camera coordinate system.

[0124] Step 202, camera coordinate system (X c ,Y c ,Z c ) to the world coordinate system (X w ,Y w ,Z w ):

[0125]

[0126] In the formula, (r 11 ,r 12 ,r 13 ,r 21 ,r 22 ,r 23 ,r 31 ,r 32 ,r 33 ) is the element of the rotation matrix R, (t x ,t y ,t z ) are the elements of the translation vector T.

[0127] Step 3: Point cloud data preprocessing:

[0128] like Figure 4 As shown, Figure 4 The flowchart of the wire rope area point cloud data preprocessing provided by the present invention is used to preprocess the point cloud data using Python-OpenCV and Open3D point cloud library. The point cloud data is filtered, segmented, and simplified using the voxel grid method to obtain smooth three-dimensional point cloud data with wire rope characteristics.

[0129] Step 301, through filtering: according to the set threshold ranges of the X, Y and Z axes, point cloud data beyond the specified range is removed to extract the point cloud within the region of interest where the wire rope is located;

[0130] Step 302, point cloud segmentation: using a region growing algorithm based on curvature priority, by calculating the curvature of the point cloud, preferentially selecting points with higher curvature as seed points for region growing. Using the similarity of normal vectors and curvature, points that meet the conditions are gradually gathered, and finally the point cloud data of the wire rope is segmented;

[0131] Step 303, point cloud simplification: Adopt an adaptive voxel grid algorithm to simplify the segmented wire rope point cloud, and dynamically adjust the voxel size by estimating the local density distribution of the wire rope point cloud, so that the voxel division can better adapt to the local geometric characteristics of the wire rope point cloud.

[0132] In this embodiment, the wire rope point cloud segmentation is performed by using the curvature-first region growing method in step 302. The specific process is as follows:

[0133] Step 3021, curvature calculation and seed point selection: The point cloud data P after the straight-through filtering is i =(x i ,y i ,z i )|i=1,2,...,N} for processing. The curvature k(p i ) is calculated by the geometric features of the local neighborhood.

[0134] Points with higher curvature and satisfying distance and normal vector constraints are preferentially selected as seed points to form a seed point set:

[0135] S={p i ∈P|k(p i )>k th ∧d(p i ,p j )<d th ∧n i ·n j >cos(θ th )} (3),

[0136] Among them, P is the entire point cloud set, ∧ is used to represent the logical "and", k th is the curvature threshold, d th is the distance threshold, θ th is the normal vector angle threshold, n i and n j is the normal vector of the corresponding point, d(p i ,p j ) represents point p i With point p j The distance between.

[0137] Step 3022: Region growing: starting from the seed point p sAt the beginning, the points in its neighborhood that meet the similarity criteria are merged into the same region R s For each point p j In the neighborhood point set N(p s ), if the following similarity criteria are met, they are merged into regions:

[0138] N(p s )={p j ∈P|d(p s ,p j )<r} (4),

[0139]

[0140] Here, r is a predefined radius used to determine the range of the neighborhood.

[0141] Step 3023, recursive process of region growing: add the newly added region R s The point is taken as the new seed point, and the similarity check of the points in its neighborhood is continued, and the points that meet the conditions are merged into the region R. s The recursive process is as follows:

[0142] R s =R s ∪{p j |satisfies the similarity criterion} (6),

[0143] Until no new points meet the conditions, region R s No longer growing.

[0144] Step 3024: Region screening: based on the preset minimum cluster threshold N min and preset maximum cluster N max Threshold, filter out the areas that meet the conditions, remove the areas that are too small or too large, and finally retain the areas that meet the conditions.

[0145] In this embodiment, in step 303, an adaptive voxel grid method based on point cloud density is used to simplify the point cloud. The specific process is as follows:

[0146] Step 3031: Assume that the original wire rope point cloud is a point set P, which contains N three-dimensional points. Each point p i The coordinates of (x i ,y i ,z i ). The simplified point cloud set P′ is obtained by downsampling method;

[0147] Step 3032: Use the PSO algorithm to search for the optimal initial voxel size v0 within a given range. The PSO algorithm simulates swarm intelligence behavior and iteratively optimizes the particle position to find the optimal solution. The PSO algorithm objective function F combines the simplification rate R, simplification time T, and the maximum error E between the original point set P and the simplification point set P′. max And the average error E avg , the objective function is defined as:

[0148] F=w1R+w2T+w3E max +w4E avg (7)

[0149] Among them, w1, w2, w3, w4 are weight coefficients corresponding to the four indicators, satisfying w1+w2+w3+w4=1.

[0150] Step 3033: For each point p in the original wire rope point cloud P i , calculate the local density ρ i , use the k nearest neighbor algorithm to estimate the local density. For each point p i , find its nearest k neighbor points. Calculate the distance between these k points and p i The average distance d i , then the local density can be estimated as:

[0151]

[0152] Step 3034: The local density of the wire rope point cloud is regarded as a statistical distribution and is divided into three parts: low density, medium density and high density. Average density Reflects the concentration trend of the overall density of the point cloud, the density standard deviation σ ρ Reflects the discrete degree of point cloud density distribution;

[0153]

[0154]

[0155] Set low density threshold Medium density threshold High Density Threshold According to different density areas, the voxel size is dynamically adjusted using linear interpolation. The voxel size adjustment strategy is:

[0156]

[0157] Step 3035: For each voxel V j , select the centroid point C j As the representative point of the voxel, a simplified point cloud set P′ is formed.j , calculate the centroid C of all points in the voxel j :

[0158]

[0159] Where N j V j The total number of midpoints.

[0160] like Figure 5 As shown, Figure 5 This is a schematic diagram of the visualized point cloud after the wire rope point cloud is simplified provided by the present invention, wherein the red arrow points to the positive direction of the X-axis, the green arrow points to the positive direction of the Y-axis, and the blue arrow points to the positive direction of the Z-axis. The lateral vibration of the wire rope is only related to the X-axis and the Z-axis.

[0161] Step 4: Calculate the three-dimensional coordinates of the center point:

[0162] like Figure 6 As shown, Figure 6 The three-dimensional schematic diagram of the point cloud provided by the present invention after the straight line fitting is performed. The three-dimensional schematic diagram of the straight line and the point cloud where the point cloud is located is fitted by the least squares method based on the simplified point cloud.

[0163] Step 401, the entire steel wire rope is evenly divided into several small segments, and the principal component analysis (PCA) dimensionality reduction is first performed on the three-dimensional point cloud data of each small segment to find the main direction of the point cloud data distribution, and then the least squares method is used to fit the space line, and the coordinates of the center point of each small segment fitting line are calculated;

[0164] Step 402: Perform weighted averaging on the center coordinates of each small segment to calculate the center point coordinates of the wire rope.

[0165] In this embodiment, in step 401, a piecewise linear fitting point cloud center extraction method based on PCA is used to obtain the three-dimensional coordinates of the center point. The specific process is as follows:

[0166] Step 4011: After a series of preprocessing, the point cloud data obtained in step 403 is used to obtain a three-dimensional space point sequence of the wire rope.

[0167] Step 4012: The height of the wire rope in the camera field of view remains unchanged. The wire rope is divided into n small segments at equal intervals to ensure that each segment contains enough points for accurate fitting and analysis.

[0168] Step 4013: Perform principal component analysis (PCA) dimensionality reduction on each segment of point cloud to find the principal component vector representing the main change direction of the point cloud data of this segment. The principal component vector is the direction of the wire rope of this segment, and the vector v j =(a j ,b j,c j ) represents the main direction vector of the jth segment.

[0169] Step 4014: Use the least squares method to perform straight line fitting on each small segment of the point cloud. Assume that the straight line equation is ax+by+cz+d=0, and define the error function d i For point P i (x i ,y i ,z i ) to the vertical distance of the fitted line:

[0170]

[0171] Step 4015: The core objective of the least square method is to minimize the sum of the squares of the vertical distances from all points to the fitted line. To this end, the objective function F is defined as:

[0172]

[0173] Where N is the number of points in the segment. By taking the partial derivatives of F with respect to a, b, c, and d respectively and setting the derivatives equal to zero, the optimal values ​​of the parameters a, b, c, and d are solved. The fitting error σ j Take the square root of the objective function for this segment:

[0174]

[0175] Step 4016: The fitting line of each segment is determined by its optimal parameters. Let the coordinates of the center point of the jth segment be p j (x j ,y j ,z j ), the fitting error is σ j , the weighted average center point coordinates of the fitted line It can be calculated by the following formula:

[0176]

[0177] Among them, σ j is the fitting error of the jth straight line, and the weight is the inverse of the fitting error of each segment, thereby improving the accuracy and stability of the fitting.

[0178] Step 5: Calculation of spatial vibration displacement:

[0179] Figure 7 The calculation flow chart of measuring wire rope vibration displacement provided by the present invention is shown in FIG8 .

[0180] Step 601: When the wire rope is stationary, first obtain the initial coordinates P0 (x0, y0, z0) of the center point of the wire rope as a reference point;

[0181] Step 602: Set an appropriate frame interval Δt, collect a depth image sequence of the wire rope, and use the above method to calculate the current center point coordinate P of the wire rope point cloud in each frame image. t (x(t),y(t),z(t));

[0182] Step 603: By calculating the displacement change between the coordinates of the center point of each frame and the coordinates of the reference point, the vibration displacement of the wire rope along the radial direction (X direction) of the drum main axis and along the axial direction (Z direction) of the drum main axis are obtained. The vibration displacement calculation formula of the wire rope in the X and Z directions is as follows:

[0183]

[0184]

[0185] Among them, t represents the time variable, d x (t) represents the vibration displacement of the wire rope in the X direction, d z (t) represents the vibration displacement of the wire rope in the Z direction, in millimeters (mm). This method dynamically tracks the motion trajectory of the center point of the wire rope point cloud in three-dimensional space, accurately calculates the displacement components of the wire rope in the XY plane and the YZ plane, and thus obtains the vibration displacement of the wire rope in two directions.

[0186] In this embodiment, the camera selected is an FM851-E2 active binocular structured light depth camera with a camera accuracy of 1mm@1000mm.

[0187] In this embodiment, the data processor is a computer, the GPU is NVIDIA GeForce RTX 3080, the CPU is Intel(R) Core(TM) i7-12700K, and the development tools and languages ​​are Open3D and PyQt5.

[0188] In this embodiment, in order to verify the feasibility of the method in this article, a laser displacement sensor is used for verification. The accuracy of the displacement sensor is 0.07mm@65~135mm, the sampling frequency is 70Hz, and the signal collected by the sensor is processed by the dynamic signal test and analysis system and then transmitted to the computer via USB.

[0189] FIG8 is a graph of the wire rope vibration displacement-time curve obtained by the laser displacement sensor and the camera provided by the present invention. The experimental results show that the X direction (such as Figure 8a ) has an average error of 0.58 mm and a maximum error of 1.13 mm; in the Z direction (such as Figure 8b) is 0.62mm, and the maximum error is 1.17mm. The depth camera-based vibration displacement measurement method for mine hoisting wire ropes proposed in the present invention provides an efficient and reliable new method for vibration measurement of mine hoisting wire ropes.

[0190] In summary, the method of the present invention has simple steps, reasonable design and good point cloud processing effect. By continuously analyzing the changes in the wire rope amplitude time domain curve and its characteristic parameters obtained by depth camera measurement, the health status of the wire rope can be grasped in time, and targeted maintenance measures can be taken to ensure the safety and service life of the wire rope, and provide reliable technical support for the safe operation of the coal mine hoisting system.

[0191] Fig. 9 is a schematic diagram of the composition structure of a mine wire rope spatial vibration displacement measuring device based on a depth camera provided in an embodiment of the present invention, such as Fig. 9 As shown, a depth camera-based spatial vibration displacement measuring device 900 for a mine wire rope includes: an acquisition module 901, which is used to acquire a depth image of the wire rope through a binocular structured light camera; a conversion module 902, which is used to convert the pixel points in the depth image into corresponding three-dimensional space coordinates based on the internal and external parameters of the binocular structured light camera to obtain initial point cloud data; a point cloud segmentation module 903, which is used to perform point cloud segmentation on the initial point cloud data after direct filtering through a region generation algorithm based on curvature priority to obtain segmented point cloud data; a point cloud simplification module 904, The computing module 905 is used to simplify the segmented point cloud data by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the wire rope; the computing module 905 is used to calculate the three-dimensional point cloud data by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the wire rope when it is in a static state; the computing module 905 is also used to calculate the spatial vibration displacement of the center point of the wire rope in a static reference coordinate system by a dynamic point tracking method; the static reference coordinate system is established with the coordinates of the center point as the origin.

[0192] In some embodiments, the point cloud segmentation module is also used to remove the point cloud data in the initial point cloud data that exceeds the threshold range based on the preset X, Y and Z axis threshold range, so as to obtain the point cloud data after the through filtering; and perform point cloud segmentation on the initial point cloud data after the through filtering through a curvature-first region generation algorithm to obtain the segmented point cloud data.

[0193] In some embodiments, the calculation module is also used to perform principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each segment of the wire rope to obtain the distribution direction of the wire rope and calculate the center point coordinates of each segment of the fitting straight line; the center point coordinates of multiple segments of the fitting straight line are weighted averaged to obtain the center point coordinates of the wire rope when it is in a static state.

[0194] In some embodiments, the calculation module is also used to obtain the three-dimensional coordinates of the initial center point of the wire rope when it is in a static state, and use the three-dimensional coordinates of the initial center point as the initial reference point and the initial position of the tracking target; calculate the three-dimensional coordinates of the corresponding center point of the wire rope in each frame of the depth image, and use the three-dimensional coordinates of the corresponding center point as the current position of the tracking target; use the dynamic point tracking method to calculate the displacement of the current position relative to the initial position, that is, to obtain the spatial vibration displacement.

[0195] In some embodiments, the point cloud segmentation module is also used to calculate the curvature of each point in the point cloud data after the straight-through filtering based on a preset area generation algorithm, and use points with higher curvature and satisfying distance and normal vector constraints as seed points; starting from the seed point, gradually merge the points in the neighborhood of the seed point that satisfy the preset similarity criteria into the same area, and determine that the generation of the area can capture the shape characteristics of the wire rope; use the points newly added to the area as new seed points, and repeat the area generation until all points that satisfy the preset similarity criteria are merged; based on the preset minimum cluster threshold and the preset maximum cluster threshold, eliminate areas that are too small or too large, and determine the qualified areas as the point cloud data after the wire rope is segmented.

[0196] In some embodiments, the point cloud simplification module is also used to determine the optimal initial voxel size using a preset particle swarm optimization algorithm; use a preset K-nearest neighbor algorithm to estimate the local point cloud density of each point in the segmented point cloud data; based on each of the local point cloud densities, divide the segmented point cloud data into areas with different density levels, and adaptively adjust the voxel size based on the density level; calculate the center of mass of each point in each voxel, and form the three-dimensional point cloud data based on the center of mass.

[0197] In some embodiments, before performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the wire rope to obtain the distribution direction of the wire rope, the device also includes: a division module, used to divide the three-dimensional point cloud data into a number of sections with equal distances along the length direction of the wire rope; the determination module is also used to perform principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the wire rope to obtain the main direction vector of the wire rope; based on the main direction vector and the center point of the three-dimensional point cloud data of each section of the wire rope, construct a spatial straight line equation, and use the least squares method to fit to obtain the optimal straight line parameters; based on the optimal straight line parameters, calculate the center point coordinates.

[0198] It should be noted that the description of the device of the embodiment of the present invention is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present invention for understanding.

[0199] It should be noted that, in the embodiment of the present invention, if the above-mentioned method for measuring the spatial vibration displacement of a mine wire rope based on a depth camera is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a terminal to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0200] Correspondingly, an embodiment of the present invention provides a mine wire rope spatial vibration displacement measurement device based on a depth camera, Fig.10 FIG. 1 is a schematic diagram of the composition structure of a mine wire rope spatial vibration displacement measurement device based on a depth camera provided in an embodiment of the present invention. Fig.10As shown, the spatial vibration displacement measuring device 1000 for steel wire rope in a mine based on a depth camera comprises at least: a processor 1001 and a computer-readable storage medium 1002 configured to store executable instructions, wherein the processor 1001 generally controls the overall operation of the spatial vibration displacement measuring device 1000 for steel wire rope in a mine based on a depth camera. The computer-readable storage medium 1002 is configured to store instructions and applications executable by the processor 1001, and can also cache data to be processed or processed by each module in the processor 1001 and the spatial vibration displacement measuring device 1000 for steel wire rope in a mine based on a depth camera, which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0201] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 1 The method shown.

[0202] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0203] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0204] As an example, executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0205] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

[0206] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0207] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0208] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for measuring spatial vibration displacement of mine wire rope based on a depth camera, characterized in that: The method comprises: The depth image of the wire rope is obtained by using a binocular structured light camera; Based on the internal and external parameters of the binocular structured light camera, the pixel points in the depth image are converted into corresponding three-dimensional space coordinates to obtain initial point cloud data; The initial point cloud data after the straight-through filtering is segmented by a region generation algorithm based on curvature priority to obtain segmented point cloud data; The segmented point cloud data is simplified by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope; The three-dimensional point cloud data is calculated by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state; The spatial vibration displacement of the center point of the wire rope in a static reference coordinate system is calculated by using a dynamic point tracking method; the static reference coordinate system is established with the center point coordinates as the origin.

2. The method according to claim 1, characterized in that The point cloud data after the straight-through filtering is segmented by a region generation algorithm based on curvature priority to obtain segmented point cloud data, including: Based on the preset threshold ranges of the X, Y and Z axes, point cloud data exceeding the threshold ranges in the initial point cloud data are removed to obtain the point cloud data after the through filtering; The initial point cloud data after straight-through filtering is segmented by a region generation algorithm based on curvature priority to obtain segmented point cloud data.

3. The method according to claim 1, characterized in that The three-dimensional point cloud data is calculated by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state, including: Performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the steel wire rope to obtain the distribution direction of the steel wire rope and calculate the coordinates of the center point of each section of the fitting straight line; The center point coordinates of the plurality of segments of the fitted straight lines are weighted averaged to obtain the center point coordinates of the steel wire rope when it is in a static state.

4. The method according to claim 1, characterized in that: The method of calculating the spatial vibration displacement of the center point of the steel wire rope in a static reference coordinate system by using a dynamic point tracking method includes: Obtaining the three-dimensional coordinates of the initial center point of the steel wire rope when it is in a static state, and using the three-dimensional coordinates of the initial center point as the initial reference point and the initial position of the tracking target; Calculate the three-dimensional coordinates of the corresponding center point of the steel wire rope in each frame of the depth image, and use the three-dimensional coordinates of the corresponding center point as the current position of the tracking target; The dynamic point tracking method is used to calculate the displacement of the current position relative to the initial position, thereby obtaining the spatial vibration displacement.

5. The method according to claim 1, characterized in that The point cloud data after the straight-through filtering is segmented by a region generation algorithm based on curvature priority to obtain segmented point cloud data, including: Based on a preset region generation algorithm, the curvature of each point in the point cloud data after the straight-through filtering is calculated, and points with higher curvature and satisfying distance and normal vector constraints are used as seed points; Starting from the seed point, points in the neighborhood of the seed point that meet a preset similarity criterion are gradually merged into the same region, and it is determined that the generation of the region can capture the shape characteristics of the steel wire rope; The points newly added to the region are used as new seed points, and the region generation is repeated until all points satisfying the preset similarity criteria are merged; Based on the preset minimum cluster threshold and the preset maximum cluster threshold, the too small or too large area is eliminated, and the qualified area is determined as the point cloud data after the wire rope segmentation.

6. The method according to claim 1, characterized in that The point cloud data after segmentation are simplified by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope, including: Using the preset particle swarm optimization algorithm, the optimal initial voxel size is determined; Using a preset K nearest neighbor algorithm, the local density of the point cloud of each point in the segmented point cloud data is estimated; Based on the local density of each point cloud, the segmented point cloud data is divided into regions with different density levels, and the voxel size is adaptively adjusted based on the density level; The centroid of each point in each voxel is calculated, and the three-dimensional point cloud data is formed based on the centroid.

7. The method according to claim 3, characterized in that Before performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the steel wire rope to obtain the distribution direction of the steel wire rope, the method further includes: Dividing the three-dimensional point cloud data into a plurality of segments at equal distances along the length direction of the steel wire rope; The principal component analysis and dimensionality reduction are performed on the three-dimensional point cloud data of each section of the steel wire rope to obtain the distribution direction of the steel wire rope and calculate the coordinates of the center point of each section of the fitting straight line, including: Performing principal component analysis and dimensionality reduction on the three-dimensional point cloud data of each section of the steel wire rope to obtain the main direction vector of the steel wire rope; Based on the main direction vector and the center point of the three-dimensional point cloud data of each section of the wire rope, a spatial straight line equation is constructed, and the optimal straight line parameters are obtained by fitting using the least square method; Based on the optimal straight line parameters, the center point coordinates are calculated.

8. A device for measuring the spatial vibration displacement of a mine wire rope based on a depth camera, characterized in that: The device comprises: An acquisition module, used for acquiring a depth image of the wire rope through a binocular structured light camera; A conversion module, used to convert the pixel points in the depth image into corresponding three-dimensional space coordinates based on the intrinsic parameters and extrinsic parameters of the binocular structured light camera to obtain initial point cloud data; A point cloud segmentation module is used to perform point cloud segmentation on the initial point cloud data after the straight-through filtering by using a region generation algorithm based on curvature priority to obtain segmented point cloud data; A point cloud simplification module, used to simplify the segmented point cloud data by an adaptive voxel grid simplification algorithm based on point cloud density to obtain three-dimensional point cloud data of the steel wire rope; A calculation module, used to calculate the three-dimensional point cloud data by a piecewise linear fitting algorithm based on principal component analysis to obtain the coordinates of the center point of the steel wire rope when it is in a static state; The calculation module is also used to calculate the spatial vibration displacement of the center point of the wire rope in a static reference coordinate system using a dynamic point tracking method; the static reference coordinate system is established with the center point coordinates as the origin.

Images