Coal gangue transport simulation device and method with image recognition technology

Abstract

This invention discloses a coal gangue transport simulation device and method equipped with image recognition technology, comprising: collecting coal gangue data and device movement data based on a preset coal gangue transport simulation device, wherein the coal gangue data includes coal gangue images, time-series video streams, and coal gangue materials; generating simulated coal gangue data based on the coal gangue materials; obtaining simulated movement parameters based on the simulated coal gangue data; setting the baffle spacing and motor speed of the coal gangue transport simulation device according to the device structure using the simulated movement parameters; performing simulated transport; drawing a coal gangue interface based on the simulated transport results; analyzing the coal gangue transport trend using the coal gangue interface and the coal gangue video stream; and obtaining simulated coal seam data and simulated coal discharge window data. This method simulates the transport of broken top coal and loose gangue during top coal caving operations, and analyzes the impact of factors such as coal seam thickness and coal discharge window opening size on coal gangue transport by collecting simulated data, thereby guiding on-site coal discharge operations and improving top coal recovery rate.

Description

Technical Field

[0001] This invention relates to the field of coal mining, and in particular to a coal gangue transport simulation device and method equipped with image recognition technology. Background Technology

[0002] In China, longwall coal mining faces often employ top coal caving technology. The broken top coal and gangue are released from the coal caving window behind or above the hydraulic support and transported out of the working face by a scraper conveyor. By analyzing a large amount of experimental data, the influence of factors such as coal seam thickness and coal caving window opening size on coal and gangue migration can be improved, the top coal recovery rate can be increased, and the coal and gangue migration during the top coal caving process can be better understood.

[0003] However, coal mine working faces have complex production conditions, poor underground visibility, and heavy coal dust, making direct observation impossible. Large-scale top coal caving simulation experimental devices built in laboratories are costly, and the preliminary preparation work is arduous, requiring high levels of technical expertise from the experimental plan and personnel. How to simulate the movement of broken top coal and gangue during top coal caving operations through image recognition, while simultaneously collecting experimental data to analyze the impact of factors such as coal seam thickness and coal caving window opening size on coal and gangue movement, to guide on-site coal caving operations and improve top coal recovery rate, has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a coal gangue transport simulation device and method equipped with image recognition technology.

[0005] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0006] The first aspect of this invention provides a method for simulating coal gangue transport using image recognition technology, comprising:

[0007] Based on a pre-set coal gangue transport simulation device, coal gangue data and device movement data are collected. The coal gangue data includes coal gangue images, time-series video streams, and coal gangue materials.

[0008] Wavelet convolution is performed on the coal and gangue images based on the coal and gangue material to obtain camera angle parameters and labeled datasets. Coal discharge window data is extracted using the camera angle parameters and time-series video streams. Simulated coal and gangue data is generated through a conditional generative adversarial network based on the coal discharge window data and labeled datasets.

[0009] Based on the simulated coal gangue data, simulated materials are laid using coal gangue materials. The input sequence of the simulated materials and the device motion data are differentially fitted to obtain simulated motion parameters. According to the device structure, the baffle spacing and motor speed of the coal gangue transport simulation device are set through the simulated motion parameters.

[0010] The simulation of coal gangue is carried out based on simulated coal gangue data and a coal gangue transport simulation device. Real-time images and coal gangue video streams are obtained based on the simulation transport results, and the coal gangue interface is drawn by labeling the dataset.

[0011] The coal and gangue interface and coal and gangue video stream were used to analyze the coal and gangue migration trend and obtain the coal and gangue migration trajectory. The coal and gangue migration trajectory was compared and calibrated based on the coal discharge window data to obtain simulated coal seam data and simulated coal discharge window data.

[0012] Furthermore, the method for obtaining the camera angle parameters and the labeled dataset includes:

[0013] Coal and gangue image groups are obtained by dividing the coal and gangue images corresponding to different device movement data. Two-dimensional wavelet convolution is performed on the coal and gangue images based on the coal and gangue material to obtain multi-scale wavelet coefficients and scale weighting factors negatively correlated with scale. The dominant angular components of image features are calculated using the scale weighting factors and the partial derivatives of the wavelet basis functions in the pixel direction. Camera angle parameters are calculated based on the dominant angular components of image features in different coal and gangue image groups and the camera angle. The formula for calculating the camera angle parameters is as follows:

[0014]

[0015]

[0016] in The dominant angular component is the feature of a single frame image. The scale weighting factor is related to the scale of the wavelet transform. negative correlation These are multi-scale wavelet coefficients, representing different scales. In translation amount Image pixels The characteristics of coal gangue at the location, Here are wavelet basis functions, representing Daubechies wavelet and Symlet wavelet. For camera angle parameters, The total number of groups For the first The camera angle of the group is No. Angular components of the group, For the first The initial tilt angle of the setup is obtained from the tilt angle of the bottom friction testing machine. For the first The contribution of the image features of the group is obtained by quantizing the directional consistency of the multi-scale wavelet coefficients. This is the initial tilt angle of the apparatus in the current experiment;

[0017] Based on multi-scale wavelet coefficients, the contrast feature function of coal gangue image pixels is calculated through gray-level co-occurrence matrix. The coal gangue texture tendency of the pixels is marked according to the contrast feature function. The magnitude and phase of multi-scale wavelet coefficients are extracted. Based on the coal gangue texture tendency, magnitude intensity and phase interval, coal gangue images are selected and marked according to preset typical coal gangue features to obtain a labeled dataset.

[0018] Furthermore, the method for obtaining the simulated coal gangue data includes:

[0019] The camera angle parameters are used to calibrate the viewpoint of the time-series video stream. The inter-frame difference method is used to analyze the dynamic region of coal and gangue movement in the calibrated time-series video stream. The length, width, and position coordinates of the coal discharge window in the dynamic region are used as the spatial boundary. Based on the device's motor data, the opening and closing amplitude of the coal discharge window and the timing of coal and gangue outflow during the coal discharge process are used as time-varying features. The spatial boundary and time-varying features are encoded into coal discharge window data.

[0020] Using coal discharge window data and labeled dataset as conditional information, a random noise vector following a Rosin-Rammler distribution is generated based on the conditional information. The conditional information and random noise vector are then input into the generator of a conditional generative adversarial network. Based on the discriminator of the conditional generative adversarial network, the true and generated binary classification probabilities are output according to the labeled dataset and simulated coal gangue data. The input is then mapped to simulated coal gangue data, which includes coal gangue distribution, particle diameter, and initial velocity of the bulk material.

[0021] Furthermore, the method for obtaining the simulated maneuver parameters includes:

[0022] Based on simulated coal gangue data, coal gangue materials are extracted. According to the distribution and particle diameter of loose coal gangue in the simulated coal gangue data, coal gangue materials are laid in different regions and layers in the coal gangue transport simulation device to obtain simulated materials. The laying time and laying thickness variation time sequence of materials in different particle size ranges are spliced ​​into a time series input vector, and the time series input vector is used as the input quantity of simulated materials.

[0023] Based on the time series input vector, the corresponding time-series device motion data is extracted. Using the initial motor speed, initial baffle spacing, and initial lifting rod inclination angle of the bottom friction testing machine from the device motion data, the simulated motor speed is iteratively fitted using a discrete differential fitting formula. The discrete differential fitting formula is as follows:

[0024]

[0025] in For a moment The simulated motor speed, The initial motor speed is obtained from the device's motion data. This refers to the frame number of the corresponding time-series video stream. This represents the maximum baffle spacing of the device. For the first The spacing between the coal feeding windows of the frame, For the first The frame's motor speed, For a function with a positive part, take non-negative values. For the first Multiscale wavelet coefficients of frames For the first Multiscale wavelet coefficients of frames For the first The amount of simulated material input in the frame. For the first The velocity modulus of the frame. The magnitude of the discrete velocity in the initial frame. This is a single-view residual correction term, representing the deviation between the actual shooting angle and the camera angle parameter.

[0026] The baffle spacing is adjusted to the corresponding time based on the simulated motor speed to obtain the simulated baffle spacing, which is used as a simulated maneuver parameter.

[0027] Furthermore, the method for obtaining the coal-gangue interface includes:

[0028] Based on the device structure, the baffle spacing and motor speed of the coal gangue transport simulation device are set by simulating the motor parameters. Based on the simulated coal gangue data, the corresponding simulated materials are obtained. The simulated materials and the coal gangue transport simulation device are used to simulate transport. Real-time images and coal gangue video streams are obtained by the digital camera of the coal gangue transport simulation device. The real-time images are matched with the texture features of coal gangue in the labeled dataset to obtain the coal area and gangue area. Based on the coal area and gangue area, three-dimensional reconstruction is performed according to the vision of the dual digital cameras and the Zhang Zhengyou calibration method to obtain the drawn coal gangue interface.

[0029] Furthermore, the method for obtaining the simulated coal seam data and the simulated coal release window data includes:

[0030] Based on the coal-gangue interface, the contour centroid coordinate changes, boundary expansion rate, and boundary contraction rate of adjacent frames in the coal-gangue video stream are extracted to obtain boundary feature points. According to the labeled dataset, the average transport velocity, average acceleration, and motion direction dispersion of coal-gangue particles in the coal-gangue video stream are statistically analyzed. The boundary feature points and coal-gangue particles are used as trajectory feature points. Based on the trajectory feature points, the coordinates of the trajectory feature points are time-series tracked using the Kalman filter algorithm to obtain the continuous motion trajectory of the trajectory feature points. The continuous motion trajectory is clustered using Euclidean distance to obtain the coal-gangue transport trajectory.

[0031] The trajectory deviation of the coal and gangue transport trajectory is calculated based on the spatial boundary and time-varying characteristics of the coal discharge window data. If the trajectory deviation of the feature point is greater than 0.1m or the lag time is greater than 0.5 seconds, it is marked as needing optimization. The simulation maneuver parameters are iteratively adjusted according to the overall coal and gangue transport trajectory that needs optimization until the position deviation of the feature point is less than or equal to 0.05m and the time deviation is less than or equal to 0.2 seconds. The marked data of the coal discharge window data in the iterative simulation is obtained. The marked data is used to obtain the simulated coal seam dataset through the coal and gangue interface image, which is used as the simulated coal seam data.

[0032] Based on the 3D reconstruction results, the coal-gangue interface is used as the actual pixel coordinates. The unloaded image corresponding to the coal-gangue interface is converted into theoretical pixel coordinates through perspective projection. The average reprojection error is calculated using the Euclidean distance between the actual pixel coordinates and the theoretical pixel coordinates. The error percentage is calculated based on the average reprojection error with a single pixel as the standard to obtain the camera view matching degree. The camera angle is adjusted until the camera view matching degree is greater than or equal to 90%. The coal discharge window data is extracted based on the optimized camera view matching degree to obtain the simulated data of the coal discharge window.

[0033] The second aspect of the present invention provides a coal gangue transport simulation device equipped with image recognition technology, comprising: a bottom friction testing machine 101, a flat plate 102, a digital camera 103, a support 104, a lifting rod 105, a data processing device 106, a roller 107, and belts 108 and 204;

[0034] The flat plate 102 is fixedly installed at one end of the bottom friction testing machine 101. The bottom friction testing machine 101 includes the roller 107 and the belt 108. The belt 108 is rotated by rotating the roller 107. A lifting rod 105 is installed on the support foot near the flat plate 102. A limiting frame 201 is installed on the upper surface of the bottom friction testing machine 101. The baffle 204 is fixedly installed inside the limiting frame 201. The bracket 104 is installed above the bottom friction testing machine 101. The digital camera 103 is fixedly installed in the middle of the bracket 104. The bottom friction testing machine 101 and the digital camera 103 are electrically connected to the data processing device 106.

[0035] Further, the flat plate 102 is characterized in that the base plate 302 and the grip rod 301 are welded together, the grip rod 301 is welded at the center of one side of the base plate 302, the base plate 302 is a rectangular horizontal plate, the grip rod 301 is a cylindrical long rod, the thickness of the base plate is less than or equal to 5mm, the length-to-height ratio of the base plate 302 to the grip rod 301 is 1:1.5, the lifting rod 105 is fixedly installed on the support foot of the friction testing machine 101 below the flat plate 102, and the lifting rod 105 is a hydraulic rod.

[0036] Further, the digital camera 103 is characterized in that it is fixedly installed at the digital camera installation position 205 and the digital camera installation position 405 respectively, the digital camera at the digital camera installation position 205 is directed towards the horizontal direction of the bottom friction tester 101 to collect image range 502, and the digital camera at the digital camera installation position 405 is directed towards the upper surface of the bottom friction tester 101 to collect image range 501.

[0037] The digital camera mounting position 205 is fixedly set at the middle of one end of the inner side of the limiting frame 201.

[0038] The simulated gangue layer 202 and simulated coal seam 203 are fixedly provided on the other end of the inner side of the limiting frame 201, and the baffle 204 is fixedly provided on the side of the simulated coal seam 203 away from the simulated gangue layer 202.

[0039] The digital camera mounting position 405 is fixedly disposed on the top of the bracket 104. The bracket 104 includes a crossbar 401, a crossbar 402, a crossbar 403, a roller 404, and the digital camera mounting position 405. The bottom of the crossbar 401 is connected to the roller 404, and the top of the crossbar 401 is vertically connected to one end of the roller crossbar 402. The top of the crossbar 403 is vertically connected to the other end of the crossbar 402. The crossbar 401 is parallel to the crossbar 402, and the digital camera mounting position 405 is disposed in the middle of the crossbar 403.

[0040] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0041] This invention simulates the gravity experienced by coal and gangue during transport by using the frictional force on the coal and gangue. It uses baffles to simulate a coal discharge window, adjusting the opening size by varying the baffle spacing. A flat plate ensures full contact between the crushed top coal and gangue aggregate and the bottom friction tester. A digital camera captures image data of the coal and gangue transport during the experiment. A lifting rod adjusts the tilt angle of the bottom friction tester. Image recognition technology is used to analyze the coal and gangue transport data, identify the coal and gangue interface, and simulate the transport of crushed top coal and gangue aggregate during top coal discharge operations. This provides guidance for on-site coal discharge operations and improves the recovery rate of top coal. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the steps of a coal gangue transport simulation method incorporating image recognition technology, as described in an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram of the overall structure of a coal gangue transport simulation device equipped with image recognition technology according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the surface structure of the bottom friction testing machine in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the flat plate structure in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the support structure in an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of digital camera observation in an embodiment of the present invention.

[0048] The attached figures are labeled as follows: 101. Bottom friction testing machine, 102. Flat plate, 103. Digital camera, 104. Support, 105. Lifting rod, 106. Data processing equipment, 107. Roller, 108. Belt, 201. Limiting frame, 202. Simulated gangue layer, 203. Simulated coal seam, 204. Baffle, 205. Digital camera installation position, 301. Holding rod of flat plate, 302. Bottom plate of flat plate, 401. Crossbar, 402. Crossbar, 403. Crossbar, 404. Roller, 405. Digital camera installation position, 501. Digital camera image acquisition range, 502. Digital camera image acquisition range. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0050] Reference Figure 1 As shown, the present invention provides a coal gangue transport simulation method equipped with image recognition technology, comprising:

[0051] S1 collects coal and gangue data and device movement data based on a preset coal and gangue transport simulation device. The coal and gangue data includes coal and gangue images, time-series video streams, and coal and gangue materials.

[0052] In the actual evaluation, the bottom friction testing machine in the pre-set coal gangue transport simulation device was set with an initial tilt angle of 15°, an initial baffle spacing of 180mm, and an initial motor speed of 12r / min, controlled by a servo driver. The camera in the device was calibrated by printing a checkerboard calibration plate with 12×9 corner points and a grid size of 20mm×20mm, which was placed at different positions on the testing machine table. Twenty sets of calibration images were captured, and the intrinsic parameter matrix of the digital camera was obtained using OpenCV. The baseline distance between the rotation matrix and translation vector of its external parameters is 200mm. Based on the preset coal gangue transport simulation device, coal gangue data and device movement data are collected. Among them, the coal gangue particle size distribution of the three groups of coal gangue materials is 7:3, 6:4, and 5:5. The coal particle size range is (1, 5mm), [5, 15mm), [15, 30mm], and the gangue particle size range is (1, 10mm), [10, 25mm), [25, 40mm]. Each group of materials is laid evenly in three times. After each laying, it is left to stand for 2 minutes to eliminate the accumulation stress. The dual cameras continuously collect 3000 frames of images at a frame rate of 30. At the same time, the device movement data is collected in real time through the EtherCAT bus at a sampling frequency of 1kHz, including motor speed, baffle spacing and hydraulic lifting rod tilt angle.

[0053] S2 performs wavelet convolution on the coal and gangue image based on the coal and gangue material to obtain camera angle parameters and labeled dataset. It then uses the camera angle parameters and time-series video stream to extract coal discharge window data and generates simulated coal and gangue data through a conditional generative adversarial network based on the coal discharge window data and labeled dataset.

[0054] In practical assessment, the collected coal gangue images were subjected to two-dimensional wavelet convolution using the Daubechies-4 wavelet basis function, with a decomposition layer of 3, to obtain multi-scale wavelet coefficients. Where a is the scale, b is the translation amount, and (x,y) are the pixel coordinates, the scale weight factor negatively correlated with the scale is calculated. The values ​​of a are 1, 2, and 3, and the scale weight factors are 0.5, 0.25, and 0.125, respectively. The dominant angular components of single-frame image features are calculated using the partial derivatives of the wavelet basis functions in the pixel direction, yielding mean values ​​of 15.8, 15.3, and 16.1. Different groups of coal and gangue images are created based on the different equipment movement data, corresponding to different coal and gangue ratios. Combined with the camera angles of 16.2, 15.8, and 16.5 for each group, and considering the initial tilt angle of the equipment is 15°, the image feature contribution is quantified using wavelet coefficient direction consistency, taking values ​​of 0.88, 0.92, and 0.85, respectively. The calculated camera angle parameter is 16.1. Based on multi-scale wavelet coefficients, the pixel contrast feature function is calculated using the gray-level co-occurrence matrix skimage.feature.greycomatrix, where the distance d=3 and the angle... The texture tendency of coal gangue in the marked pixels was determined, with the contrast of coal areas less than 100 and the contrast of gangue areas greater than 150. The modulus and phase of the wavelet coefficients were extracted and combined with the texture tendency of coal gangue. The LabelImg tool was used to filter and mark 1500 frames of coal gangue images, marking the coal areas, gangue areas and interface contours to obtain the marked dataset.

[0055] In the actual evaluation, camera angle parameters were used to calibrate the viewing angle of 2000 frames of images acquired in the time-series video stream. The inter-frame difference method was employed, with a threshold of 30, and a 3×3 neighborhood analysis was used to examine the dynamic region of coal and gangue transport. The length (120-200mm), width (60-120mm), and position coordinates of the coal discharge window within the dynamic region were used as spatial boundaries. Based on the device's motion data, the opening and closing amplitude of the window (0-80mm) during coal discharge was extracted, and the proportion of coal and gangue outflow per frame was used as a time-varying feature. This time-varying feature consisted of a 0.2-0.5mm variation per frame and an outflow proportion of 0.5%-3% per frame. The spatial boundaries and time-varying features were encoded into a JSON-formatted coal discharge window. The data was used to construct a CGAN model. The generator consisted of 3 layers of transposed convolutions with a kernel size of 4×4 and a stride of 2. The discriminator consisted of 3 layers of convolutions with the same kernel size and stride. Using coal release window data and labeled datasets as conditional information, a 100-dimensional random noise vector following a Rosin-Rammler distribution was generated, with parameter n=2.5 and median feature particle size of 15mm. The conditional information and noise vector were input into the generator, with a learning rate of 0.0002 and a batch size of 64 for training. The discriminator output the binary classification probabilities of the real data and the generated data, obtaining 1000 sets of simulated coal gangue data, where the initial velocity of the loose material was 0.5-2cm / s.

[0056] S3 uses the simulated coal gangue data to lay simulated materials, performs differential fitting on the input sequence of the simulated materials and the device motion data to obtain simulated motion parameters, and sets the baffle spacing and motor speed of the coal gangue transport simulation device according to the device structure through the simulated motion parameters;

[0057] In the actual assessment, the coal and gangue material ratio was extracted based on simulated coal and gangue data. The coal and gangue material was then laid in zones and layers on a coal and gangue transport simulation device. It was laid in three layers according to particle diameter ranges, with a 10-second time interval between the laying of materials in different particle diameter ranges. The time-series encoding of the thickness variation was used as a time series input vector. , For the laying thickness of the first layer at time t, the motor speed, baffle spacing, and lifting rod tilt angle of the time series input vector corresponding to the time series device motion data are extracted. The simulated motor speed is iteratively fitted based on the discrete differential fitting formula, and the calculated value of the 100th frame is 15.3 r / min. Based on the simulated motor speed obtained by fitting, the baffle spacing is adjusted to the coal discharge window spacing of 160 mm at the corresponding time, and the simulated motion parameters are obtained.

[0058] S4 simulates coal and gangue transport based on simulated coal and gangue data and a coal and gangue transport simulation device. It acquires real-time images and coal and gangue video streams based on the simulation transport results and draws the coal and gangue interface through a labeled dataset.

[0059] In the actual evaluation, the material corresponding to the simulated coal gangue data was laid on the device, the simulated motion parameters were loaded, the coal gangue transport simulation device was started, and the dual cameras were turned on to collect real-time images and coal gangue video streams. The real-time images were matched with the coal gangue texture features in the labeled dataset, and the wavelet coefficient modulus, phase, and texture tendency were matched. Threshold segmentation was used, and the coal area and gangue area were obtained according to the preset threshold of 128. The Zhang Zhengyou calibration parameters of binocular vision were used to perform three-dimensional reconstruction through triangulation to obtain the three-dimensional point cloud of the coal gangue interface with a point spacing of 0.5 mm. The three-dimensional point cloud was fitted into a B-spline surface.

[0060] S5 uses the coal-gangue interface and coal-gangue video stream to analyze the coal-gangue migration trend, obtain the coal-gangue migration trajectory, and compare and calibrate the coal-gangue migration trajectory based on the coal discharge window data to obtain simulated coal seam data and simulated coal discharge window data.

[0061] In practical evaluation, based on the coal-gangue interface, the changes in the centroid coordinates, boundary expansion rate, and boundary contraction rate of adjacent frames in the coal-gangue video stream were extracted to obtain boundary feature points. The change in centroid coordinates was less than 2mm per frame, and the boundary expansion rate was 0.1-0.5mm / frame. Large coal-gangue particles with clear textures (diameter greater than 20mm) were marked in the video stream, and their average transport velocity was 1-3cm / s, and their average acceleration was 0-0.2cm / s². 2 The dispersion of the motion direction is less than 12. Using the interface boundary feature points and large coal gangue particles as trajectory feature points, the Kalman filter algorithm is used to perform time-series tracking of the coordinates of the trajectory feature points. The process noise covariance is... The measurement noise covariance is The continuous motion trajectory is obtained, and the continuous trajectory is clustered by Euclidean distance with a neighborhood radius of 10 mm and a minimum sample size of 5 to obtain 3 to 5 main migration trajectories with a clustering purity of more than 90% to obtain the coal gangue migration trajectory.

[0062] In the actual evaluation, the trajectory deviation of the coal and gangue transport trajectory is calculated based on the spatial boundary and time-varying characteristics of the coal discharge window data. The simulation maneuver parameters are iteratively adjusted for the trajectory to be optimized, with fine-tuning of the baffle spacing (+2mm or -2mm each time) and motor speed (+0.2r / min or -0.2r / min each time) to match the coal and gangue transport speed with the window opening and closing rhythm, until the positional deviation is less than or equal to 0.05m and the temporal deviation is less than or equal to 0.2s. This yields optimized simulated coal seam data. The simulated coal seam data has an average coal seam thickness distribution of 65mm, a standard deviation of 5mm, and a coefficient of variation of coal and gangue mixing uniformity of less than 0.2. Based on the 3D reconstruction results, the coal and gangue interface is used as the actual pixel coordinates. The unloaded image corresponding to the coal and gangue interface is converted to theoretical pixel coordinates through perspective projection. The average reprojection error is calculated using the Euclidean distance between the actual and theoretical pixel coordinates. The initial average reprojection error is 1.8 pixels. The percentage error is calculated using a physical size of 0.1mm corresponding to a single pixel as the standard. The matching degree was 98.2%, where 100mm is the width of the coal discharge window. Since the matching degree is greater than 90%, there is no need to adjust the shooting angle. The coal discharge window data can be directly obtained as the coal discharge window simulation data.

[0063] In this embodiment, the method for obtaining the camera angle parameters and the labeled dataset includes:

[0064] Coal and gangue image groups are obtained by dividing the coal and gangue images corresponding to different device movement data. Two-dimensional wavelet convolution is performed on the coal and gangue images based on the coal and gangue material to obtain multi-scale wavelet coefficients and scale weighting factors negatively correlated with scale. The dominant angular components of image features are calculated using the scale weighting factors and the partial derivatives of the wavelet basis functions in the pixel direction. Camera angle parameters are calculated based on the dominant angular components of image features in different coal and gangue image groups and the camera angle. The formula for calculating the camera angle parameters is as follows:

[0065]

[0066]

[0067] in The dominant angular component is the feature of a single frame image. The scale weighting factor is related to the scale of the wavelet transform. negative correlation These are multi-scale wavelet coefficients, representing different scales. In translation amount Image pixels The characteristics of coal gangue at the location, Here are wavelet basis functions, representing Daubechies wavelet and Symlet wavelet. For camera angle parameters, The total number of groups For the first The camera angle of the group is No. Angular components of the group, For the first The initial tilt angle of the setup is obtained from the tilt angle of the bottom friction testing machine. For the first The contribution of the image features of the group is obtained by quantizing the directional consistency of the multi-scale wavelet coefficients. This is the initial tilt angle of the apparatus in the current experiment;

[0068] Based on multi-scale wavelet coefficients, the contrast feature function of coal gangue image pixels is calculated through gray-level co-occurrence matrix. The coal gangue texture tendency of the pixels is marked according to the contrast feature function. The magnitude and phase of multi-scale wavelet coefficients are extracted. Based on the coal gangue texture tendency, magnitude intensity and phase interval, coal gangue images are selected and marked according to preset typical coal gangue features to obtain a labeled dataset.

[0069] In this embodiment, the method for obtaining the simulated coal gangue data includes:

[0070] The camera angle parameters are used to calibrate the viewpoint of the time-series video stream. The inter-frame difference method is used to analyze the dynamic region of coal and gangue movement in the calibrated time-series video stream. The length, width, and position coordinates of the coal discharge window in the dynamic region are used as the spatial boundary. Based on the device's motor data, the opening and closing amplitude of the coal discharge window and the timing of coal and gangue outflow during the coal discharge process are used as time-varying features. The spatial boundary and time-varying features are encoded into coal discharge window data.

[0071] Using coal discharge window data and labeled dataset as conditional information, a random noise vector following a Rosin-Rammler distribution is generated based on the conditional information. The conditional information and random noise vector are then input into the generator of a conditional generative adversarial network. Based on the discriminator of the conditional generative adversarial network, the true and generated binary classification probabilities are output according to the labeled dataset and simulated coal gangue data. The input is then mapped to simulated coal gangue data, which includes coal gangue distribution, particle diameter, and initial velocity of the bulk material.

[0072] In this embodiment, the method for obtaining the simulated maneuver parameters includes:

[0073] Based on simulated coal gangue data, coal gangue materials are extracted. According to the distribution and particle diameter of loose coal gangue in the simulated coal gangue data, coal gangue materials are laid in different regions and layers in the coal gangue transport simulation device to obtain simulated materials. The laying time and laying thickness variation time sequence of materials in different particle size ranges are spliced ​​into a time series input vector, and the time series input vector is used as the input quantity of simulated materials.

[0074] Based on the time series input vector, the corresponding time-series device motion data is extracted. Using the initial motor speed, initial baffle spacing, and initial lifting rod inclination angle of the bottom friction testing machine from the device motion data, the simulated motor speed is iteratively fitted using a discrete differential fitting formula. The discrete differential fitting formula is as follows:

[0075]

[0076] in For a moment The simulated motor speed, The initial motor speed is obtained from the device's motion data. This refers to the frame number of the corresponding time-series video stream. This represents the maximum baffle spacing of the device. For the first The spacing between the coal feeding windows of the frame, For the first The frame's motor speed, For a function with a positive part, take non-negative values. For the first Multiscale wavelet coefficients of frames For the first Multiscale wavelet coefficients of frames For the first The amount of simulated material input in the frame. For the first The velocity modulus of the frame. The magnitude of the discrete velocity in the initial frame. This is a single-view residual correction term, representing the deviation between the actual shooting angle and the camera angle parameter.

[0077] The baffle spacing is adjusted to the corresponding time based on the simulated motor speed to obtain the simulated baffle spacing, which is used as a simulated maneuver parameter.

[0078] In this embodiment, the method for obtaining the coal-gangue interface includes:

[0079] Based on the device structure, the baffle spacing and motor speed of the coal gangue transport simulation device are set by simulating the motor parameters. Based on the simulated coal gangue data, the corresponding simulated materials are obtained. The simulated materials and the coal gangue transport simulation device are used to simulate transport. Real-time images and coal gangue video streams are obtained by the digital camera of the coal gangue transport simulation device. The real-time images are matched with the texture features of coal gangue in the labeled dataset to obtain the coal area and gangue area. Based on the coal area and gangue area, three-dimensional reconstruction is performed according to the vision of the dual digital cameras and the Zhang Zhengyou calibration method to obtain the drawn coal gangue interface.

[0080] In this embodiment, the method for obtaining the simulated coal seam data and the simulated coal release window data includes:

[0081] Based on the coal-gangue interface, the contour centroid coordinate changes, boundary expansion rate, and boundary contraction rate of adjacent frames in the coal-gangue video stream are extracted to obtain boundary feature points. According to the labeled dataset, the average transport velocity, average acceleration, and motion direction dispersion of coal-gangue particles in the coal-gangue video stream are statistically analyzed. The boundary feature points and coal-gangue particles are used as trajectory feature points. Based on the trajectory feature points, the coordinates of the trajectory feature points are time-series tracked using the Kalman filter algorithm to obtain the continuous motion trajectory of the trajectory feature points. The continuous motion trajectory is clustered using Euclidean distance to obtain the coal-gangue transport trajectory.

[0082] The trajectory deviation of the coal and gangue transport trajectory is calculated based on the spatial boundary and time-varying characteristics of the coal discharge window data. If the trajectory deviation of the feature point is greater than 0.1m or the lag time is greater than 0.5 seconds, it is marked as needing optimization. The simulation maneuver parameters are iteratively adjusted according to the overall coal and gangue transport trajectory that needs optimization until the position deviation of the feature point is less than or equal to 0.05m and the time deviation is less than or equal to 0.2 seconds. The marked data of the coal discharge window data in the iterative simulation is obtained. The marked data is used to obtain the simulated coal seam dataset through the coal and gangue interface image, which is used as the simulated coal seam data.

[0083] Based on the 3D reconstruction results, the coal-gangue interface is used as the actual pixel coordinates. The unloaded image corresponding to the coal-gangue interface is converted into theoretical pixel coordinates through perspective projection. The average reprojection error is calculated using the Euclidean distance between the actual pixel coordinates and the theoretical pixel coordinates. The error percentage is calculated based on the average reprojection error with a single pixel as the standard to obtain the camera view matching degree. The camera angle is adjusted until the camera view matching degree is greater than or equal to 90%. The coal discharge window data is extracted based on the optimized camera view matching degree to obtain the simulated data of the coal discharge window.

[0084] A second aspect of the present invention also provides a coal gangue transport simulation device equipped with image recognition technology, comprising:

[0085] Bottom friction testing machine 101, flat plate 102, digital camera 103, bracket 104, lifting rod 105, data processing equipment 106, roller 107, belt 108, baffle 204;

[0086] The flat plate 102 is fixedly installed at one end of the bottom friction testing machine 101z. The bottom friction testing machine 101 includes the roller 107 and the belt 108. The belt 108 is rotated by rotating the roller 107. A lifting rod 105 is installed on the support foot near the flat plate 102. A limiting frame 201 is installed on the upper surface of the bottom friction testing machine 101. The baffle 204 is fixedly installed inside the limiting frame 201. The bracket 104 is installed above the bottom friction testing machine 101. The digital camera 103 is fixedly installed in the middle of the bracket 104. The bottom friction testing machine 101 and the digital camera 103 are electrically connected to the data processing device 106.

[0087] In this embodiment, the flat plate 102 is welded together with a base plate 302 and a gripping rod 301. The gripping rod 301 is welded to the center of one side of the base plate 302. The base plate 302 is a rectangular horizontal plate, and the gripping rod 301 is a cylindrical long rod. The thickness of the base plate is less than or equal to 5 mm. The length-to-height ratio of the base plate 302 to the gripping rod 301 is 1:1.5. The lifting rod 105 is fixedly installed on the support foot of the friction testing machine 101 below the flat plate 102. The lifting rod 105 is a hydraulic rod.

[0088] In this embodiment, the digital camera 103 is fixedly installed at the digital camera installation position 205 and the digital camera installation position 405 respectively. The digital camera at the digital camera installation position 205 is positioned to capture an image range 502 in the horizontal direction of the bottom friction tester 101, and the digital camera at the digital camera installation position 405 is positioned to capture an image range 501 in the upper surface of the bottom friction tester 101.

[0089] The digital camera mounting position 205 is fixedly set at the middle of one end of the inner side of the limiting frame 201.

[0090] The simulated gangue layer 202 and simulated coal seam 203 are fixedly provided on the other end of the inner side of the limiting frame 201, and the baffle 204 is fixedly provided on the side of the simulated coal seam 203 away from the simulated gangue layer 202.

[0091] The digital camera mounting position 405 is fixedly disposed on the top of the bracket 104. The bracket 104 includes a crossbar 401, a crossbar 402, a crossbar 403, a roller 404, and the digital camera mounting position 405. The bottom of the crossbar 401 is connected to the roller 404, and the top of the crossbar 401 is vertically connected to one end of the roller crossbar 402. The top of the crossbar 403 is vertically connected to the other end of the crossbar 402. The crossbar 401 is parallel to the crossbar 402, and the digital camera mounting position 405 is disposed in the middle of the crossbar 403.

[0092] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A method for simulating coal gangue transport using image recognition technology, characterized in that, Includes the following steps: Based on a pre-set coal gangue transport simulation device, coal gangue data and device movement data are collected. The coal gangue data includes coal gangue images, time-series video streams, and coal gangue materials. Wavelet convolution is performed on the coal and gangue images based on the coal and gangue material to obtain camera angle parameters and labeled datasets. Coal discharge window data is extracted using the camera angle parameters and time-series video streams. Simulated coal and gangue data is generated through a conditional generative adversarial network based on the coal discharge window data and labeled datasets. Based on the simulated coal gangue data, simulated materials are laid using coal gangue materials. The input sequence of the simulated materials and the device motion data are differentially fitted to obtain simulated motion parameters. According to the device structure, the baffle spacing and motor speed of the coal gangue transport simulation device are set through the simulated motion parameters. The simulation of coal gangue is carried out based on simulated coal gangue data and a coal gangue transport simulation device. Real-time images and coal gangue video streams are obtained based on the simulation transport results, and the coal gangue interface is drawn by labeling the dataset. The coal and gangue interface and coal and gangue video stream were used to analyze the coal and gangue migration trend and obtain the coal and gangue migration trajectory. The coal and gangue migration trajectory was compared and calibrated based on the coal discharge window data to obtain simulated coal seam data and simulated coal discharge window data.

2. The coal gangue transport simulation method based on image recognition technology according to claim 1, characterized in that, The method for obtaining the camera angle parameters and the labeled dataset includes: Coal and gangue image groups are obtained by dividing the coal and gangue images corresponding to different device movement data. Two-dimensional wavelet convolution is performed on the coal and gangue images based on the coal and gangue material to obtain multi-scale wavelet coefficients and scale weighting factors negatively correlated with scale. The dominant angular components of image features are calculated using the scale weighting factors and the partial derivatives of the wavelet basis functions in the pixel direction. Camera angle parameters are calculated based on the dominant angular components of image features and the camera angle of different coal and gangue image groups. The formula for calculating the camera angle parameters is as follows: ； ； in The dominant angle component is a feature of a single frame image. The scale weighting factor is related to the scale of the wavelet transform. negative correlation These are multi-scale wavelet coefficients, representing different scales. In translation amount Image pixels The characteristics of coal gangue at the location, Here are wavelet basis functions, representing Daubechies wavelet and Symlet wavelet. For camera angle parameters, The total number of groups For the first The camera angle of the group, For the first Angular components of the group, For the first The initial tilt angle of the setup is obtained from the tilt angle of the bottom friction testing machine. For the first The contribution of the image features of the group is obtained by quantizing the directional consistency of the multi-scale wavelet coefficients. This is the initial tilt angle of the apparatus in the current experiment; Based on multi-scale wavelet coefficients, the contrast feature function of coal gangue image pixels is calculated through gray-level co-occurrence matrix. The coal gangue texture tendency of the pixels is marked according to the contrast feature function. The magnitude and phase of multi-scale wavelet coefficients are extracted. Based on the coal gangue texture tendency, magnitude intensity and phase interval, coal gangue images are selected and marked according to preset typical coal gangue features to obtain a labeled dataset.

3. The coal gangue transport simulation method based on image recognition technology according to claim 1, characterized in that, The method for obtaining the simulated coal gangue data includes: The camera angle parameters are used to calibrate the viewpoint of the time-series video stream. The inter-frame difference method is used to analyze the dynamic region of coal and gangue movement in the calibrated time-series video stream. The length, width, and position coordinates of the coal discharge window in the dynamic region are used as the spatial boundary. Based on the device's motor data, the opening and closing amplitude of the coal discharge window and the timing of coal and gangue outflow during the coal discharge process are used as time-varying features. The spatial boundary and time-varying features are encoded into coal discharge window data. Using coal discharge window data and labeled dataset as conditional information, a random noise vector following a Rosin-Rammler distribution is generated based on the conditional information. The conditional information and random noise vector are then input into the generator of a conditional generative adversarial network. Based on the discriminator of the conditional generative adversarial network, the true and generated binary classification probabilities are output according to the labeled dataset and simulated coal gangue data. The input is then mapped to simulated coal gangue data, which includes coal gangue distribution, particle diameter, and initial velocity of the bulk material.

4. The coal gangue transport simulation method based on image recognition technology according to claim 1, characterized in that, The method for obtaining the simulated maneuver parameters includes: Based on simulated coal gangue data, coal gangue materials are extracted. According to the distribution and particle diameter of loose coal gangue in the simulated coal gangue data, coal gangue materials are laid in different regions and layers in the coal gangue transport simulation device to obtain simulated materials. The laying time and laying thickness variation time sequence of materials in different particle size ranges are spliced ​​into a time series input vector, and the time series input vector is used as the input quantity of simulated materials. Based on the time series input vector, the corresponding time-series device motion data is extracted. Using the initial motor speed, initial baffle spacing, and initial lifting rod inclination angle of the bottom friction testing machine from the device motion data, the simulated motor speed is iteratively fitted using a discrete differential fitting formula. The discrete differential fitting formula is as follows: ； in For a moment The simulated motor speed, The initial motor speed is obtained from the device's motion data. This refers to the frame number of the corresponding time-series video stream. This represents the maximum baffle spacing of the device. For the first The spacing between the coal feeding windows of the frame, For the first The frame's motor speed, For a function with a positive part, take non-negative values. For the first Multiscale wavelet coefficients of a frame For the first Multiscale wavelet coefficients of a frame For the first The amount of simulated material input in the frame. For the first The velocity modulus of the frame. The magnitude of the discrete velocity in the initial frame. This is a single-view residual correction term, representing the deviation between the actual shooting angle and the camera angle parameter. The baffle spacing is adjusted to the corresponding time based on the simulated motor speed to obtain the simulated baffle spacing, which is used as a simulated maneuver parameter.

5. The coal gangue transport simulation method based on image recognition technology according to claim 1, characterized in that, The method for obtaining the coal-gangue interface includes: Based on the device structure, the baffle spacing and motor speed of the coal gangue transport simulation device are set by simulating the motor parameters. Based on the simulated coal gangue data, the corresponding simulated materials are obtained. The simulated materials and the coal gangue transport simulation device are used to simulate transport. Real-time images and coal gangue video streams are obtained by the digital camera of the coal gangue transport simulation device. The real-time images are matched with the texture features of coal gangue in the labeled dataset to obtain the coal area and gangue area. Based on the coal area and gangue area, three-dimensional reconstruction is performed according to the vision of the dual digital cameras and the Zhang Zhengyou calibration method to obtain the drawn coal gangue interface.

6. The coal gangue transport simulation method based on image recognition technology according to claim 1, characterized in that, The method for obtaining the simulated coal seam data and the simulated coal release window data includes: Based on the coal-gangue interface, the contour centroid coordinate changes, boundary expansion rate, and boundary contraction rate of adjacent frames in the coal-gangue video stream are extracted to obtain boundary feature points. According to the labeled dataset, the average transport velocity, average acceleration, and motion direction dispersion of coal-gangue particles in the coal-gangue video stream are statistically analyzed. The boundary feature points and coal-gangue particles are used as trajectory feature points. Based on the trajectory feature points, the coordinates of the trajectory feature points are time-series tracked using the Kalman filter algorithm to obtain the continuous motion trajectory of the trajectory feature points. The continuous motion trajectory is clustered using Euclidean distance to obtain the coal-gangue transport trajectory. The trajectory deviation of the coal and gangue transport trajectory is calculated based on the spatial boundary and time-varying characteristics of the coal discharge window data. If the trajectory deviation of the feature point is greater than 0.1m or the lag time is greater than 0.5 seconds, it is marked as needing optimization. The simulation maneuver parameters are iteratively adjusted according to the overall coal and gangue transport trajectory that needs optimization until the position deviation of the feature point is less than or equal to 0.05m and the time deviation is less than or equal to 0.2 seconds. The marked data of the coal discharge window data in the iterative simulation is obtained. The marked data is used to obtain the simulated coal seam dataset through the coal and gangue interface image, which is used as the simulated coal seam data. Based on the 3D reconstruction results, the coal-gangue interface is used as the actual pixel coordinates. The unloaded image corresponding to the coal-gangue interface is converted into theoretical pixel coordinates through perspective projection. The average reprojection error is calculated using the Euclidean distance between the actual pixel coordinates and the theoretical pixel coordinates. The error percentage is calculated based on the average reprojection error with a single pixel as the standard to obtain the camera view matching degree. The camera angle is adjusted until the camera view matching degree is greater than or equal to 90%. The coal discharge window data is extracted based on the optimized camera view matching degree to obtain the simulated data of the coal discharge window.

7. A coal and gangue transport simulation device equipped with image recognition technology, used to execute the coal and gangue transport simulation method equipped with image recognition technology as described in any one of claims 1 to 6, characterized in that, The device includes: a bottom friction testing machine (101), a flat plate (102), a digital camera (103), a bracket (104), a lifting rod (105), a data processing device (106), and a baffle (204). The flat plate (102) is fixedly installed at one end of the bottom friction tester (101). A lifting rod (105) is installed on the support foot of the bottom friction tester (101) near the flat plate (102). A limiting frame (201) is installed on the upper surface of the bottom friction tester (101). The baffle (204) is fixedly installed inside the limiting frame (201). The bracket (104) is installed above the bottom friction tester (101). The digital camera (103) is fixedly installed in the middle of the bracket (104). The bottom friction tester (101) and the digital camera (103) are electrically connected to the data processing device (106).

8. A coal gangue transport simulation device equipped with image recognition technology according to claim 7, characterized in that, include: The flat plate (102) is welded together with a base plate (302) and a handle (301). The handle (301) is welded to the center of one side of the base plate (302). The base plate (302) is a rectangular horizontal plate. The handle (301) is a cylindrical long rod. The thickness of the base plate is less than or equal to 5 mm. The length-to-height ratio of the base plate (302) to the handle (301) is 1:1.

5. The lifting rod (105) is fixedly installed on the support foot of the bottom friction testing machine (101) below the flat plate (102). The lifting rod (105) is a hydraulic rod.

9. A coal gangue transport simulation device equipped with image recognition technology according to claim 7, characterized in that, include: The digital cameras (103) are fixedly installed at the first digital camera installation position (205) and the second digital camera installation position (405). The digital camera at the first digital camera installation position (205) is directed towards the horizontal direction of the bottom friction tester (101) to capture the image range (502). The digital camera at the second digital camera installation position (405) is directed towards the upper surface of the bottom friction tester (101) to capture the image range (501). The first digital camera installation position (205) is fixedly set at the middle of one end of the inner side of the limiting frame 201. The other end of the inner side of the limiting frame (201) is fixedly set with a simulated gangue layer (202) and a simulated coal seam (203). The baffle (204) is fixedly set on the side of the simulated coal seam (203) away from the simulated gangue layer (202). The second digital camera mounting position (405) is fixedly set on the top of the bracket (104). The bracket (104) includes a first crossbar (401), a second crossbar (402), a third crossbar (403), a roller (404), and the second digital camera mounting position (405). The bottom of the first crossbar (401) is connected to the roller (404), and the top of the first crossbar (401) is vertically connected to one end of the roller and the second crossbar (402). The top of the third crossbar (403) is vertically connected to the other end of the second crossbar (402). The first crossbar (401) is parallel to the second crossbar (402), and the second digital camera mounting position (405) is set in the middle of the third crossbar (403).

Images