Method and system for monitoring slag falling amount of slag drying machine in real time

Abstract

The invention relates to a real-time monitoring method and system for the slag falling amount of a slag drying machine. The method comprises the following steps: acquiring multi-modal data of falling ash at a slag falling opening of a slag drying machine; preprocessing the multi-modal data to obtain preprocessed multi-modal data; fusing the preprocessed multi-modal data to obtain fused data; performing ash texture feature extraction on the fused data by using the trained deep learning model to obtain an ash texture feature vector; according to the ash texture feature vector, performing ash classification on each point in the fusion data to obtain an ash classification result of each point in the fusion data; and calculating the total volume of ash particles and the total volume of slag blocks in the falling ash according to the preprocessed multi-modal data and the ash classification result. According to the method, the specific content of ash and slag can be accurately measured, so that a power plant can reasonably plan a flow according to the specific content of ash and slag in ash slag in a subsequent treatment link, resource waste is effectively avoided, the treatment cost is reduced, and the production refinement level is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated monitoring, and particularly relates to a real-time monitoring method and system for the slag dropping amount of a dry slag machine. Background Art

[0002] In the daily production and operation of power plants, the effective monitoring of the slag dropping amount of dry slag machines is of great significance to the entire power generation process. On the one hand, the slag dropping amount is related to the stable operation of the boiler, the combustion efficiency of the boiler, and the energy utilization rate; on the other hand, the reasonable treatment of the dropped slag is also crucial for environmental protection and resource recycling and reuse.

[0003] In the prior art, the slag dropping amount monitoring technology commonly used in power plants mainly focuses on monitoring the total amount of ash and slag. Although this monitoring method can meet the basic requirements for controlling the total amount of ash and slag, it exposes obvious shortcomings when facing complex subsequent processing. In the subsequent ash and slag treatment process of power plants, due to different physical and chemical properties, ash and slag have completely different treatment methods and application routes. The current technology cannot accurately know the respective contents of ash and slag, making it difficult for power plants to reasonably plan the treatment process, which is extremely likely to cause resource waste and cost increase. Summary of the Invention

[0004] Based on this, it is necessary to provide a real-time monitoring method and system for the slag dropping amount of a dry slag machine that can calculate the total volume of ash particles and slag blocks in the falling ash and slag in real time in view of the above technical problems.

[0005] In a first aspect, the present application provides a real-time monitoring method for the slag dropping amount of a dry slag machine, including:

[0006] Obtaining multimodal data of the falling ash and slag at the slag dropping port of the dry slag machine; the multimodal data includes image data, point cloud data, and sensor data;

[0007] Preprocessing the multimodal data to obtain preprocessed multimodal data; fusing the preprocessed multimodal data to obtain fused data;

[0008] Using a trained deep learning model to extract ash and slag texture features from the fused data to obtain an ash and slag texture feature vector; the ash and slag texture feature vector characterizes the texture structure of the falling ash and slag in three-dimensional space;

[0009] According to the ash and slag texture feature vector, classifying each point in the fused data into ash or slag to obtain the ash and slag classification result of each point in the fused data, and the ash and slag classification result is ash or slag;

[0010] Calculating the total volume of ash particles and the total volume of slag blocks in the falling ash and slag according to the preprocessed multimodal data and the ash and slag classification result.

[0011] In a second aspect, the present application further provides a real-time monitoring system for the slag dropping amount of a dry slag machine, including:

[0012] A multi-modal data acquisition module, configured to acquire multi-modal data of the falling ash slag at the slag dropping opening of the dry slag machine; the multi-modal data includes image data, point cloud data, and sensor data;

[0013] A data preprocessing and fusion module, configured to preprocess the multi-modal data to obtain preprocessed multi-modal data; fuse the preprocessed multi-modal data to obtain fused data;

[0014] An ash slag texture feature extraction module, configured to use a trained deep learning model to extract ash slag texture features from the fused data to obtain an ash slag texture feature vector; the ash slag texture feature vector characterizes the texture structure of the falling ash slag in three-dimensional space;

[0015] An ash slag classification module, configured to classify each point in the fused data according to the ash slag texture feature vector to obtain the ash slag classification result of each point in the fused data, and the ash slag classification result is ash or slag;

[0016] An ash slag dropping amount calculation module, configured to calculate the total volume of ash particles and the total volume of slag blocks in the falling ash slag according to the preprocessed multi-modal data and the ash slag classification result.

[0017] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, it implements a real-time monitoring method for the slag dropping amount of a dry slag machine as in the first aspect.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements a real-time monitoring method for the slag dropping amount of a dry slag machine as in the first aspect.

[0019] For the above real-time monitoring method and system for the slag dropping amount of a dry slag machine, by acquiring multi-modal data such as images, point clouds, and sensors of the falling ash slag at the slag dropping opening of the dry slag machine, preprocessing and fusing the multi-modal data, using a trained deep learning model to extract the ash slag texture feature vector, classifying the ash slag for the points in the fused data, and calculating the total volume of ash particles and slag blocks based on the preprocessed multi-modal data and the classification result. This method can accurately measure the specific content of ash and slag, so that the power plant can reasonably plan the process according to the specific content of ash and slag in the ash slag in the subsequent processing link, effectively avoid waste of resources, reduce the processing cost, and improve the production refinement level. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0021] Figure 1 It is a schematic flow chart of a method for real-time monitoring of the slag falling amount of a dry slag machine provided by the present invention;

[0022] Figure 2 It is a schematic structural diagram of a system for real-time monitoring of the slag falling amount of a dry slag machine provided by the present invention. Detailed implementation manners

[0023] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following further details the present application in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] Refer to Figure 1 , which shows a schematic flow chart of a method for real-time monitoring of the slag falling amount of a dry slag machine provided by the present application. The method includes the following steps:

[0025] S101: Obtain multi-modal data of the falling ash slag at the slag falling port of the dry slag machine; the multi-modal data includes image data, point cloud data and sensor data.

[0026] Specifically, for the acquisition of image data: An industrial camera can be installed at a suitable position near the slag falling port of the dry slag machine. By setting appropriate shooting parameters, such as frame rate, resolution, exposure time, etc., it is ensured that the appearance characteristics of the falling ash slag can be clearly captured, including color, shape, surface details, etc.

[0027] For the acquisition of point cloud data: A lidar device can be used to scan the area of the slag falling port of the dry slag machine. The lidar emits laser beams and receives the reflected light, and can accurately measure the position information of the ash slag in space, thereby generating point cloud data. This data can reflect the distribution form of the ash slag in three-dimensional space, including the volume contour and spatial position relationship of the ash slag.

[0028] For the acquisition of sensor data: Multiple types of sensors, such as pressure sensors and temperature sensors, can be arranged around the slag falling port. The pressure sensor can measure the pressure change generated when the falling ash slag impacts the receiving surface below the slag falling port, indirectly reflecting information such as the mass or flow rate of the ash slag; the temperature sensor is used to monitor the temperature of the ash slag, and different temperature characteristics may be related to the composition and state of the ash slag. Through the above sensors, physical characteristic data related to the ash slag is obtained.

[0029] S102: Preprocess the multi-modal data to obtain preprocessed multi-modal data; fuse the preprocessed multi-modal data to obtain fused data.

[0030] Specifically, for the preprocessing of image data: First, the image data can be enhanced through methods such as histogram equalization and contrast stretching to improve the clarity and detail expressiveness of the image, making the characteristics of the ash more easily recognizable. Then, the image data is filtered, for example, using algorithms such as Gaussian filtering and median filtering to remove noise interference in the image, such as random noise generated during the camera imaging process, to improve the image quality. Next, the image data is segmented, separating the ash area in the image from the background, for example, using methods such as threshold segmentation, edge detection, and machine learning segmentation to obtain an accurate ash image area.

[0031] For the preprocessing of point cloud data: First, the point cloud data can be denoised, for example, through algorithms such as statistical filtering and radius filtering to remove outlier points in the point cloud data caused by measurement errors or environmental interference, making the point cloud data more accurately reflect the true shape of the ash. Then, the point cloud data is registered. When there are multiple sets of point cloud data, they are unified into the same coordinate system for subsequent analysis. The point cloud data can also be thinned, for example, using voxel filtering to reduce the amount of point cloud data without losing key features and improve the processing efficiency.

[0032] For the preprocessing of sensor data: The data collected by the sensor can be calibrated. According to the characteristics of the sensor and known standard samples, the measurement data is corrected to improve the accuracy of the data. Then, the data collected by the sensor is smoothed, for example, through methods such as moving average and weighted average to remove high-frequency noise and fluctuations in the data, making the data more stable and regular.

[0033] For data fusion: The feature-level fusion method can be used to extract the features of each modal data, such as extracting texture features from images, geometric features from point cloud data, and physical property features from sensor data, and then combining these features into a new feature vector as the fused data. The decision-level fusion method can also be used. First, preliminary analysis and decisions are made based on each modal data respectively, such as judging the approximate category of the ash based on image data, estimating the volume range of the ash based on point cloud data, and inferring the physical state of the ash based on sensor data. Finally, these decision results are integrated to obtain the fused data.

[0034] S103: Use the trained deep learning model to extract the ash texture features from the fused data to obtain an ash texture feature vector; the ash texture feature vector characterizes the texture structure of the falling ash in three-dimensional space.

[0035] Specifically, the deep learning model can select models suitable for processing multi-modal data, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) and their variants (such as LSTMs, GRUs), etc.

[0036] When training the model, a large number of labeled slag fusion data are prepared as the training set, and the labeling information can include the true texture features, categories, etc. of the slag. During the training process, the parameters of the model are continuously adjusted through the backpropagation algorithm to minimize the error between the prediction result of the model and the labeling information. For example, the cross-entropy loss function is used to measure the difference between the prediction result and the true label, and optimization algorithms such as stochastic gradient descent are used to update the model parameters. After multiple rounds of iterative training, the model achieves better performance.

[0037] After completing the model training, the fusion data is input into the trained deep learning model. The model processes the data layer by layer and extracts features through its internal network structure, and finally outputs a slag texture feature vector. Each element in this vector represents the texture feature of the slag in a specific dimension, and the overall vector comprehensively characterizes the texture structure of the falling slag in three-dimensional space.

[0038] S104: According to the slag texture feature vector, classify each point in the fusion data into slag, and obtain the slag classification result of each point in the fusion data. The slag classification result is ash or slag.

[0039] Specifically, the classification algorithm can adopt classification algorithms based on machine learning, such as support vector machines (SVMs), random forests, etc., or utilize the classification ability of the deep learning model. For example, during the training process of the deep learning model, a classification layer, such as a Softmax layer, can be added to the last layer of the network to map the slag texture feature vector to two categories of ash and slag, and by calculating the probability values corresponding to each category, the classification result of each point is determined.

[0040] During classification, the feature vector corresponding to each point in the fusion data (based on the previously extracted slag texture feature vector) is input into the classification algorithm. For algorithms based on machine learning, such as SVM, by finding the optimal classification hyperplane, the feature vector is divided into the categories of ash or slag; for the deep learning model, according to the probability values calculated by the network, the category with the highest probability is selected as the classification result of this point. Through the above method, all points in the fusion data are classified to obtain the slag classification result of each point, clearly distinguishing between ash and slag.

[0041] S105: According to the preprocessed multi-modal data and the slag classification result, calculate the total volume of ash particles and the total volume of slag blocks in the falling slag.

[0042] Specifically, for the point cloud part classified as ash, the 3D surface model of ash particles can be constructed by using point cloud data processing algorithms such as Delaunay triangulation, etc., and then the volume of ash particles can be obtained by calculating the volume contained in this model. Similarly, for the point cloud part classified as slag, the 3D surface model of slag blocks is constructed and its volume is calculated. For example, in the point cloud data, according to the ash-slag classification result, the point set belonging to ash and the point set belonging to slag are extracted, and 3D modeling and volume calculation are respectively performed on these two point sets.

[0043] At the same time, when calculating the volume, image data can also be combined to assist the calculation. For example, the two-dimensional contour information of ash and slag provided by the image data can be used to assist in the calibration and optimization of volume calculation. For example, the two-dimensional contour of ash and slag obtained by image segmentation can be projected and compared with the point cloud data on the two-dimensional plane to check whether the 3D model constructed by the point cloud data is accurate, and the volume calculation result is corrected. If there is a difference between the ash-slag area shown in the image and the area projected by the point cloud model, the parameters of the point cloud model can be appropriately adjusted to calculate the volume of ash particles and slag blocks more accurately.

[0044] Finally, the volumes of each ash particle and slag block calculated from the point cloud data are accumulated to obtain the total volume of ash particles and the total volume of slag blocks in the falling ash and slag respectively. During the calculation process, considering the measurement accuracy and data error, reasonable error analysis and estimation can be performed on the results to ensure the accuracy of the total volume calculation.

[0045] The above real-time monitoring method for the slag discharge amount of a dry slag machine, by acquiring multi-modal data such as images, point clouds and sensors of the falling ash and slag at the slag discharge port of the dry slag machine, after preprocessing and fusing the multi-modal data, uses a trained deep learning model to extract the texture feature vectors of ash and slag, classifies the points in the fused data into ash and slag, and calculates the total volumes of ash particles and slag blocks based on the preprocessed multi-modal data and classification results. This method can accurately measure the specific contents of ash and slag, so that the power plant can reasonably plan the process according to the specific contents of ash and slag in the ash and slag in the subsequent processing links, effectively avoid resource waste, reduce the processing cost, and improve the production refinement level.

[0046] In an alternative embodiment, acquiring the multi-modal data of the falling ash and slag at the slag discharge port of the dry slag machine includes the following steps:

[0047] Using an image acquisition device to acquire multi-angle image data of the falling ash and slag;

[0048] Using a laser scanning device to scan the surface contour of the falling ash and slag at a specific frequency to generate laser point cloud data;

[0049] Using a laser velocity sensor, based on the principle of the Doppler effect of laser, the instantaneous velocity vector of the falling ash residue is measured to obtain velocity data;

[0050] The multi-angle image data, laser point cloud data, and velocity data are used as multi-modal data.

[0051] Specifically, to obtain comprehensive appearance information of the ash residue, the falling ash residue is photographed from multiple angles. At least three cameras with different angles can be set, for example, cameras are installed at the positions directly in front of the slag outlet, 45° above the side, and 30° obliquely below the other side respectively. By taking multi-angle photos in this way, images of different sides of the ash residue can be obtained.

[0052] For the selection of the laser scanning device, a 3D lidar can be used. Such devices have the characteristics of high precision, high resolution, and fast scanning, and can accurately measure the three-dimensional coordinate information of each point on the surface of the ash residue.

[0053] The selection of the specific frequency can depend on multiple factors, including the falling speed of the ash residue, the requirements for scanning accuracy, and the performance of the device, etc. If the falling speed of the ash residue is relatively fast, in order to accurately capture the dynamic changes of its surface contour, a higher scanning frequency is required, such as 100 - 200 scans per second; while for the ash residue with a slower falling speed, a lower scanning frequency (such as 50 - 100 scans per second) can meet the requirements.

[0054] The principle of generating point cloud data is as follows: when the laser scanning device works, it emits a laser beam to the surface of the ash residue and receives the laser signal reflected from the surface of the ash residue. By measuring the time difference between the emission and reception of the laser beam and combining the propagation speed of the laser, the distance between the lidar and each point on the surface of the ash residue is calculated. At the same time, the internal angle measurement device of the device is used to determine the emission angle of the laser beam. According to the distance and angle information, the coordinate positions of each point on the surface of the ash residue in the three-dimensional space can be calculated. As the lidar rotates or translates for scanning, a large amount of point coordinate information is continuously obtained, and these points ultimately form the point cloud data reflecting the surface contour of the ash residue.

[0055] The working principle of the laser velocity sensor is as follows: when the laser beam irradiates the falling ash residue particles, due to the movement of the ash residue particles, the frequency of the reflected light will change, and this frequency change is proportional to the movement speed of the ash residue particles. The laser velocity sensor detects the frequency difference (i.e., Doppler frequency shift) between the reflected light and the emitted light, and uses a specific formula to calculate the speed of the ash residue particles. For example, according to the Doppler effect formula where v a is the object movement speed, λ is the laser wavelength, f d is the Doppler frequency shift, and θ is the angle between the laser beam and the object movement direction.

[0056] When measuring the instantaneous velocity vector of the falling ash residue, the sensor emits multiple laser beams, irradiating the ash residue particles from different angles, and determines the instantaneous velocity vector of the ash residue in three-dimensional space by measuring and calculating the velocity components in multiple directions.

[0057] In an alternative embodiment, the real-time monitoring method for the slag discharge amount of the dry slag machine further includes: calibrating the image acquisition device using a calibration plate to obtain calibration data.

[0058] Preprocessing the multi-modal data to obtain preprocessed multi-modal data, including the following steps:

[0059] S11: Perform first preprocessing on the multi-angle image data to obtain preprocessed image data.

[0060] Among them, the first preprocessing includes:

[0061] Perform optical distortion correction processing on the multi-angle image data according to the calibration data;

[0062] Use the bilateral filtering algorithm to denoise the multi-angle image data.

[0063] S12: Perform second preprocessing on the laser point cloud data to obtain preprocessed point cloud data.

[0064] Among them, the second preprocessing includes:

[0065] According to the installation attitude and installation position of the laser scanner, combined with the calibration data, unify the coordinate system of the laser point cloud data with the coordinate system of the multi-angle image data;

[0066] Use the statistical filtering algorithm to remove the outliers in the laser point cloud data.

[0067] S13: Associate and store the velocity data with the preprocessed point cloud data according to a specific timestamp to obtain preprocessed velocity data.

[0068] Fuse the preprocessed multi-modal data to obtain fused data, including the following steps:

[0069] Using the calibration data, map the two-dimensional image pixel coordinates of the preprocessed image data to three-dimensional space, and combine them with the preprocessed point cloud data at the corresponding position and the same moment to form three-dimensional point cloud data with texture information;

[0070] Take the preprocessed velocity data as the motion attribute of each point in the three-dimensional point cloud data and attach it to the three-dimensional point cloud data to form fused data.

[0071] Specifically, a calibration board can use a checkerboard calibration board or a dot calibration board. The size and pattern spacing of the calibration board can be determined according to the field of view of the image acquisition device and the measurement accuracy requirements. For a multi-angle image acquisition device, the calibration board is photographed multiple times from the perspective of each camera.

[0072] The calibration process and principle are as follows: Use computer vision algorithms (such as Zhang Zhengyou calibration method) to process the photographed calibration board images. This algorithm is based on the projection transformation relationship from the plane to the image. By detecting the corner points or dot points on the calibration board, the conversion relationships between the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system are established. During the calibration process, according to the known size and pattern spacing of the calibration board, the internal parameter matrix (including focal length, principal point position, etc.) and the external parameter matrix (the rotation and translation parameters of the camera) of the camera are calculated. These parameters are used as calibration data, which can reflect the imaging characteristics and position and attitude of the camera.

[0073] For step S11:

[0074] Optical distortion mainly includes radial distortion and tangential distortion, which is caused by the physical characteristics of the camera lens. Using the internal parameter matrix and external parameter matrix obtained during the calibration process, according to the distortion model (such as the Brown distortion model), the pixel coordinates in the image are converted into ideal undistorted coordinates.

[0075] For radial distortion, a polynomial function can be used to describe it. Its correction formula is x corrected = x(1 + k1r 2 + k2r 4 + k3r 6 ) and y corrected = y(1 + k1r 2 + k2r 4 + k3r 6 ), where x and y are the distorted pixel coordinates, x corrected and y corrected are the corrected pixel coordinates, r is the distance from the pixel point to the center of the image, and k1, k2, k3 are the radial distortion coefficients.

[0076] For tangential distortion, the correction formula is x corrected = x + [2p1xy + p2(r 2 + 2x 2 )] and y corrected = y + [p1(r 2 + 2y 2 ) + 2p2xy], where p1 and p2 are the tangential distortion coefficients. Through the above correction formula, each pixel point in the image can be corrected for distortion, so that the straight lines in the image appear as straight lines in the physical world, improving the geometric accuracy of the image.

[0077] The bilateral filtering algorithm is a non-linear filtering method that can preserve the edge information of an image while removing noise. Its principle is to determine the weights based on the spatial distance and gray-scale difference between pixels. The calculation formula is where I filtered (x, y) is the pixel value after filtering, I(i, j) is the original pixel value, f(i, j) is the spatial distance weight, g(‖p - q‖) is the gray-scale difference weight, and W p is the normalization factor. This algorithm can effectively remove noise in the image, such as Gaussian noise, salt-and-pepper noise, etc., making the image smoother, while avoiding blurring important edges and details in the image.

[0078] For step S12:

[0079] The installation attitude and position of the laser scanner can be represented by measuring the rotation matrix R and translation vector t of the laser scanner relative to a certain reference coordinate system (such as the world coordinate system). Using the external parameter matrix of the camera in the calibration data, the laser point cloud data is transformed from its own scanner coordinate system to the world coordinate system, and then from the world coordinate system to the camera coordinate system. For example, for the point P in the point cloud data laser , the coordinate transformation is performed through the transformation formula . Among them, P laser represents the coordinates of the point in the laser scanner coordinate system, that is, the original coordinates of the initially acquired point cloud data; represents the rotation matrix from the laser scanner coordinate system to the world coordinate system. represents the translation vector from the laser scanner coordinate system to the world coordinate system; represents the rotation matrix from the world coordinate system to the camera coordinate system; represents the translation vector from the world coordinate system to the camera coordinate system. This method can unify the laser point cloud data and multi-angle image data into the same coordinate system, facilitating subsequent fusion and joint analysis.

[0080] The statistical filtering algorithm determines whether a point is an outlier based on the distance distribution between each point in the point cloud data and its neighbor points. For each point in the point cloud, calculate the average distance from it to a certain number of neighbor points, and compare this average distance with a set threshold. Suppose the average distance of the k neighbor points of point p is d. If d is greater than the set threshold d th , then this point is considered an outlier and is removed from the point cloud data. This threshold can be determined through statistical analysis according to the density and distribution of the point cloud. For example, it can be set according to the average distance and standard deviation of the point cloud, such as d th= μ + kσ, where μ is the average distance; σ is the standard deviation; k is an adjustable coefficient, usually taking values between 1 and 3. This method can effectively remove outliers caused by measurement errors, environmental noise, or other interference factors, enabling the point cloud data to more accurately reflect the actual shape and position of the ash residue.

[0081] For step S13:

[0082] Both the velocity data and the point cloud data contain corresponding timestamp information. By comparing the timestamps of the velocity data and the point cloud data, the velocity data and the point cloud data at the same moment or at similar moments are associated. For example, for a point P in the point cloud data, find its acquisition time t P , and then search for the velocity measurement result v P in the velocity data that is closest to t b , and store v b as the velocity attribute of point P.

[0083] After the above preprocessing steps for the multi-modal data, the preprocessed multi-modal data is fused to obtain fused data. According to the internal parameter matrix and the external parameter matrix in the calibration data, the pixel coordinates in the preprocessed image data can be converted into coordinates in three-dimensional space. For example, using the projection formulas X = (u - c x )Z / f x and Y = (v - c y )Z / f y , where u and v are pixel coordinates, c x and c y are the principal point coordinates, f x and f y are the focal lengths, and Z is the depth information. For each pixel point in the image, it is mapped to three-dimensional space in this way.

[0084] Fuse the mapped three-dimensional pixel points with the preprocessed point cloud data at the same position and at the same moment. Since the point cloud data is already in three-dimensional space, for the pixel points and the point cloud points at the same position, the texture information in the image (such as color, grayscale value, etc.) can be assigned to the point cloud points, thus forming three-dimensional point cloud data with texture information. This fusion method can enrich the information of the point cloud data, enabling the point cloud to contain not only the spatial position information of the ash residue but also the texture information.

[0085] For each point in the three-dimensional point cloud data, add the velocity data associated with it as the motion attribute of the point. In this way, each point not only has position information and texture information but also has velocity information, forming a more complete data set.

[0086] In an alternative embodiment, the real-time monitoring method for the slag discharge amount of the dry slag machine further includes: obtaining ambient light data at the slag discharge port of the dry slag machine by using a light sensor. The first preprocessing further includes: adaptively adjusting the brightness and contrast of the multi-angle image data by using an enhancement algorithm based on the Retinex theory in combination with the ambient light data.

[0087] Specifically, the Retinex theory is a color constancy theory based on the human visual system, aiming to simulate the mechanism of the human eye perceiving the color and brightness of objects. Its core idea is to decompose an image into a reflection component and an illumination component, and to achieve image enhancement by estimating and adjusting the illumination component. The basic formula of the Retinex theory is I(x,y) = L(x,y)R(x,y), where I(x,y) is the observed image, L(x,y) is the illumination component, and R(x,y) is the reflection component.

[0088] In an alternative embodiment, the deep learning model is a slag texture feature extraction model based on an improved 3D Transformer, including:

[0089] An input layer for receiving the fusion data; wherein, the input layer performs spatial partitioning processing on the input fusion data through a spatial indexing algorithm, and the spatial partitioning processing includes: dividing the points in the fusion data that are adjacent in space and have similar spatial positions and velocity states into multiple local regions.

[0090] A feature extraction layer based on 3D convolution for extracting features from the fusion data; wherein, the 3D convolution kernel of the feature extraction layer slides in three-dimensional space to perform a convolution operation on the fusion data of the local region, and extracts the texture features, geometric shape inherent features, and dynamic features brought by the motion attributes of the slag as local region features; fuses and encodes the local region features to generate a low-level feature map.

[0091] A 3D Transformer encoding layer for performing deep feature extraction on the low-level feature map; wherein, the encoding layer is composed of multiple 3D Transformer blocks, and each 3D Transformer block uses a multi-head self-attention mechanism in the three-dimensional space dimension and the velocity dimension to capture the correlation of the slag texture changes at different speeds and different spatial positions, and outputs the correlation features; each 3D Transformer block uses a feed-forward neural network to perform a deep non-linear transformation on the correlation features to obtain deep features.

[0092] A feature fusion layer for, after performing global pooling processing on the deep features, splicing and fusing the global feature vector obtained by the global pooling processing with the local region features to generate a comprehensive texture feature vector as the slag texture feature vector.

[0093] Among them, the trained deep learning model is used to extract the ash texture features from the fused data, and an ash texture feature vector is obtained, including: inputting the fused data into the trained ash texture feature extraction model based on the improved 3D Transformer, and outputting the ash texture feature vector.

[0094] Specifically, the function of the input layer is, first, as the entrance of the deep learning model, to receive the fused data, which contains ash information collected from different angles, such as three-dimensional point cloud data with texture information and additional velocity data, etc. Second, the input fused data is spatially partitioned through a spatial indexing algorithm, which aims to divide points that are adjacent in space and have similar spatial positions and velocity states into multiple local regions. For example, an algorithm based on spatial hashing can be used to map points to different hash buckets according to their three-dimensional spatial coordinates and velocity vectors, and each hash bucket can be regarded as a local region.

[0095] For the feature extraction layer based on 3D convolution:

[0096] The 3D convolution kernel slides in the three-dimensional space and performs convolution operations on the fused data in the local region. The size and shape of the convolution kernel can be designed according to needs, such as using a 3×3×3 or 5×5×5 convolution kernel.

[0097] During the convolution process, the convolution kernel multiplies and sums with the data in the local region element by element, extracting the texture features of the ash, the inherent geometric shape features, and the dynamic features brought by the motion attributes. For example, for texture features, the convolution kernel can learn the local patterns of the ash surface texture; for the inherent geometric shape features, the convolution kernel can capture the local shape information of the ash, such as edges, corners, etc.; for the dynamic features brought by the motion attributes, the convolution kernel can extract the features generated due to speed changes, such as the texture deformation on the ash surface at different speeds.

[0098] The extracted local region features are fused and encoded to generate a low-level feature map. This process is similar to the feature map generation in traditional 2D convolutional neural networks, but since it is carried out in three-dimensional space, more dimensional information is considered. The fusion encoding can be achieved through simple summation, weighted averaging, or more complex connection operations, combining the features extracted by different convolution kernels to form a low-level feature map.

[0099] For the 3D Transformer encoding layer:

[0100] Each 3D Transformer block applies the multi-head self-attention mechanism in the three-dimensional space dimension and the velocity dimension. The multi-head self-attention mechanism allows the model to simultaneously focus on the correlation of ash texture changes at different spatial positions and velocities.

[0101] For the input low-level feature map, it is transformed into query, key, and value vectors through multiple different linear mappings. In the three-dimensional space and velocity dimension, the correlation scores between each query vector and all key vectors are calculated, and based on these scores, the value vectors are weighted and summed to obtain the attention information.

[0102] The multi-head self-attention mechanism learns information from different representation subspaces by parallelly using multiple attention heads, capturing multi-faceted correlation information of ash slag textures at different speeds and different spatial positions. For example, one head can pay more attention to the similarity of the ash slag surface texture, and another head can pay more attention to the texture deformation correlation caused by speed changes, etc.

[0103] Each 3D Transformer block uses a feed-forward neural network to perform a deep non-linear transformation on the associated features to obtain deep-level features. The feed-forward neural network usually consists of multiple fully connected layers, and activation functions (such as ReLU) can be added in the middle to perform non-linear mapping on the input features, enabling the model to learn more complex feature representations. For example, through the combination of multiple fully connected layers, the associated features are subjected to multiple non-linear transformations, gradually abstracting the features from low-level local information to more discriminative deep-level features, which helps in the accurate description and classification of ash slag textures.

[0104] For the feature fusion layer:

[0105] Perform global pooling on the deep-level features to compress the high-level feature map into a global feature vector. Average pooling or max pooling can be used to calculate the global average or maximum value of the features on each feature channel to reduce the dimension of the feature map and obtain a compact global feature vector. For example, for a feature map of N×M×P, average pooling will calculate the average value on each channel and convert it into a 1×1×1 vector.

[0106] Concatenate and fuse the global feature vector obtained from global pooling with the local region features to generate a comprehensive texture feature vector as the ash slag texture feature vector. The concatenation operation combines features at different levels. The finally obtained ash slag texture feature vector contains both local detailed information (from local region features) and global overall information (from the global feature vector), providing a comprehensive feature representation for subsequent ash slag classification and volume calculation.

[0107] In an optional embodiment, according to the ash slag texture feature vector, ash slag classification is performed on each point in the fusion data to obtain the ash slag classification result of each point in the fusion data, including the following steps:

[0108] Input the ash and slag texture feature vectors into a pre-trained classification neural network to output the ash and slag class probabilities corresponding to each point in the fusion data;

[0109] Based on the ash and slag class probabilities corresponding to each point in the fusion data and a preset class probability threshold, divide each point in the fusion data into ash or slag as the ash and slag classification result.

[0110] Specifically, various types of neural networks can be selected as classifiers. For example, variants of multi-layer perceptrons (MLPs), convolutional neural networks (CNNs), or long short-term memory networks (LSTMs), etc., can be selected according to the characteristics of the fusion data and the requirements of ash and slag classification. For example, if an MLP is selected as the classifier, the MLP consists of multiple fully connected layers, and through the combination of linear transformation and non-linear activation functions (such as ReLU), the input feature vectors are processed to obtain the ash and slag class probabilities corresponding to each point in the fusion data.

[0111] The preset class probability threshold can be set according to the distribution of the training data and the specific requirements of the classification task. If the sample numbers of ash and slag in the training data are unbalanced, the threshold can be adjusted to balance the misclassification rate. For example, if the sample number of ash is much larger than that of slag, in order to avoid misclassifying too many points as ash, the class probability threshold of ash can be appropriately increased to improve the classification accuracy of slag.

[0112] In an alternative embodiment, based on the preprocessed multi-modal data and the ash and slag classification result, calculate the total volume of ash particles and the total volume of slag blocks in the falling ash and slag, including the following steps:

[0113] Calculate the volume of each ash particle and each slag block according to the preprocessed point cloud data and the preprocessed velocity data;

[0114] Sum up the volumes of all ash particles to obtain the total volume of all ash particles in the falling ash and slag; sum up the volumes of all slag blocks to obtain the total volume of all slag blocks in the falling ash and slag.

[0115] Specifically, based on the volume calculation principle of point cloud data:

[0116] The preprocessed point cloud data represents the position information of ash and slag in three-dimensional space. For each ash particle or slag block, it can be regarded as a geometric shape composed of a series of discrete points. When calculating the volume, these discrete points can be connected into an approximate geometric shape, and three-dimensional surface reconstruction can be performed according to the point cloud data.

[0117] The surface reconstruction method can use a triangulation algorithm (such as Delaunay triangulation) to convert point cloud data into a triangular mesh representation. For each ash particle or slag block, triangular patches are formed by connecting adjacent points, and these triangular patches together constitute the surface of the ash particle or slag block. The area of each triangular patch is calculated according to Heron's formula, and then the area of the entire surface is calculated.

[0118] On the basis of obtaining the surface, a surface-based volume calculation method is used, such as volume calculation based on tetrahedron triangulation. For the surface reconstructed from point cloud data, its internal space is divided into multiple tetrahedrons, and according to the volume formula of the tetrahedron (S is the base area and h is the height), the volume of each tetrahedron is calculated, and then the volumes of all tetrahedrons are added together to obtain the volume of the ash particle or slag block. For complex shapes, an integration method can be used for volume calculation, and the volume of the object is expressed as an integral form V = ∫∫∫ V dV. Under the discretized point cloud data, this integral can be approximated by summing the volumes of each small region.

[0119] In an alternative embodiment, according to the preprocessed point cloud data and preprocessed velocity data, the volumes of each ash particle and each slag block are calculated, including the following steps:

[0120] Based on the shapes of each ash particle and each slag block outlined by the points in the preprocessed point cloud data, determine the areas of each ash particle and each slag block at a specific cross-section;

[0121] Based on the areas of each ash particle and each slag block at the specific cross-section, combined with the preprocessed velocity data corresponding to each point in the preprocessed point cloud data, calculate the volumes of each ash particle and each slag block passing through the specific cross-section per unit time by means of integration;

[0122] Among them, the volumes of each ash particle and each slag block passing through the specific cross-section per unit time are calculated by means of integration, and the calculation formula is:

[0123]

[0124] Among them, V1 represents the volume of each ash particle passing through the specific cross-section per unit time, S1(t) represents the area of each ash particle at the specific cross-section at time t, and v1(t) represents the velocity corresponding to each ash particle at time t; V2 represents the volume of each slag block passing through the specific cross-section per unit time, S2(t) represents the area of each slag block at the specific cross-section at time t, and v2(t) represents the velocity corresponding to each slag block at time t; the unit time is T = t2 - t1.

[0125] Specifically, a plane perpendicular to the average direction of ash and slag falling can be selected as the specific cross-section. This is because this direction can maximize the display of the geometric features of ash particles and slag blocks, reducing the area calculation deviation caused by inclined or irregular angles. When determining this cross-section, the average direction vector of ash and slag falling can be obtained by analyzing the preprocessed velocity data. Based on this vector, the plane equation perpendicular to it is constructed, and then the position and direction of the specific cross-section are defined.

[0126] When projecting the point cloud data belonging to each ash particle and slag block onto the specific cross-section, the convex hull algorithm can be used to determine the boundary of the projected point set. The convex hull algorithm can find the smallest convex polygon that contains all the projected points, and the boundary of this polygon represents the contour of the ash particle or slag block on the specific cross-section. For the polygon area composed of the convex hull boundary points, the shoelace formula can be used to calculate the area. This formula can quickly and accurately calculate the area of the polygon through specific operations on the vertex coordinates of the polygon. If the shape formed by the projected point set is relatively complex and the simple convex hull and shoelace formula cannot meet the accuracy requirements, a more advanced polygon approximation algorithm can be used at this time. For example, the complex shape is approximated as multiple simple polygons, the area of each simple polygon is calculated separately, and then the sum is obtained as the total area. In this way, the areas of each ash particle and each slag block on the specific cross-section can be obtained, that is, S1(t) (the area of the ash particle on the specific cross-section at time t) and S2(t) (the area of the slag block on the specific cross-section at time t).

[0127] For ash particles, the exact calculation formula for the volume V1 passing through the specific cross-section per unit time is When the ash particle velocity and cross-sectional area do not change much per unit time, for the sake of simplified calculation, the approximate formula V1≈S1(t)·v1(t)dt can be used. The calculation method for slag blocks is similar.

[0128] The above real-time monitoring method for the slag discharge amount of a dry slag machine uses multi-modal data acquisition means to obtain multi-angle image data, laser point cloud data and velocity data of the falling ash and slag at the slag discharge port of the dry slag machine by using an image acquisition device, a laser scanning device and a laser velocity sensor, and obtains environmental light data by means of a light sensor; preprocesses the multi-modal data, fuses it and inputs it into a deep learning model based on an improved 3D Transformer to extract the ash and slag texture feature vectors; classifies the points in the fused data with a classification neural network; and obtains the total volume of ash particles and slag blocks through specific calculation methods according to the preprocessed multi-modal data and the ash and slag classification results. This solution can accurately distinguish ash and slag and calculate their respective volumes, providing key data support for the operation and management of the dry slag machine, helping to optimize the production process, improve efficiency and reduce resource waste.

[0129] It should be understood that although each step in the flowcharts involved in the embodiments described above is displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0130] Based on the same inventive concept, an embodiment of the present application also provides a system for implementing the real-time monitoring method of the slag discharge amount of the dry slag machine involved above. The implementation solution provided by this system to solve the problem is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the real-time monitoring system of the dry slag machine slag discharge amount provided below can refer to the limitations on a real-time monitoring method of the dry slag machine slag discharge amount in the above text, and will not be repeated here.

[0131] In an exemplary embodiment, as Figure 2 shown, a real-time monitoring system 20 of the dry slag machine slag discharge amount is provided, including:

[0132] A multi-modal data acquisition module 21, configured to acquire multi-modal data of the falling ash slag at the slag discharge port of the dry slag machine; the multi-modal data includes image data, point cloud data, and sensor data.

[0133] A data preprocessing and fusion module 22, configured to preprocess the multi-modal data to obtain preprocessed multi-modal data; fuse the preprocessed multi-modal data to obtain fused data.

[0134] An ash slag texture feature extraction module 23, configured to extract ash slag texture features from the fused data by using a trained deep learning model to obtain an ash slag texture feature vector; the ash slag texture feature vector characterizes the texture structure of the falling ash slag in three-dimensional space.

[0135] An ash slag classification module 24, configured to classify each point in the fused data into ash or slag according to the ash slag texture feature vector to obtain the ash slag classification result of each point in the fused data, and the ash slag classification result is ash or slag.

[0136] An ash slag falling amount calculation module 25, configured to calculate the total volume of ash particles and the total volume of slag blocks in the falling ash slag according to the preprocessed multi-modal data and the ash slag classification result.

[0137] Optionally, the multi-modal data acquisition module 21 includes:

[0138] An image data acquisition unit 251, configured to acquire multi-angle image data of the falling ash residue by using an image acquisition device.

[0139] A point cloud data acquisition unit 252, configured to scan the surface contour of the falling ash residue at a specific frequency by using a laser scanning device to generate laser point cloud data.

[0140] A speed data acquisition unit 253, configured to measure the instantaneous velocity vector of the falling ash residue according to the principle of the Doppler effect of the combined laser by using a laser velocity sensor to obtain speed data.

[0141] A multi-modal data confirmation unit 254, configured to use the multi-angle image data, the laser point cloud data, and the speed data as multi-modal data.

[0142] Optionally, the dry slag machine slag discharge amount real-time monitoring system 20 further includes a calibration data acquisition module 26, configured to calibrate the image acquisition device by using a calibration plate to obtain calibration data.

[0143] The data preprocessing and fusion module 22 includes:

[0144] A first preprocessing unit 221, configured to perform first preprocessing on the multi-angle image data to obtain preprocessed image data. Among them, the first preprocessing includes:

[0145] Performing optical distortion correction processing on the multi-angle image data according to the calibration data;

[0146] Performing denoising processing on the multi-angle image data by using a bilateral filtering algorithm.

[0147] A second preprocessing unit 222, configured to perform second preprocessing on the laser point cloud data to obtain preprocessed point cloud data. Among them, the second preprocessing includes:

[0148] Unifying the coordinate system of the laser point cloud data with the coordinate system of the multi-angle image data according to the installation attitude and installation position of the laser scanner and combining the calibration data;

[0149] Using a statistical filtering algorithm to remove the outlier points of the laser point cloud data.

[0150] A third preprocessing unit 223, configured to associate and store the speed data and the preprocessed point cloud data according to a specific timestamp to obtain preprocessed speed data.

[0151] A data fusion unit 224, configured to perform the following steps:

[0152] Using the calibration data, map the two-dimensional image pixel coordinates of the preprocessed image data to the three-dimensional space, and combine them with the preprocessed point cloud data at the corresponding position and the same moment to form three-dimensional point cloud data with texture information;

[0153] Use the preprocessed speed data as the motion attribute of each point in the three-dimensional point cloud data, and attach it to the three-dimensional point cloud data to form fusion data.

[0154] Optionally, the dry slag machine slag discharge amount real-time monitoring system 20 further includes an ambient light data acquisition module 27, which is used to obtain the ambient light data at the slag discharge port of the dry slag machine by using a light sensor.

[0155] The first preprocessing in the first preprocessing unit 211 further includes: using an enhancement algorithm based on the Retinex theory, combining the ambient light data, and adaptively adjusting the brightness and contrast of the multi-angle image data.

[0156] Optionally, the deep learning model in the ash slag texture feature extraction module 23 is an ash slag texture feature extraction model based on an improved 3D Transformer, including:

[0157] An input layer, which is used to receive the fusion data; among them, the input layer performs spatial partitioning processing on the input fusion data through a spatial indexing algorithm, and the spatial partitioning processing includes: dividing the points in the fusion data that are adjacent in space and have similar spatial positions and speed states into multiple local regions.

[0158] A feature extraction layer based on 3D convolution, which is used to extract features from the fusion data; among them, the 3D convolution kernel of the feature extraction layer slides in the three-dimensional space, performs convolution operations on the fusion data of the local regions, extracts the texture features, geometric shape inherent features and dynamic features brought by the motion attributes of the ash slag as local region features; fuses and encodes the local region features to generate a low-level feature map.

[0159] A 3D Transformer encoding layer, which is used to perform deep feature extraction on the low-level feature map; among them, the encoding layer is composed of multiple 3D Transformer blocks, and each 3D Transformer block uses a multi-head self-attention mechanism in the three-dimensional space dimension and the speed dimension to capture the correlation of ash slag texture changes at different speeds and different spatial positions, and outputs correlation features; each 3D Transformer block uses a feed-forward neural network to perform deep non-linear transformation on the correlation features to obtain deep features.

[0160] A feature fusion layer, which is used to splice and fuse the global feature vector obtained by global pooling processing with the local region features after global pooling processing of the deep features to generate a comprehensive texture feature vector as the ash slag texture feature vector.

[0161] Optionally, the ash and slag classification module 24 includes:

[0162] A probability prediction unit 241, configured to input the ash and slag texture feature vectors into a pre-trained classification neural network, and output the ash and slag category probabilities corresponding to each point in the fusion data.

[0163] A category judgment unit 242, configured to divide each point in the fusion data into ash or slag based on the ash and slag category probabilities corresponding to each point in the fusion data and a preset category probability threshold, as the ash and slag classification result.

[0164] Optionally, the ash and slag quality calculation module 25 includes:

[0165] A preliminary calculation unit 251, configured to calculate the volumes of each ash particle and each slag block according to the preprocessed point cloud data and the preprocessed velocity data.

[0166] A volume summary unit 252, configured to summarize the volumes of each ash particle to obtain the volume of all ash particles in the falling ash and slag; summarize the volumes of each slag block to obtain the volume of all slag blocks in the falling ash and slag.

[0167] Optionally, the preliminary calculation unit 251 includes:

[0168] A cross-sectional area calculation subunit 2511, configured to determine the areas of each ash particle and each slag block at a specific cross-section according to the shapes of each ash particle and each slag block outlined by the points in the preprocessed point cloud data.

[0169] A volume calculation subunit 2512, configured to calculate the volumes of each ash particle and each slag block passing through a specific cross-section per unit time by integrating according to the areas of each ash particle and each slag block at the specific cross-section and the preprocessed velocity data corresponding to each point in the preprocessed point cloud data.

[0170] Wherein, the volumes of each ash particle and each slag block passing through a specific cross-section per unit time are calculated by integration, and the calculation formula is:

[0171]

[0172] Wherein, V1 represents the volume of each ash particle passing through a specific cross-section per unit time, S1(t) represents the area of each ash particle at the specific cross-section at time t, v1(t) represents the velocity of each ash particle corresponding to time t; V2 represents the volume of each slag block passing through a specific cross-section per unit time, S2(t) represents the area of each slag block at the specific cross-section at time t, v2(t) represents the velocity of each slag block corresponding to time t; the unit time is T = t2 - t1.

[0173] Embodiments of the present application also provide a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the foregoing method embodiments are implemented.

[0174] Embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0175] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial descriptions of the method embodiments. The device embodiments described above are only illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present disclosure. A person of ordinary skill in the art can understand and implement it without creative efforts.

[0176] The above embodiments only represent several implementation manners of the embodiments of the present application. The descriptions are relatively specific and detailed, but should not be construed as a limitation on the patent scope of the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A real-time monitoring method for the slag falling amount of a dry slag machine, characterized in that, The method includes: Obtaining multi-modal data of the falling ash and slag at the slag discharge opening of the dry slag machine; the multi-modal data includes image data, point cloud data, and sensor data; Preprocessing the multi-modal data to obtain preprocessed multi-modal data; fusing the preprocessed multi-modal data to obtain fused data; Using a trained deep learning model to extract ash and slag texture features from the fused data to obtain an ash and slag texture feature vector; the ash and slag texture feature vector characterizes the texture structure of the falling ash and slag in three-dimensional space; According to the ash and slag texture feature vector, classifying the ash and slag for each point in the fused data to obtain the ash and slag classification result for each point in the fused data, and the ash and slag classification result is ash or slag; According to the preprocessed multi-modal data and the ash and slag classification result, calculating the total volume of ash particles and the total volume of slag blocks in the falling ash and slag.

2. The method according to claim 1, wherein The obtaining of the multi-modal data of the falling ash and slag at the slag discharge opening of the dry slag machine includes: Using an image acquisition device to obtain multi-angle image data of the falling ash and slag; Using a laser scanning device to scan the surface contour of the falling ash and slag at a specific frequency to generate laser point cloud data; Using a laser velocity sensor to measure the instantaneous velocity vector of the falling ash and slag according to the principle of the Doppler effect of the combined laser to obtain velocity data; Taking the multi-angle image data, the laser point cloud data, and the velocity data as the multi-modal data.

3. The method according to claim 2, wherein The method further includes: calibrating the image acquisition device using a calibration plate to obtain calibration data; The preprocessing of the multi-modal data to obtain preprocessed multi-modal data includes: S11: Performing first preprocessing on the multi-angle image data to obtain preprocessed image data; Wherein, the first preprocessing includes: Performing optical distortion correction processing on the multi-angle image data according to the calibration data; Performing denoising processing on the multi-angle image data using a bilateral filtering algorithm; S12: Performing second preprocessing on the laser point cloud data to obtain preprocessed point cloud data; Wherein, the second preprocessing includes: Unifying the coordinate system of the laser point cloud data with the coordinate system of the multi-angle image data according to the installation attitude and installation position of the laser scanner and in combination with the calibration data; Using a statistical filtering algorithm to remove the outlier points of the laser point cloud data; S13: Associating and storing the velocity data with the preprocessed point cloud data according to a specific timestamp to obtain preprocessed velocity data; The fusing of the preprocessed multi-modal data to obtain fused data includes: Using the calibration data to map the two-dimensional image pixel coordinates of the preprocessed image data into three-dimensional space and combining them with the preprocessed point cloud data at the corresponding position and at the same moment to form three-dimensional point cloud data with texture information; Taking the preprocessed velocity data as the motion attribute of each point in the three-dimensional point cloud data and attaching it to the three-dimensional point cloud data to form the fused data.

4. The method according to claim 3, wherein The method further includes: obtaining environmental light data at the slag discharge opening of the dry slag machine using a light sensor; The first preprocessing further includes: using an enhancement algorithm based on the Retinex theory, and combining the ambient light data to adaptively adjust the brightness and contrast of the multi-angle image data.

5. The method according to claim 3 or 4, characterized in that, The deep learning model is a slag texture feature extraction model based on an improved 3D Transformer, and includes: An input layer for receiving the fused data; wherein, the input layer performs spatial partitioning processing on the input fused data through a spatial indexing algorithm, and the spatial partitioning processing includes: dividing points in the fused data that are adjacent in space and have similar spatial positions and velocity states into multiple local regions; A feature extraction layer based on 3D convolution for extracting features from the fused data; wherein, the 3D convolution kernel of the feature extraction layer slides in three-dimensional space to perform a convolution operation on the fused data in the local region, and extracts the texture features, geometric shape inherent features, and dynamic features brought by the motion attributes of the slag as local region features; fusing and encoding the local region features to generate a low-level feature map; A 3D Transformer encoding layer for performing deep feature extraction on the low-level feature map; wherein, the encoding layer is composed of multiple 3D Transformer blocks, and each 3D Transformer block uses a multi-head self-attention mechanism in the three-dimensional space dimension and the velocity dimension to capture the correlation of slag texture changes at different velocities and different spatial positions, and outputs associated features; each 3D Transformer block uses a feed-forward neural network to perform a deep non-linear transformation on the associated features to obtain deep-level features; A feature fusion layer for, after performing global pooling processing on the deep-level features, splicing and fusing the global feature vector obtained by the global pooling processing with the local region features to generate a comprehensive texture feature vector as the slag texture feature vector; Among them, using the trained deep learning model to extract the slag texture features from the fused data to obtain a slag texture feature vector includes: inputting the fused data into the trained slag texture feature extraction model based on the improved 3D Transformer, and outputting the slag texture feature vector.

6. The method according to claim 1, characterized in that, The classifying the slag for each point in the fused data according to the slag texture feature vector to obtain the slag classification result for each point in the fused data includes: Inputting the slag texture feature vector into a pre-trained classification neural network, and outputting the slag category probability corresponding to each point in the fused data; Based on the slag category probability corresponding to each point in the fused data and a preset category probability threshold, dividing each point in the fused data into ash or slag as the slag classification result.

7. The method according to claim 3, characterized in that The calculating the total volume of ash particles and the total volume of slag blocks in the falling slag according to the preprocessed multi-modal data and the slag classification result includes: Calculating the volume of each ash particle and each slag block according to the preprocessed point cloud data and the preprocessed velocity data; Sum up the volumes of each ash particle to obtain the volume of all ash particles in the falling ash slag; sum up the volumes of each slag block to obtain the volume of all slag blocks in the falling ash slag.

8. The method according to claim 7, wherein The calculating the volumes of each ash particle and each slag block according to the preprocessed point cloud data and the preprocessed velocity data includes: Determine the areas of each ash particle and each slag block at a specific cross-section according to the shapes of each ash particle and each slag block outlined by the points in the preprocessed point cloud data; According to the areas of each ash particle and each slag block at the specific cross-section, and in combination with the preprocessed velocity data corresponding to each point in the preprocessed point cloud data, calculate the volumes of each ash particle and each slag block passing through the specific cross-section per unit time by means of integration; Wherein, the calculating the volumes of each of the ash particles and each of the slag blocks passing through the specific cross-section per unit time by means of integration has the following calculation formula: Wherein, V1 represents the volume of each ash particle passing through the specific cross-section per unit time, S1(t) represents the area of each ash particle at the specific cross-section at time t, and v1(t) represents the velocity corresponding to each ash particle at time t; V2 represents the volume of each slag block passing through the specific cross-section per unit time, S2(t) represents the area of each slag block at the specific cross-section at time t, and v2(t) represents the velocity corresponding to each slag block at time t; the unit time is T = t2 - t1.

9. A real-time monitoring system for the slag dropping amount of a dry slag machine, characterized in that, The system includes: A multi-modal data acquisition module for acquiring multi-modal data of the falling ash slag at the slag discharge port of the dry slag machine; the multi-modal data includes image data, point cloud data, and sensor data; A data preprocessing and fusion module for preprocessing the multi-modal data to obtain preprocessed multi-modal data; fusing the preprocessed multi-modal data to obtain fused data; An ash slag texture feature extraction module for extracting ash slag texture features from the fused data by using a trained deep learning model to obtain an ash slag texture feature vector; the ash slag texture feature vector characterizes the texture structure of the falling ash slag in three-dimensional space; An ash slag classification module for classifying each point in the fused data into ash or slag according to the ash slag texture feature vector to obtain an ash slag classification result for each point in the fused data; the ash slag classification result is ash or slag; An ash slag falling amount calculation module for calculating the total volume of ash particles and the total volume of slag blocks in the falling ash slag according to the preprocessed multi-modal data and the ash slag classification result.

10. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

Images