A non-destructive detection method, system and storage medium for coal ash content based on multi-modal data fusion

Abstract

The present invention relates to a method, system and storage medium for non-destructive detection of coal ash by multi-modal data fusion, belonging to the field of coal ash detection, including: identifying the coal contour and image quality of multiple coal images to obtain multiple coal contours and multiple image quality coefficients; performing point cloud fitting according to multiple coal contours in multiple point cloud data to obtain multiple coal point clouds and multiple fitting accuracy coefficients; constructing a three-dimensional coal model based on the coal point cloud, processing to obtain the coal volume, and calculating the coal density according to the coal volume and coal mass; calculating and configuring the ash analysis accuracy according to the fitting accuracy coefficient and the image quality coefficient, and performing integrated ash analysis on the coal density to obtain ash information. Through the present invention, the technical problems that the traditional method cannot quickly and effectively identify the ash content and there are deficiencies in detection timeliness and accuracy can be solved; more efficient and accurate coal ash detection can be realized, and the applicability, efficiency and accuracy of ash content detection can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of coal ash detection, and particularly to a non-destructive detection method, system and storage medium for coal ash by multi-modal data fusion. Background Art

[0002] Coal is one of the main energy sources globally, and its quality directly affects energy utilization efficiency and environmental protection levels.

[0003] During the production, transportation and combustion of coal, the ash content of coal is one of the important indicators to measure the quality of coal. Excessive ash content will lead to a decrease in coal combustion efficiency and an increase in pollutant emissions, which has a negative impact on the energy utilization of industries such as electricity and steel. Therefore, accurately and quickly measuring the ash content of coal is crucial for optimizing coal utilization, improving combustion efficiency and reducing environmental pollution.

[0004] However, traditional coal ash detection methods usually rely on laboratory methods such as chemical analysis, X-ray fluorescence, infrared spectroscopy, etc. These methods cannot quickly and effectively identify the ash content, especially in real-time monitoring and large-scale production processes, and it is difficult to meet the requirements of rapid coal ash detection. Summary of the Invention

[0005] Aiming at the technical problems that traditional coal ash detection methods cannot quickly and effectively identify the ash content, lack timeliness and accuracy in detection, and are difficult to meet the requirements of real-time ash monitoring, the present invention provides a non-destructive detection method, system and storage medium for coal ash by multi-modal data fusion to solve.

[0006] The technical solutions of the present invention to solve the above technical problems are as follows:

[0007] In a first aspect, the present invention provides a non-destructive detection method for coal ash by multi-modal data fusion, including: performing multi-angle image acquisition, multi-angle point cloud data acquisition and quality acquisition on coal to obtain multiple coal images, multiple point cloud data and coal quality; performing coal contour recognition and image quality recognition on the multiple coal images to obtain multiple coal contours and multiple image quality coefficients, where the multi-angle includes angles on both sides of the coal; respectively performing coal point cloud iterative fitting in the multiple point cloud data according to the multiple coal contours to obtain multiple coal point clouds and multiple fitting accuracy coefficients, where the point cloud iterative fitting is adjusted according to the multiple image quality coefficients for the multiple coal contours; constructing a three-dimensional model of coal based on the multiple coal point clouds, processing to obtain coal volume information, and calculating coal density according to the coal volume information and coal quality; calculating and configuring the ash analysis accuracy according to the multiple fitting accuracy coefficients and multiple image quality coefficients, and performing integrated ash analysis on the coal density to obtain the ash information of the coal.

[0008] In a second aspect, the present invention further provides an electronic device, including:

[0009] At least one processor; a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the method according to any one of the above first aspects.

[0010] In a third aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the steps of the method according to any one of the above first aspects.

[0011] The beneficial effects of the present invention are as follows: By performing multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition on coal, multiple coal images, multiple point cloud data, and coal quality are obtained, where the multi-angle includes the angles on both sides of the coal; then, coal contour recognition and image quality recognition are performed on the multiple coal images to obtain multiple coal contours and multiple image quality coefficients; then, coal point cloud iterative fitting is respectively performed within the multiple point cloud data according to the multiple coal contours to obtain multiple coal point clouds and multiple fitting accuracy coefficients, where the point cloud iterative fitting of the multiple coal contours is adjusted respectively according to the multiple image quality coefficients; further, a coal three-dimensional model is constructed based on the multiple coal point clouds, coal volume information is processed and obtained, and the coal density is calculated according to the coal volume information and the coal quality; finally, the ash analysis accuracy is calculated and configured according to the multiple fitting accuracy coefficients and the multiple image quality coefficients, and integrated ash analysis is performed on the coal density to obtain the ash information of the coal; that is to say, through multi-modal data fusion and integrated learning analysis, more efficient and accurate coal ash detection can be achieved, significantly improving the applicability, efficiency, and accuracy of ash content detection, so as to meet the real-time monitoring requirements of ash in the coal production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic flowchart of a method for non-destructive detection of coal ash by multi-modal data fusion provided by the present invention;

[0013] Figure 2 It is a schematic structural diagram of the electronic device provided by the present invention;

[0014] Figure 3 It is a schematic structural diagram of a computer-readable storage medium provided by the present invention.

[0015] In the drawings, the components represented by the reference numerals are described as follows:

[0016] Electronic device 500, memory 510, processor 520, first computer program 511, computer-readable storage medium 600, second computer program 611. Detailed implementation manners

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0019] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or having more advantages than other embodiments. The following description is provided to enable any person skilled in the art to make and use the present invention. In the following description, details are set forth for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present invention can be implemented without these specific details. In other instances, well-known structures and processes are not described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0020] Embodiment 1, as Figure 1 shown, the embodiment of the present invention provides a method for non-destructive detection of coal ash content by multi-modal data fusion, which specifically includes the following steps:

[0021] S100: Perform multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition on coal to obtain a plurality of coal images, a plurality of point cloud data, and coal quality. Perform coal contour recognition and image quality recognition on the plurality of coal images to obtain a plurality of coal contours and a plurality of image quality coefficients, where the multi-angle includes the angles on both sides of the coal.

[0022] Specifically, first, multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition are carried out on the coal on the conveyor belt. Among them, multi-angle includes the angles on both sides of the coal (the left and right sides of the conveyor belt), that is, the coal is transported on the conveyor belt, and image acquisition devices are installed on both sides of the conveyor belt. Through multi-angle shooting, it is ensured that most areas of the coal surface can be covered. The images obtained by each camera will include information from different angles, which helps to improve the recognition accuracy of the coal surface contour; laser scanners are installed on both sides of the coal conveyor belt. In order to avoid the coal being blocked or omitted due to angle problems, the point cloud acquisition system scans the coal surface from two different angles simultaneously to generate three-dimensional point cloud data covering the entire coal surface. These data can accurately reflect the morphological characteristics and spatial distribution of the coal; quality acquisition refers to the real-time acquisition of the weight of the coal through high-precision sensors or weighing devices to obtain coal weight data; through data acquisition, multiple coal images, multiple point cloud data, and coal quality are obtained.

[0023] Through multi-angle image, point cloud data acquisition, and coal quality data acquisition, multi-dimensional information of the coal can be comprehensively and accurately obtained, and these data provide strong support for subsequent ash content detection and analysis.

[0024] Furthermore, step S100 of the present invention further includes:

[0025] S110: Pre-train the coal image recognition channel, where the coal image recognition channel includes a coal contour recognition path and a noise recognition branch.

[0026] Furthermore, step S110 of the present invention further includes:

[0027] S111: According to the coal detection data within the historical time, collect a set of sample coal images, label the ratio of the coal contour and noise pixel points in each sample coal image to obtain a set of sample coal contours and a set of sample noise ratios; S112: Use the set of sample coal images as input data, and use the set of sample coal contours and the set of sample noise ratios as supervision data respectively. Based on the convolutional neural network, supervise and train the coal contour recognition path and the noise recognition branch respectively; S113: Combine the trained coal contour recognition path and the noise recognition branch to obtain the coal image recognition channel.

[0028] Specifically, obtain the coal detection data within a historical time period (such as within the most recent month), and collect sample coal images under different times and different production conditions according to the coal detection data to obtain a set of sample coal images. Then, label the ratio of the coal contour and noise pixel points in each sample coal image. For example, use an image processing algorithm (such as edge detection) to identify the actual contour of the coal in each sample coal image, and accurately extract the contour information on the coal surface; noise pixel points are usually abnormal points in the image, manifested as points that do not conform to the coal shape or background characteristics. By setting a threshold or using an image classification model based on machine learning, the ratio of noise can be effectively identified; obtain a set of sample coal contours and a set of sample noise ratios, where the sample coal images, sample coal contours, and sample noise ratios correspond one by one.

[0029] Convolutional neural network (CNN) is a type of deep learning model widely used to process multi-dimensional data such as images and videos. It automatically extracts features from the input data through convolutional operations and can learn more complex and efficient feature representations. Then, based on the convolutional neural network, construct a coal contour recognition path and a noise recognition branch. Among them, both the coal contour recognition path and the noise recognition branch include an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layer is responsible for extracting local features in the image through convolutional operations; the pooling layer is used to reduce the spatial resolution of the feature map while retaining the most important feature information; the fully connected layer is used to fuse and output the final result based on the features extracted by the convolutional layer and the pooling layer, that is, map the high-dimensional features to the output space. The input data of the input layer of the coal contour recognition path is the coal image, and the output data of the output layer is the coal contour; the input data of the input layer of the noise recognition branch is the coal image, and the output data of the output layer is the noise ratio.

[0030] Then, using the sample coal images as the input and the sample coal contours as the supervision, the coal contour recognition path is supervised and trained using the set of sample coal images and the set of sample coal contours. First, through the input coal images, forward propagation is performed to calculate the output of each convolutional layer, and finally the prediction result of the network (i.e., the predicted coal contour) is generated. Then, according to the difference between the predicted coal contour and the true label, the loss value is calculated through the loss function. Further, by calculating the gradients of the loss function with respect to each weight and bias, the gradients are backpropagated and the model parameters are updated, and the weights and biases of the network are updated according to the optimization algorithm (such as Adam, SGD, etc.). The sample data is used for iterative optimization until the loss function meets the expected convergence condition, and the trained coal contour recognition path is obtained. On the other hand, using the sample coal images as the input and the sample noise ratio as the supervision, the set of sample coal images and the set of sample noise ratios are used as training data to supervise and train the noise recognition branch until convergence, and the trained noise recognition branch is obtained. The training method of the noise recognition branch is the same as that of the coal contour recognition path and will not be elaborated here. Through the coal contour recognition path and the noise recognition branch constructed based on the convolutional neural network, the accuracy and efficiency of coal ash detection can be greatly improved, thus providing support for realizing real-time, automated, and accurate monitoring of coal ash content.

[0031] Finally, the trained coal contour recognition path and the noise recognition branch are combined to obtain the coal image recognition channel, that is, the input of the coal image recognition channel is the coal image, and the output is the coal contour and the noise ratio.

[0032] S120: Respectively input the multiple coal images into the coal image recognition channel to recognize and obtain multiple coal contours and multiple noise ratios; S130: Subtract the multiple noise ratios from 1 to obtain multiple image quality coefficients.

[0033] Specifically, then the multiple coal images are respectively input into the coal image recognition channel for recognition, and multiple coal contours and multiple noise ratios are output; then 1 is respectively subtracted from the multiple noise ratios, and the difference between 1 and the noise ratio is set as the image quality coefficient to obtain multiple image quality coefficients. Among them, a smaller image quality coefficient indicates a poorer image quality; a larger image quality coefficient indicates a better image quality. By calculating the noise ratio and obtaining the image quality coefficient, the quality of each coal image can be quantified, providing support for subsequent coal point cloud data fitting and coal ash prediction.

[0034] S200: Iteratively fit the coal point clouds within the multiple point cloud data according to the multiple coal contours respectively, to obtain multiple coal point clouds and multiple fitting accuracy coefficients, wherein the iterative point cloud fitting of the multiple coal contours is adjusted according to the multiple image quality coefficients respectively.

[0035] Further, step S200 of the present invention further includes:

[0036] S210: Iteratively fit the coal point clouds within the first point cloud data among the multiple point cloud data according to the first coal contour within the multiple coal contours, to obtain a first initial fitted coal point cloud and a first initial fitting accuracy coefficient.

[0037] Further, step S210 of the present invention further includes:

[0038] S211: Randomly select a region within the first point cloud data according to the first coal contour to obtain a first selected coal point cloud; S212: Calculate the ratio of the point cloud area ratio within the first selected coal point cloud to the area of the first coal contour as the first selection accuracy coefficient; S213: Continue to randomly select a region within the first point cloud data using the first coal contour for iterative selection and fitting of the coal point clouds until the convergence selection fitting times are reached, and output the maximum selection accuracy coefficient and the corresponding selected coal point cloud as the first initial fitting accuracy coefficient and the first initial fitted coal point cloud.

[0039] Specifically, first, randomly select any one of the multiple coal contours as the first coal contour, and randomly select a region within the first point cloud data according to the first coal contour to obtain a first selected coal point cloud, where the first selected coal point cloud is the point cloud falling within the first coal contour; then, calculate the point cloud area ratio within the first selected coal point cloud, such as multiplying the area occupied by a single point within the first selected coal point cloud by the total number of points to obtain the point cloud area, and the point cloud area can also be estimated by calculating the spatial distribution of each sampling point in the point cloud. Then, take the ratio of the point cloud area ratio within the first selected coal point cloud to the area of the first coal contour as the first selection accuracy coefficient, where the selection accuracy coefficient reflects the matching degree between the point cloud in the current selected region and the coal contour. The closer the accuracy coefficient is to 1, the better the matching degree between the selection result and the contour, and the better the fitting effect.

[0040] Then continue to use the first coal contour, randomly select a box within the first point cloud data, and perform iterative box selection fitting for coal point clouds, that is, perform multiple iterative random box selections until the convergence box selection fitting times are reached. The convergence box selection fitting times can be set according to the point cloud fitting accuracy requirements, such as 100 times; obtain multiple box-selected coal point clouds and multiple box selection accuracy coefficients; finally, output the maximum box selection accuracy coefficient as the first initial fitting accuracy coefficient, and output the maximum box selection accuracy coefficient and the corresponding box-selected coal point cloud as the first initial fitting coal point cloud. By repeatedly iterating box selection and fitting, the matching degree between the coal point cloud and the coal contour can be maximized, and the accuracy and precision of the point cloud fitting result can be improved.

[0041] S220: Subtract the first image quality coefficient in the multiple image quality coefficients from 1 to obtain the first contour adjustment coefficient. Use the first contour adjustment coefficient as the first contour adjustment ratio to randomly adjust the contour within the first contour adjustment ratio of the first coal contour to obtain the first adjusted coal contour; S230: Use the first adjusted coal contour to perform iterative fitting of coal point clouds within the first point cloud data to obtain the first adjusted fitting coal point cloud and the first adjusted fitting accuracy coefficient; S240: Continue to perform the adjustment of the first coal contour and the iterative fitting of coal point clouds until convergence, output the maximum fitting accuracy coefficient and the fitting coal point cloud, and obtain the first coal point cloud and the first fitting accuracy coefficient; S250: Continue to use the second coal contour, perform contour adjustment according to the second image quality coefficient, and perform iterative fitting of coal point clouds within the second point cloud data until convergence to obtain the second coal point cloud and the second fitting accuracy coefficient.

[0042] Specifically, first, randomly select any one of the multiple image quality coefficients and set it as the first image quality coefficient. There is a corresponding relationship between the first image quality coefficient and the first coal contour; then, subtract the first image quality coefficient from 1, and set the difference as the first contour adjustment coefficient. The contour adjustment coefficient is used to determine the adjustment amplitude of the coal contour. The lower the image quality coefficient, the larger the contour adjustment coefficient, which means that the contour needs more adjustments to improve the fitting accuracy; for example, assume that the first image quality coefficient is 0.95, then the first contour adjustment coefficient is 0.05, that is, the first coal contour will be adjusted according to this ratio (0.05). Each time it is adjusted, the length and shape of the contour will change slightly.

[0043] Next, use the first contour adjustment coefficient as the first contour adjustment ratio to randomly adjust the contour within the first coal contour at the first contour adjustment ratio. For example, assume the first image quality coefficient is 0.95, then the first contour adjustment coefficient is 0.05, that is, the first contour adjustment ratio is 0.05; the first coal contour will be adjusted according to this ratio (0.05). Each time it is adjusted, the length and shape of the contour will change slightly. The adjustment ratio is 5% of the length or shape of the coal contour. For example, select a part with 5% of the length within the first coal contour to randomly adjust the shape or length. For example, the length of the coal contour of the part with 5% of the length is 10 cm, and after random adjustment, it becomes 12 cm; obtain the first adjusted coal contour.

[0044] Then, use the first adjusted coal contour to perform iterative fitting of coal point clouds within the first point cloud data until a predetermined number of fitting times is reached. Output the maximum frame selection accuracy coefficient and set it as the first adjusted fitting accuracy coefficient, and set the fitting coal point cloud corresponding to the first adjusted fitting accuracy coefficient as the first adjusted fitting coal point cloud. Output the first adjusted fitting coal point cloud and the first adjusted fitting accuracy coefficient. Next, continue to adjust the first coal contour according to the first contour adjustment ratio, and continue to perform iterative fitting of coal point clouds according to the adjusted second adjusted coal contour to obtain the second adjusted fitting coal point cloud and the second adjusted fitting accuracy coefficient; further use the same method to continue to adjust the first coal contour and perform iterative fitting of coal point clouds until a predetermined convergence condition is met (that is, the predetermined number of contour adjustments and point cloud fitting times is reached), obtain multiple fitting coal point clouds and multiple fitting accuracy coefficients, and output the maximum fitting accuracy coefficient and set it as the first fitting accuracy coefficient, and set the fitting coal point cloud corresponding to the maximum fitting accuracy coefficient as the first coal point cloud.

[0045] Next, randomly select a second coal contour within the multiple coal contours, where the second coal contour is different from the first coal contour; then use the second image quality coefficient to adjust the contour of the second coal contour, and perform iterative fitting of coal point clouds within the second point cloud data until convergence to obtain the second coal point cloud and the second fitting accuracy coefficient; use the same method to obtain multiple coal point clouds and multiple fitting accuracy coefficients in turn. By dynamically adjusting the coal contour and performing multiple iterative fittings, the coal point cloud that best matches the point cloud data can be finally obtained, which can significantly improve the adaptability and accuracy of point cloud fitting, and thus can further improve the construction accuracy of the coal three-dimensional model.

[0046] S300: Construct a coal three-dimensional model based on the multiple coal point clouds, process to obtain coal volume information, and calculate the coal density based on the coal volume information and coal quality.

[0047] Furthermore, step S300 of the present invention further includes:

[0048] S310: Construct a three-dimensional coal model based on the first coal point cloud and the second coal point cloud within the multiple coal point clouds; S320: Calculate and obtain the coal volume information based on the three-dimensional coal model; S330: Calculate the ratio of the coal mass and the coal volume information to obtain the coal density.

[0049] Specifically, select the first coal point cloud and the second coal point cloud within the multiple coal point clouds, that is, the coal point clouds at two different angles (the left and right angles of the conveyor belt); then, align the first coal point cloud and the second coal point cloud collected at different angles. Generally, point cloud registration technology (such as the ICP algorithm) is used to achieve the spatial alignment of point cloud data at different angles, ensuring that the data from different perspectives of the same object can be accurately docked; then, on the basis of the alignment, fuse the two sets of point cloud data. During the fusion process, redundant point cloud information can be reduced by methods such as duplicate removal or weighted averaging, while ensuring the integrity and accuracy of the point cloud. The purpose of point cloud fusion is to generate a more complete three-dimensional model based on multiple perspectives; then, based on the fused point cloud data, use surface reconstruction technology (such as Poisson reconstruction, etc.) to convert the point cloud into a three-dimensional mesh model. These technologies generate polygon meshes by connecting adjacent points in the point cloud, and then construct the three-dimensional surface of the coal to obtain the three-dimensional coal model. Through the acquisition and processing of point cloud data from multiple angles, combined with point cloud registration, fusion, and three-dimensional mesh reconstruction technologies, the three-dimensional model of coal can be accurately constructed, which not only improves the accuracy and details of the model, but also provides strong support for subsequent coal ash analysis and quality detection.

[0050] Then calculate and obtain the coal volume information based on the three-dimensional coal model. For example, calculate the volume of the coal through the mesh surface. The three-dimensional mesh model is composed of several polyhedrons (usually triangular patches). For each triangular patch, use the area of the triangular patch and the corresponding height information (that is, the distance from the triangular patch to the coal reference plane) to estimate the volume. Through the geometric relationships of these patches, the overall volume of the coal can be calculated. Then calculate the ratio of the coal mass (coal weight) and the coal volume information to obtain the coal density. By using the three-dimensional model and mass data of the coal, combined with mathematical models and algorithms, the volume and density of the coal can be calculated efficiently, providing effective data support for coal ash detection.

[0051] S400: Calculate and configure the ash analysis accuracy based on the multiple fitting accuracy coefficients and the multiple image quality coefficients, perform integrated ash analysis on the coal density, and obtain the ash information of the coal.

[0052] Furthermore, step S400 of the present invention further includes:

[0053] S410: Calculate the mean of the multiple fitting accuracy coefficients to obtain the average fitting accuracy coefficient, and calculate the mean of the multiple image quality coefficients to obtain the average image quality coefficient; S420: Subtract the average fitting accuracy coefficient and the average image quality coefficient from 1 respectively to obtain the fitting error coefficient and the image error coefficient, and sum them up to obtain the error coefficient.

[0054] Specifically, calculate the mean of the multiple fitting accuracy coefficients to obtain the average fitting accuracy coefficient; calculate the mean of the multiple image quality coefficients to obtain the average image quality coefficient; then subtract the average fitting accuracy coefficient from 1 to obtain the fitting error coefficient; subtract the average image quality coefficient from 1 to obtain the image error coefficient. Finally, add the fitting error coefficient and the image error coefficient together to obtain the error coefficient, that is, the overall error.

[0055] S430: Train an ash analysis channel including P ash analysis branches, where P is an integer greater than or equal to 1.

[0056] Further, step S430 of the present invention further includes:

[0057] S431: According to the historical test data of coal ash, collect a sample coal density set and a sample coal ash set as ash analysis supervised training data; S432: Randomly select P pieces of data from the ash analysis supervised training data with replacement to obtain P pieces of branch supervised training data; S433: Use the P pieces of branch supervised training data to supervise and train P ash analysis branches respectively, where the loss function for each ash analysis branch supervised training is as follows: ; where LOSS is the loss, M is the number of supervised training data in each piece of branch supervised training data, is the sample coal ash in the i-th group of supervised training data, is the coal ash output by the ash analysis branch; S434: Combine the P trained ash analysis branches to obtain the ash analysis channel.

[0058] Specifically, first, according to the historical test data of coal ash, collect the sample coal density and the ash content under different sample coal densities to obtain the sample coal density set and the sample coal ash (ash content) set, and use the sample coal density set and the sample coal ash set as the ash analysis supervised training data. Then divide the ash analysis supervised training data into P equal parts to obtain P data sets, where P is an integer greater than or equal to 1, and the value of P can be set according to actual needs, such as 30; and select P times with replacement from the P data sets to construct the first branch supervised training data, and use the same method to iterate and select P times to obtain P pieces of branch supervised training data.

[0059] Next, P ash analysis branches are constructed based on the BP neural network. Here, the ash analysis branch is a BP neural network model in machine learning that can be iteratively optimized, including an input layer, multiple hidden layers, and an output layer. The input data of its input layer is the coal density, and the output data is the coal ash (ash content). Then, using the sample coal density as the input and the sample coal ash as the supervision, the P ash analysis branches are respectively supervised and trained using the P sets of branch supervision training data. First, the coal density is used as the input, passed through the input layer to the hidden layer, and then calculated through each hidden layer, and finally the prediction result (the ash content of the coal) is output. Next, the difference (error) between the prediction result and the true coal ash content is calculated through the loss function. The expression of the loss function is: ; where LOSS is the loss, M is the number of supervision training data in each set of branch supervision training data, is the sample coal ash in the i-th set of supervision training data, is the coal ash output by the ash analysis branch; then the gradient of each layer is calculated according to the loss function, the error is backpropagated using the chain rule, the weights and biases of each layer are adjusted backward from the output layer, and the gradient descent algorithm is used to adjust the weights and biases in the network according to the calculated gradient. The goal is to minimize the loss function; iterative training is carried out, and the weights and biases are optimized through multiple iterations until the loss function converges (less than the expected index), and the P trained ash analysis branches are obtained. Finally, the P trained ash analysis branches are combined to obtain the ash analysis channel.

[0060] S440: Multiply the error coefficient by P and take the integer to obtain Q, where Q is an integer greater than or equal to 1 and less than P; S450: Input the coal density into randomly selected Q ash analysis branches, output to obtain Q branch coal ashes, and calculate the mean to obtain the ash information of the coal.

[0061] Specifically, multiply the error coefficient by P and take the integer to obtain Q, where Q is an integer greater than or equal to 1 and less than P. For example, assume the error coefficient is 0.12 and P is 30, then Q is the integer obtained by rounding 0.12 * 30, which is 4. Then randomly call Q ash analysis branches among the P ash analysis branches of the ash analysis channel, and input the coal density into the randomly selected Q ash analysis branches, and output to obtain Q branch coal ashes (ash content); finally, calculate the mean of the Q branch coal ashes to obtain the ash information of the coal.

[0062] By matching and calling an appropriate number of ash analysis branches according to the error coefficient, that is, when the error coefficient is large, it indicates that the prediction accuracy of ash analysis is low, or the original data (such as coal image quality, point cloud fitting accuracy, etc.) is poor. At this time, by configuring more ash analysis branches to predict the coal ash content, the robustness and accuracy of the model can be increased. By integrating the prediction results of multiple branches, the deviation of a single branch can be effectively reduced, and the overall prediction accuracy can be improved. When the error coefficient is small, indicating that the prediction accuracy of the model is high, or the data quality of coal images, point clouds, etc. is good, the number of branches participating in the prediction will be reduced. This not only improves the calculation efficiency, but also reduces redundant calculations and the burden on the system, thus accelerating the ash detection process. At the same time, a small number of branches can still provide sufficient prediction accuracy. Especially in the case of good data quality, the processing speed can be increased without loss of accuracy. By controlling the selection of the number of branches through the error coefficient, computing resources can be dynamically allocated to achieve efficient and accurate ash detection.

[0063] The coal ash non-destructive detection method based on multi-modal data fusion provided by the embodiments of the present invention has at least the following technical effects:

[0064] By collecting multi-angle images, multi-angle point cloud data and quality data of coal, multiple coal images, multiple point cloud data and coal quality are obtained, where multi-angle includes the angles on both sides of the coal; then, coal contour recognition and image quality recognition are performed on the multiple coal images to obtain multiple coal contours and multiple image quality coefficients; then, coal point cloud iterative fitting is performed in the multiple point cloud data according to the multiple coal contours respectively to obtain multiple coal point clouds and multiple fitting accuracy coefficients, where the point cloud iterative fitting of the multiple coal contours is adjusted according to the multiple image quality coefficients respectively; further, a coal three-dimensional model is constructed according to the multiple coal point clouds, the coal volume information is processed, and the coal density is calculated according to the coal volume information and the coal quality; finally, the ash analysis accuracy is calculated and configured according to the multiple fitting accuracy coefficients and the multiple image quality coefficients, and the integrated ash analysis is performed on the coal density to obtain the ash information of the coal; that is to say, through multi-modal data fusion and integrated learning analysis, more efficient and accurate coal ash detection can be realized, significantly improving the applicability, efficiency and accuracy of ash content detection, so as to meet the real-time monitoring requirements of ash in the coal production process.

[0065] Embodiment 2, please refer to Figure 2 , Figure 2 which is a schematic diagram of the embodiment of the electronic device provided by the embodiments of the present invention. As Figure 2As shown in the figure, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a first computer program 511 stored on the memory 510 and operable on the processor 520. When the processor 520 executes the first computer program 511, the following steps are implemented: performing multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition on coal to obtain a plurality of coal images, a plurality of point cloud data, and coal quality; performing coal contour recognition and image quality recognition on the plurality of coal images to obtain a plurality of coal contours and a plurality of image quality coefficients, where the multi-angle includes angles on both sides of the coal; respectively performing coal point cloud iterative fitting in the plurality of point cloud data according to the plurality of coal contours to obtain a plurality of coal point clouds and a plurality of fitting accuracy coefficients, where respectively performing adjusted point cloud iterative fitting on the plurality of coal contours according to the plurality of image quality coefficients; constructing a coal three-dimensional model according to the plurality of coal point clouds, processing to obtain coal volume information, and calculating to obtain coal density according to the coal volume information and coal quality; calculating and configuring the ash analysis accuracy according to the plurality of fitting accuracy coefficients and the plurality of image quality coefficients, and performing integrated ash analysis on the coal density to obtain the ash information of the coal.

[0066] Embodiment Three, please refer to Figure 3 , Figure 3 which is a schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. As Figure 3 shown, this embodiment provides a computer-readable storage medium 600, on which a second computer program 611 is stored. When the second computer program 611 is executed by a processor, the following steps are implemented: performing multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition on coal to obtain a plurality of coal images, a plurality of point cloud data, and coal quality; performing coal contour recognition and image quality recognition on the plurality of coal images to obtain a plurality of coal contours and a plurality of image quality coefficients, where the multi-angle includes angles on both sides of the coal; respectively performing coal point cloud iterative fitting in the plurality of point cloud data according to the plurality of coal contours to obtain a plurality of coal point clouds and a plurality of fitting accuracy coefficients, where respectively performing adjusted point cloud iterative fitting on the plurality of coal contours according to the plurality of image quality coefficients; constructing a coal three-dimensional model according to the plurality of coal point clouds, processing to obtain coal volume information, and calculating to obtain coal density according to the coal volume information and coal quality; calculating and configuring the ash analysis accuracy according to the plurality of fitting accuracy coefficients and the plurality of image quality coefficients, and performing integrated ash analysis on the coal density to obtain the ash information of the coal.

[0067] It should be noted that in the above embodiments, the descriptions of each embodiment have their own focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0068] Those skilled in the art will understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0069] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0070] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are performed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0072] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts.

[0073] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A non-destructive detection method for coal ash content based on multi-modal data fusion, characterized in that, The method includes: Performing multi-angle image acquisition, multi-angle point cloud data acquisition, and quality acquisition on coal to obtain a plurality of coal images, a plurality of point cloud data, and coal quality. Performing coal contour recognition and image quality recognition on the plurality of coal images to obtain a plurality of coal contours and a plurality of image quality coefficients, where the multi-angle includes angles on both sides of the coal; Performing coal point cloud iterative fitting in the plurality of point cloud data respectively according to the plurality of coal contours to obtain a plurality of coal point clouds and a plurality of fitting accuracy coefficients, where the point cloud iterative fitting of the plurality of coal contours is adjusted respectively according to the plurality of image quality coefficients; Constructing a three-dimensional coal model based on the plurality of coal point clouds, processing to obtain coal volume information, and calculating the coal density according to the coal volume information and coal quality; Calculating and configuring the ash analysis accuracy according to the plurality of fitting accuracy coefficients and the plurality of image quality coefficients, and performing integrated ash analysis on the coal density to obtain the ash information of the coal; Performing coal point cloud iterative fitting in the plurality of point cloud data respectively according to the plurality of coal contours to obtain a plurality of coal point clouds and a plurality of fitting accuracy coefficients, including: Performing coal point cloud iterative fitting in the first point cloud data among the plurality of point cloud data according to the first coal contour among the plurality of coal contours to obtain a first initial fitting coal point cloud and a first initial fitting accuracy coefficient; Subtracting the first image quality coefficient in the plurality of image quality coefficients from 1 to obtain a first contour adjustment coefficient, using the first contour adjustment coefficient as a first contour adjustment ratio, and randomly adjusting the contour with the first contour adjustment ratio in the first coal contour to obtain a first adjusted coal contour; Performing coal point cloud iterative fitting in the first point cloud data using the first adjusted coal contour to obtain a first adjusted fitting coal point cloud and a first adjusted fitting accuracy coefficient; Continuing to perform the adjustment of the first coal contour and coal point cloud iterative fitting until convergence, outputting the maximum fitting accuracy coefficient and the fitting coal point cloud, and obtaining a first coal point cloud and a first fitting accuracy coefficient; Continuing to use the second coal contour, performing contour adjustment according to the second image quality coefficient, and performing coal point cloud iterative fitting in the second point cloud data until convergence to obtain a second coal point cloud and a second fitting accuracy coefficient.

2. The non-destructive detection method for coal ash content by multi-modal data fusion according to claim 1, characterized in that, Performing coal contour recognition and image quality recognition on the plurality of coal images to obtain a plurality of coal contours and a plurality of image quality coefficients, including: Pre-training a coal image recognition channel, where the coal image recognition channel includes a coal contour recognition path and a noise recognition branch; Inputting the plurality of coal images into the coal image recognition channel respectively to recognize a plurality of coal contours and a plurality of noise ratios; Subtracting the plurality of noise ratios from 1 to obtain a plurality of image quality coefficients.

3. The non-destructive detection method for coal ash content by multi-modal data fusion according to claim 2, characterized in that, Pre-training a coal image recognition channel, including: Collecting a sample coal image set according to coal detection data within a historical time, annotating the proportion of coal contours and noise pixel points in each sample coal image to obtain a sample coal contour set and a sample noise ratio set; Using the set of sample coal images as input data, and respectively using the set of sample coal contours and the set of sample noise ratios as supervision data, based on a convolutional neural network, respectively supervise and train the coal contour recognition path and the noise recognition branch; Combine the trained coal contour recognition path and the noise recognition branch to obtain a coal image recognition channel.

4. The non-destructive detection method for coal ash content with multi-modal data fusion according to claim 1, characterized in that, According to the first coal contour within the multiple coal contours, perform iterative fitting of coal point clouds within the first point cloud data among the multiple point cloud data to obtain a first initial fitted coal point cloud and a first initial fitting accuracy coefficient, including: According to the first coal contour, perform random box selection within the first point cloud data to obtain a first box-selected coal point cloud; Calculate the ratio of the point cloud area ratio within the first box-selected coal point cloud to the area of the first coal contour as the first box-selection accuracy coefficient; Continue to use the first coal contour to perform random box selection within the first point cloud data for iterative box selection fitting of coal point clouds until the convergence box selection fitting times are reached, and output the maximum box-selection accuracy coefficient and the corresponding box-selected coal point cloud as the first initial fitting accuracy coefficient and the first initial fitted coal point cloud.

5. The non-destructive detection method for coal ash content with multi-modal data fusion according to claim 1, characterized in that, According to the multiple coal point clouds, construct a 3D coal model, process to obtain coal volume information, and calculate the coal density according to the coal volume information and the coal quality, including: Construct a 3D coal model according to the first coal point cloud and the second coal point cloud within the multiple coal point clouds; Calculate the coal volume information according to the 3D coal model; Calculate the ratio of the coal quality to the coal volume information to obtain the coal density.

6. The non-destructive detection method for coal ash content by multi-modal data fusion according to claim 1, characterized in that, According to the multiple fitting accuracy coefficients and multiple image quality coefficients, calculate and configure the ash analysis accuracy, perform integrated ash analysis on the coal density, and obtain the ash information of the coal, including: Calculate the mean of the multiple fitting accuracy coefficients to obtain the average fitting accuracy coefficient, and calculate the mean of the multiple image quality coefficients to obtain the average image quality coefficient; Respectively use 1 minus the average fitting accuracy coefficient and the average image quality coefficient to obtain the fitting error coefficient and the image error coefficient, and sum them to obtain the error coefficient; Train an ash analysis channel including P ash analysis branches, where P is an integer greater than or equal to 1; Multiply the error coefficient by P and round down to obtain Q, where Q is an integer greater than or equal to 1 and less than P; Input the coal density into randomly selected Q ash analysis branches, output to obtain Q branch coal ash, and calculate the mean to obtain the ash information of the coal.

7. The non-destructive detection method for coal ash content of multimodal data fusion according to claim 6, wherein Training an ash analysis channel including P ash analysis branches, including: According to the historical test data of coal ash, collect a set of sample coal densities and a set of sample coal ashes as ash analysis supervision training data; Randomly select P data from the ash analysis supervision training data with replacement to obtain P data for branch supervision training; Respectively use the P data for branch supervision training to supervise and train P ash analysis branches, where the loss function for each ash analysis branch supervision training is as follows: ； Where LOSS is the loss, M is the number of supervised training data in each branch supervised training data, is the sample coal ash in the i-th group of supervised training data, is the coal ash output by the ash analysis branch; Combine the trained P ash analysis branches to obtain an ash analysis channel.

8. An electronic device, characterized in that, Including: A memory for storing computer software programs; A processor for reading and executing the computer software programs, thereby implementing the steps of the non-destructive coal ash detection method for multimodal data fusion according to any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium, characterized in that, The computer software programs are stored in the storage medium, and when the computer software programs are executed by the processor, the steps of the non-destructive coal ash detection method for multimodal data fusion according to any one of claims 1 to 7 are implemented.

Images