Image-based urban solid waste calorific value prediction method

Abstract

The invention discloses an image-based urban solid waste (MSW) calorific value prediction method. According to the method, on the basis of a monitoring video of a garbage storage pool, an in-furnace garbage image is captured and screened manually, and an efficient multi-module fusion learning framework is developed by using a deep learning technology. According to the framework, through extracting time sequence features and image visual features of calorific value labels and introducing a variational auto-encoder to process high-dimensional combined features, dynamic prediction of the incineration calorific value of the garbage entering the furnace can be realized. According to the actual working condition, two different time sequence information input strategies are explored, one strategy is that the real heat value at the previous moment is used as input, the other strategy is that the predicted heat value at the previous moment is used as input, and effective information reference and decision support can be provided for incineration intelligent control of the waste incineration power plant.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a multi-module deep learning framework applied to garbage calorific value prediction. The present invention provides an image-based urban solid waste calorific value prediction method. Background Art

[0002] With the rapid development of urbanization, the treatment of urban solid waste has become an important and challenging task. Incineration, as a way of garbage disposal, has the advantages of reducing environmental impact and utilizing waste heat. Among them, the calorific value of garbage is a parameter that is controlled and optimized during the incineration process. However, the current prediction of the calorific value of garbage entering the furnace relies solely on the operator's experience and judgment, and cannot be quantitatively predicted. After entering the furnace for incineration, the calorific value of the garbage is calculated by reverse calculation based on the amount of steam generated, which has no guiding significance for combustion control, resulting in inefficient process control. Therefore, accurate and timely prediction of the calorific value of garbage is of great significance for the optimization of the incineration process. Deep learning models can analyze complex visual data and extract effective patterns related to the physical and chemical properties of garbage, providing new ideas for calorific value prediction.

[0003] However, most existing calorific value prediction models rely on genetic algorithms or neural networks, and cannot predict the calorific value of garbage during incineration based on image recognition technology. To this end, this paper proposes a multi-module deep learning framework that integrates time series features and image features, providing a more accurate real-time solution for calorific value prediction tasks. Summary of the invention

[0004] The invention provides an image-based municipal solid waste (MSW) calorific value prediction method, which can realize dynamic prediction of the incineration calorific value of the garbage entering the furnace.

[0005] The image-based urban solid waste calorific value prediction method provided by the present invention comprises:

[0006] Step S101, based on the monitoring video of the garbage storage pool, obtain the image of the garbage entering the furnace, collect the time series calorific value data for time tagging, take the image of the garbage entering the furnace in the monitoring video of the garbage storage pool as the starting time, and retrieve the average calorific value within 40 to 60 minutes after the starting time in the time series calorific value data as the time series calorific value label of the image of the garbage entering the furnace, so as to obtain the data set to be processed;

[0007] Step S102, constructing a deep learning model, using the data set to be processed to extract time series features and image features in the data set to be processed, and adopting a planned sampling training strategy to simulate different application scenarios, using a supervised learning method to train the model, and gradually adjusting the model hyperparameters and structure to further improve the prediction accuracy;

[0008] Step S103, after the training is completed, the test set is input into the model for performance evaluation. The prediction effect of the model on the test set is evaluated by calculating indicators such as mean square error (MSE) and mean absolute error (MAE), and combining model decision visualization technology; after the test is completed, the solid waste calorific value prediction model is finally obtained;

[0009] Step S104, performing real-time analysis on the monitoring video of the garbage storage pool, extracting real-time images and time information data of the garbage entering the furnace, and inputting them into the solid waste calorific value prediction model to obtain the predicted calorific value of the solid waste.

[0010] Furthermore, the image-based urban solid waste calorific value prediction method of the present invention, step S101, comprises:

[0011] Obtain real-time monitoring video streams from the monitoring system of the garbage storage pool, manually filter out garbage images entering the furnace from the monitoring video, and the garbage images represent garbage samples that are about to enter the incinerator. Time-stamp each garbage image entering the furnace and record the time point when the image was captured;

[0012] Obtain the time series calorific value data recorded synchronously by the surveillance video, which includes the calorific value of waste incineration at different time points;

[0013] For each image of garbage entering the furnace, a time window is set with the time point when each image of garbage entering the furnace is captured as the starting time. The range of the window is 40 to 60 minutes from the starting time. The selection of the time window is set according to the transmission time of the garbage entering the furnace before incineration;

[0014] In a determined time window, corresponding calorific value data points are extracted from the time series calorific value data, and the average value of these calorific value data points is calculated as the time series calorific value label of the waste image entering the furnace;

[0015] Pair each waste image with its corresponding time series calorific value label to form a data pair including image and label. Arrange the pre-processed waste images and corresponding time series calorific value labels to obtain the data set to be processed.

[0016] The preprocessed data is divided into training set, validation set and test set. The training set and validation set are used for model training, and the test set is used to evaluate the performance of the model.

[0017] Furthermore, the image-based urban solid waste calorific value prediction method of the present invention, step S102 comprises:

[0018] The constructed multi-module fusion deep learning model mainly consists of three modules: temporal feature extraction module, visual feature extraction module and encoding and decoding module;

[0019] For time series data, the LSTM layer in the time series feature extraction module is used to capture the time dependency and sequence features in the data;

[0020] For image data, the convolutional layer in the visual feature extraction module is used to extract the texture and shape features of the image;

[0021] The spliced ​​temporal features and image features are input into the downstream encoding and decoding module for processing;

[0022] The training samples are selected from the data set to be processed by planned sampling, and the deep learning model is trained using the selected training samples. During the training process, the prediction value is calculated by forward propagation, and the model parameters are updated by back propagation to minimize the loss function.

[0023] Set appropriate training rounds, learning rate, batch size hyperparameters, and monitor the loss and accuracy indicators during the training process to evaluate the training effect of the model. According to the performance on the validation set, adjust the model hyperparameters or optimization strategy, and repeat the training and validation process until the performance of the model on the validation set meets the expected requirements. When the performance indicators of the validation set are stable and meet the requirements, save the trained model.

[0024] Furthermore, the image-based urban solid waste calorific value prediction method of the present invention, step S103 comprises:

[0025] After training is completed, the image data and time data in the test set are input into the trained model to independently evaluate the generalization performance of the model. During the test, the mean square error (MSE), root mean square error (RMSE) and other indicators on the test set are calculated to measure the prediction accuracy and robustness of the model on unseen data. The test results can be used as the final evaluation basis for the model performance.

[0026] The class activation map visualization technology is used to display the key areas that the model focuses on when processing the images of garbage entering the furnace. The model's prediction results are further qualitatively analyzed to intuitively display the model's areas of interest in the image and verify whether the image features it focuses on are highly correlated with the calorific value of incineration.

[0027] Furthermore, the image-based urban solid waste calorific value prediction method of the present invention, step S104 comprises:

[0028] Conduct real-time analysis on the monitoring video of the garbage storage pool, capture the image data of the garbage entering the furnace, integrate the time series calorific value data of the previous time step and the real-time image data of the garbage entering the furnace, substitute them into the solid waste calorific value prediction model, trigger the model to make predictions, and the model will output the predicted calorific value of the solid waste in real time based on the input data, display the predicted calorific value on the interface of the monitoring system, record it in the log file, and send it to the control system synchronously.

[0029] Beneficial effects of the present invention:

[0030] The present invention combines time series features and image visual features, and introduces variational autoencoders to process high-dimensional combined features. The present invention can more accurately predict the incineration calorific value of municipal solid waste (MSW). This multi-module fusion learning framework significantly improves the accuracy of prediction and provides reliable data support for the intelligent control of incineration in waste incineration power plants.

[0031] The present invention relies on the monitoring video of the garbage storage pool to capture and screen the images of the garbage entering the furnace in real time, and predict the calorific value based on this. The real-time nature enables the power plant to adjust the incineration parameters in time, optimize the combustion process, and improve energy utilization efficiency.

[0032] The present invention explores two different strategies for inputting time series information, namely using the real calorific value of the previous moment or using the predicted calorific value of the previous moment as input. This flexibility enables the model to adapt to different working scenarios and data conditions, and improves the generalization ability of the model. By providing accurate prediction of the incineration calorific value, the present invention provides a scientific decision-making basis for the incineration control of waste incineration power plants. This helps power plants optimize their operating strategies, reduce operating costs, and improve economic benefits.

[0033] The present invention applies deep learning technology to the field of calorific value prediction of municipal solid waste, providing new ideas and methods for research and application in related fields. This helps to promote technological progress and innovative development in this field. By improving the accuracy of incineration calorific value prediction, the present invention helps waste incineration power plants to use energy more efficiently and reduce pollution emissions. This is of great significance for promoting environmental protection and sustainable development.

[0034] In summary, the present invention realizes the accurate prediction of the calorific value of urban solid waste incineration by combining deep learning technology and a multi-module fusion learning framework, provides strong support for the intelligent control of waste incineration power plants, and brings significant economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A schematic diagram showing the municipal solid waste incineration power generation process.

[0037] Figure 2A schematic diagram showing some image samples of the dataset.

[0038] Figure 3 Schematic diagram showing the model prediction performance for two input strategies.

[0039] Figure 4 The diagram shows the prediction errors and visual analysis of some samples.

[0040] Figure 5 The overall framework schematic diagram of the image-based urban solid waste calorific value prediction method proposed in this patent is shown.

[0041] Figure 6 A schematic diagram of the image-based urban solid waste calorific value prediction method proposed in this patent is shown. DETAILED DESCRIPTION

[0042] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0043] Step S101, based on the monitoring video of the garbage storage pool, obtain the image of the garbage entering the furnace, collect the time series calorific value data for time tagging, take the image of the garbage entering the furnace in the monitoring video of the garbage storage pool as the starting time, and retrieve the average calorific value within 40 to 60 minutes after the starting time in the time series calorific value data as the time series calorific value label of the image of the garbage entering the furnace, so as to obtain the data set to be processed;

[0044] Real-time monitoring video stream acquisition,First, access the real-time monitoring video stream from the monitoring system of the garbage storage pool.

[0045] The acquired video stream is decoded and converted into a processable image sequence.

[0046] From the decoded image sequence, manually filter out the garbage images entering the furnace. The screening process should be based on the moment when the garbage is about to enter the incinerator, so that the selected images represent the garbage samples that are about to be incinerated. Each screened garbage image entering the furnace is time-stamped to record the specific time point when the image was captured. The time stamp should be accurate to seconds to ensure accurate correspondence of subsequent time series calorific value data.

[0047] Time series calorific value data is acquired synchronously by connecting with the control system or data recording system of the waste incinerator. The time series calorific value data should include the calorific value records of waste incineration at different time points. The acquired time series calorific value data is consistent with the time format of the image data, which is convenient for subsequent time correspondence and data processing.

[0048] For each image of garbage entering the furnace, a time window is set with the time point when it is captured as the starting time. The range of the window is set to 40 to 60 minutes after the starting time. The selection of the time window is based on the transmission time of the garbage entering the furnace before incineration and the response time of the incinerator.

[0049] In a certain time window, the corresponding calorific value data points are extracted from the time series calorific value data, and the average value of these calorific value data points is calculated as the time series calorific value label of the waste image entering the furnace.

[0050] Each image of waste entering the furnace is paired with its corresponding time series thermal value label to form a data pair containing an image and a label.

[0051] The paired data is cleaned to remove outliers or missing values. Outliers may include cases where the calorific value data deviates significantly from the normal range, while missing values ​​may be caused by data recording errors or transmission problems.

[0052] In order to increase the diversity of data and improve the generalization ability of the model, the cleaned data is enhanced by performing operations such as random rotation, scaling, and cropping of images, as well as smoothing of time series data.

[0053] Training set, validation set and test set division: The preprocessed data is divided into training set, validation set and test set according to a certain ratio. Usually, the training set is used for the model training process, the validation set is used to adjust the model hyperparameters and monitor the training effect, and the test set is used to finally evaluate the performance of the model.

[0054] Step S102, constructing a deep learning model, using the data set to be processed to extract time series features and image features in the data set to be processed, and adopting a planned sampling training strategy to simulate different application scenarios, using a supervised learning method to train the model, and gradually adjusting the model hyperparameters and structure to further improve the prediction accuracy;

[0055] The constructed multi-module fusion deep learning model mainly consists of three core modules: time series feature extraction module, visual feature extraction module and encoding and decoding module. These three modules work together to make full use of the information in time series data and image data to achieve accurate prediction of the calorific value of municipal solid waste.

[0056] The time series feature extraction module receives time series calorific value data as input, which is the calorific value of waste incineration arranged in chronological order and is used to capture the changing trend of calorific value over time.

[0057] LSTM layer: The long short-term memory network (LSTM) layer is used to extract the time dependency and sequence features in the time series data. LSTM is a special recurrent neural network (RNN) that can effectively handle the long-term dependency problem in sequence data.

[0058] Parameter setting: Set the number of LSTM layers to 4, the hidden layer size to 64, and the input layer size to 2 (corresponding to the heat value data of each time step). Output the features of the last time step as the representation of the time series features.

[0059] The visual feature extraction module receives as input images of incoming waste. These images are captured through surveillance videos of the waste storage pool and represent samples of waste that are about to enter the incinerator.

[0060] Convolutional layer: The convolutional layer in the convolutional neural network (CNN) is used to extract the texture and shape features of the image. Different convolutional neural network architectures can be selected, such as DenseNet, to meet different image feature extraction requirements.

[0061] DenseNet network: Through dense connections between layers, it avoids the problems of gradient vanishing and feature loss, and fully extracts and fuses image features at different levels.

[0062] The feature splicing and encoding and decoding module splices the temporal features output by the temporal feature extraction module and the image features output by the visual feature extraction module to form a high-dimensional combined feature.

[0063] Variational Autoencoder (VAE): Variational Autoencoder is introduced to process high-dimensional combined features. VAE compresses high-dimensional features into low-dimensional latent space through the encoder, and then reconstructs them through the decoder to extract representative features. During the training process, the reconstruction error is minimized to optimize the model performance.

[0064] The training strategy of planned sampling is adopted to select training samples from the dataset to be processed. The generalization ability of the model is improved by simulating different application scenarios. The model is trained using supervised learning methods. The prediction value is calculated through forward propagation, and the model parameters are updated through back propagation to minimize the loss function.

[0065] Set appropriate hyperparameters such as the number of training rounds, learning rate, and batch size, and monitor the loss indicators during training. Based on the performance on the validation set, gradually adjust the model hyperparameters until the model performance meets the expected requirements.

[0066] When the performance indicators on the validation set are stable and meet the requirements, save the trained model. This model will be used for subsequent real-time calorific value prediction tasks.

[0067] Step S103, after the training is completed, the test set is input into the model for performance evaluation. The prediction effect of the model on the test set is evaluated by calculating indicators such as mean square error (MSE) and mean absolute error (MAE), and combining model decision visualization technology; after the test is completed, the solid waste calorific value prediction model is finally obtained;

[0068] After training is completed, the test set is input into the model for performance evaluation to ensure the effectiveness and reliability of the model in practical applications. This step specifically includes the following technical details:

[0069] Test set division: In the data preprocessing stage, the data set is divided into training set, validation set and test set. The test set is independent of the training set and validation set and is used to evaluate the generalization ability of the model.

[0070] Data preprocessing: Preprocess the image data and time data in the test set, including image resizing and normalization, to ensure that the input data format is consistent with that during training.

[0071] Mean Square Error (MSE): Calculates the average of the squared differences between the model's predicted calorific value and the actual calorific value, used to measure the magnitude of the prediction error. The smaller the MSE, the more accurate the model prediction.

[0072] Mean Absolute Error (MAE): Calculates the average absolute difference between the model's predicted calorific value and the actual calorific value, which is used to measure the average level of prediction error. The smaller the MAE, the more stable the model prediction.

[0073] The image data and time data in the test set are input into the trained model, and the prediction value of the model is calculated independently. Avoid fine-tuning the model using the training set or validation set data to ensure the objectivity and fairness of the evaluation results.

[0074] The three evaluation indicators of MSE, MAE and R2 are calculated at the same time to comprehensively measure the prediction performance of the model. By comparing the evaluation results of different models, the model with the best performance is selected as the final solid waste calorific value prediction model.

[0075] Class Activation Map (GradCAM++): Use class activation map visualization technology to show the key areas that the model focuses on when processing images of incoming garbage. By generating a heat map, the model's emphasis on different areas in the image is intuitively displayed, further verifying the reliability and interpretability of the model's predictions. Combined with the visualization results, the model's prediction results are qualitatively analyzed. By comparing the areas that the model focuses on with the areas related to the actual high calorific value features, it is verified whether the model can effectively capture the key information in the image and make accurate calorific value predictions. Record evaluation results: The evaluation results on the test set are recorded in a log file, including quantitative indicators such as MSE, MAE, R2, and visualization analysis results.

[0076] Feedback on the model performance is provided based on the evaluation results. If the model performance does not meet the expected requirements, the process returns to step S102 to adjust the model structure or hyperparameters, and retrain and evaluate; if the model performance meets the requirements, the process proceeds to the next step to perform real-time calorific value prediction.

[0077] Step S104, performing real-time analysis on the monitoring video of the garbage storage pool, extracting real-time images and time information data of the garbage entering the furnace, and inputting them into the solid waste calorific value prediction model to obtain the predicted calorific value of the solid waste.

[0078] Real-time video stream processing: First, the real-time monitoring video stream is continuously obtained from the monitoring system of the garbage storage pool. Using video frame capture technology, image frames are extracted from the video stream at a preset frame rate (such as 10 frames per second).

[0079] The extracted image frames are preprocessed, including resizing (such as uniformly adjusting to 224×224 pixels), denoising, grayscale conversion (if the model is trained based on grayscale images), etc., to meet the requirements of model input.

[0080] Time information synchronization: For each captured image frame, record its capture timestamp to ensure the accuracy of time information. Synchronously obtain the time series calorific value data corresponding to the image frame timestamp, which may come from the calorific value monitoring system of the incinerator.

[0081] Data integration and input: According to the timestamp of the image frame, the calorific value data of the previous moment (such as the previous time step) is extracted from the time series calorific value data. Here, the real calorific value or the predicted calorific value can be selected according to the two strategies described in step S101. The real-time image frame and the corresponding calorific value data of the previous moment are integrated into a data sample.

[0082] The integrated data samples are input into the trained solid waste calorific value prediction model.

[0083] After receiving the input data, the model processes it through its internal multi-module fusion deep learning framework. The model first extracts the visual features of the image frame and combines them with the temporal features for fusion processing. After being processed by the encoding and decoding module, the model outputs the predicted calorific value of solid waste.

[0084] The predicted calorific value results are displayed in real time on the monitoring system interface of the waste incineration power plant for real-time monitoring by operators. The predicted results are recorded in a log file for subsequent data analysis and auditing. The predicted results are synchronously sent to the control system of the incinerator, and the control system can adjust the incineration parameters (such as combustion temperature, air supply, etc.) according to the predicted calorific value to achieve a more efficient incineration process.

[0085] In actual applications, abnormal situations may occur, such as prediction results beyond a reasonable range, model prediction performance degradation, etc. In response to these situations, design corresponding exception handling mechanisms, such as issuing alarms, automatically adjusting model parameters, or retraining models, etc. Collect feedback data during the exception handling process for subsequent model optimization and improvement.

[0086] Specifically, the image-based urban solid waste calorific value prediction method of the present invention, step S101, comprises:

[0087] Obtain real-time monitoring video streams from the monitoring system of the garbage storage pool, manually filter out garbage images entering the furnace from the monitoring video, and the garbage images represent garbage samples that are about to enter the incinerator. Time-stamp each garbage image entering the furnace and record the time point when the image was captured;

[0088] Obtain the time series calorific value data recorded synchronously by the surveillance video, which includes the calorific value of waste incineration at different time points;

[0089] For each image of garbage entering the furnace, a time window is set with the time point when each image of garbage entering the furnace is captured as the starting time. The range of the window is 40 to 60 minutes from the starting time. The selection of the time window is set according to the transmission time of the garbage entering the furnace before incineration;

[0090] In a determined time window, corresponding calorific value data points are extracted from the time series calorific value data, and the average value of these calorific value data points is calculated as the time series calorific value label of the waste image entering the furnace;

[0091] Pair each waste image with its corresponding time series calorific value label to form a data pair including image and label. Arrange the pre-processed waste images and corresponding time series calorific value labels to obtain the data set to be processed.

[0092] The preprocessed data is divided into training set, validation set and test set. The training set and validation set are used for model training, and the test set is used to evaluate the performance of the model.

[0093] In step S101, the image-based method for predicting the calorific value of municipal solid waste involves obtaining images of waste entering the furnace from the monitoring video of the waste storage pool, and collecting corresponding time series calorific value data for time tagging. Step S101: Data preparation and preprocessing

[0094] Obtain real-time monitoring video streams from the monitoring system of the garbage storage pool. The video streams should cover the key areas before the garbage enters the incinerator to ensure that clear images of the garbage entering the incinerator can be captured.

[0095] Operators need to monitor the video stream in real time and manually capture or mark images of the garbage that is about to be sent to the incinerator. These images should accurately reflect the appearance characteristics of the batch of garbage.

[0096] Each image of the waste that is screened out is time-stamped to record the exact time point when the image is captured. The accuracy of the time stamp should be high enough to ensure the accurate matching of the subsequent time series calorific value data.

[0097] Synchronously obtain the time series calorific value data corresponding to the monitoring video records. These data are usually provided by the online monitoring system of the waste incinerator and include the calorific value of waste incineration at different time points.

[0098] For each image of the garbage entering the furnace, a time window is set with the time point when it is captured as the starting time. The range of this time window is set to 40 to 60 minutes after the starting time. This range is selected based on the transmission and processing time of the garbage entering the furnace in the system before incineration to ensure that the extracted calorific value data can accurately reflect the actual incineration calorific value of the batch of garbage.

[0099] In the set time window, the corresponding calorific value data points are extracted from the time series calorific value data. These data points should cover all available calorific value data within the entire time window. The average value of the extracted calorific value data points is calculated as the time series calorific value label of the garbage image entering the furnace. This average value can more accurately reflect the average calorific value level of the batch of garbage during the incineration process.

[0100] Pair each waste image with its corresponding time series calorific value label to form a data pair containing an image and a label. Arrange all preprocessed data pairs to form a dataset to be processed. The dataset should contain enough samples to ensure the adequacy of model training and the accuracy of prediction.

[0101] The preprocessed data is divided into training set, validation set and test set. The training set and validation set are used for model training and optimization, and the test set is used to evaluate the final performance of the model. The proportion of data division should be reasonably set according to actual conditions and needs.

[0102] Through the above steps, step S101 completes the tasks of acquiring images of waste entering the furnace from the monitoring video, collecting time series calorific value data, setting the time window, extracting and calculating time series calorific value labels, pairing images and labels, and organizing the data sets to be processed, providing a solid foundation for subsequent model training and prediction.

[0103] Specifically, the image-based urban solid waste calorific value prediction method of the present invention, step S102 includes:

[0104] The constructed multi-module fusion deep learning model mainly consists of three modules: temporal feature extraction module, visual feature extraction module and encoding and decoding module;

[0105] For time series data, the LSTM layer in the time series feature extraction module is used to capture the time dependency and sequence features in the data;

[0106] For image data, the convolutional layer in the visual feature extraction module is used to extract the texture and shape features of the image;

[0107] The spliced ​​temporal features and image features are input into the downstream encoding and decoding module for processing;

[0108] The training samples are selected from the data set to be processed by planned sampling, and the deep learning model is trained using the selected training samples. During the training process, the prediction value is calculated by forward propagation, and the model parameters are updated by back propagation to minimize the loss function.

[0109] Set appropriate training rounds, learning rate, batch size hyperparameters, and monitor the loss and accuracy indicators during the training process to evaluate the training effect of the model. According to the performance on the validation set, adjust the model hyperparameters or optimization strategy, and repeat the training and validation process until the performance of the model on the validation set meets the expected requirements. When the performance indicators of the validation set are stable and meet the requirements, save the trained model.

[0110] The multi-module fusion deep learning model mainly consists of three core modules: temporal feature extraction module, visual feature extraction module and encoding and decoding module.

[0111] Time series feature extraction module: This module is responsible for capturing time dependency and sequence features from time series data. Long short-term memory network (LSTM) is used as the core component to build a network structure with 4 LSTM layers. The hidden layer size of each LSTM layer is set to 64, and the input layer size is 2 (corresponding to the two dimensions of the time series data). The output of the LSTM layer will be used as the time series feature for subsequent processing.

[0112] Visual feature extraction module: This module is responsible for extracting texture and shape features from the waste images. Dense convolutional network (DenseNet) or Transformer model is used as the image feature extractor. DenseNet avoids the problem of gradient vanishing and feature loss through dense connections between layers, and can fully extract and fuse image features at different levels. The Transformer model can associate features of different regions in the image under complex feature backgrounds and effectively extract global features. The choice of these two models depends on the specific application scenario and data characteristics.

[0113] Encoding and decoding module: This module is responsible for processing the spliced ​​time series features and image features, and outputs the final prediction results. The variational autoencoder (VAE) is used as the core component of the encoding and decoding module. VAE compresses high-dimensional features into a low-dimensional latent space by improving the encoder and decoder structure, and then comprehensively processes the combined features. This processing method can more effectively capture representative features, thereby improving the prediction performance of downstream regression tasks.

[0114] In order to simulate different application scenarios and improve the generalization ability of the model, a planned sampling training strategy is adopted. Specifically, according to the actual changes in the calorific value of the garbage entering the incineration process, different sampling schemes are designed to cover various possible calorific value change patterns. During the training process, different application scenarios are simulated by randomly selecting sampling schemes, thereby improving the adaptability of the model.

[0115] Supervised learning, supervised learning methods are used to train the model. During the training process, the predicted value is calculated through forward propagation, and the model parameters are updated through back propagation to minimize the loss function. The loss function includes mean square error (MSE) and KL divergence loss, which are used to measure the difference between the predicted value and the actual value and the rationality of the latent space distribution.

[0116] Hyperparameter adjustment: To further improve prediction accuracy, gradually adjust the model's hyperparameters and structure. Hyperparameters include the number of training rounds, learning rate, batch size, etc. The selection of parameters has an important impact on the training effect and performance of the model. The training effect of the model is evaluated by monitoring the loss and accuracy indicators during the training process, and the hyperparameters or optimization strategies are adjusted based on the performance on the validation set.

[0117] Model saving: When the performance indicators of the validation set are stable and meet the requirements, save the trained model.

[0118] Specifically, the image-based urban solid waste calorific value prediction method of the present invention, step S103 includes:

[0119] After training is completed, the image data and time data in the test set are input into the trained model to independently evaluate the generalization performance of the model. During the test, the mean square error (MSE), root mean square error (RMSE) and other indicators on the test set are calculated to measure the prediction accuracy and robustness of the model on unseen data. The test results can be used as the final evaluation basis for the model performance.

[0120] The class activation map visualization technology is used to display the key areas that the model focuses on when processing the images of garbage entering the furnace. The model's prediction results are further qualitatively analyzed to intuitively display the model's areas of interest in the image and verify whether the image features it focuses on are highly correlated with the calorific value of incineration.

[0121] After the model training is completed, step S103 aims to comprehensively evaluate the prediction performance of the model to ensure its effectiveness and reliability in practical applications. The specific steps are as follows: a test set and a test set are divided from the preprocessed data set. The data in the test set should maintain a data distribution similar to that of the training set and the validation set, but should not contain duplicate data points.

[0122] Performance evaluation indicators;

[0123] Mean Square Error (MSE): Calculates the average of the squares of the difference between the calorific value predicted by the model and the actual calorific value. The smaller the MSE, the smaller the prediction error of the model.

[0124] Mean Absolute Error (MAE): Calculates the average of the absolute values ​​of the difference between the calorific value predicted by the model and the actual calorific value. The smaller the MAE, the smaller the prediction error of the model.

[0125] Root mean square error (RMSE): The square root of the MSE, similar to the MSE, but with the same dimensions as the original data for easier understanding.

[0126] R-squared value (R2): Indicates the degree of fit between the model prediction value and the actual value. The closer R2 is to 1, the better the model fit is.

[0127] During the evaluation process, the image data and time data in the test set are input into the trained model, and the model will output the predicted calorific value based on these data. For each sample in the test set, the MSE, MAE, RMSE and R2 between its predicted calorific value and the actual calorific value are calculated. The evaluation indicators of all test samples are summarized and the average or median is calculated to obtain the performance of the model on the entire test set.

[0128] Model decision visualization uses visualization techniques such as class activation maps (Grad-CAM++) to show the key areas that the model focuses on when processing images of garbage entering the furnace. Through visual analysis, it is verified whether the image features that the model focuses on are highly correlated with the calorific value of incineration, thereby enhancing the interpretability of the model's prediction results.

[0129] According to the performance evaluation results on the test set, analyze the prediction performance of the model on different types of samples and identify potential prediction biases or deficiencies. According to the deficiencies of the model, adjust the model structure, hyperparameters or training strategies to further optimize the prediction performance of the model. Repeat steps S102 and S103 until the performance of the model on the test set meets the expected requirements and has high robustness and generalization ability.

[0130] The performance evaluation results of the model on the test set are recorded in the log file, including the specific values ​​of each evaluation indicator and the visual analysis results.

[0131] Specifically, the image-based urban solid waste calorific value prediction method of the present invention, step S104 includes:

[0132] Conduct real-time analysis on the monitoring video of the garbage storage pool, capture the image data of the garbage entering the furnace, integrate the time series calorific value data of the previous time step and the real-time image data of the garbage entering the furnace, substitute them into the solid waste calorific value prediction model, trigger the model to make predictions, and the model will output the predicted calorific value of the solid waste in real time based on the input data, display the predicted calorific value on the interface of the monitoring system, record it in the log file, and send it to the control system synchronously.

[0133] Video stream capture: Real-time capture of surveillance video streams from the monitoring system of the garbage storage pool. This requires the system to have efficient video processing capabilities and be able to continuously and stably obtain high-quality video data.

[0134] Image frame extraction: Use image processing technology to extract images frame by frame from real-time video streams. According to the frame rate setting of the surveillance video, the system should be able to ensure the real-time and accuracy of image extraction.

[0135] Garbage image screening: Preprocess the extracted image frames to screen out images that represent garbage samples that are about to enter the incinerator. This usually involves techniques such as image segmentation and object detection to identify and locate garbage images.

[0136] Timestamp recording: Record a timestamp for each filtered junk image to mark the exact time point when it was captured. This helps to accurately align with the time series heat value data later.

[0137] Time series data synchronization: obtain the time series calorific value data recorded synchronously with the monitoring video, so that the timestamp corresponds to the calorific value data point one by one. The time series calorific value data should include the calorific value of garbage incineration in the incinerator at different time points.

[0138] Integrate the time series calorific value data of the previous time step and the real-time filtered garbage image data. If the predicted calorific value of the previous moment is used as the input strategy, it is also necessary to obtain the predicted value from the prediction model. Input the integrated data into the solid waste calorific value prediction model. This requires the model to be able to process real-time input data and quickly generate prediction results.

[0139] The solid waste calorific value prediction model dynamically generates the predicted value of the calorific value of waste incineration based on the input time series calorific value data and waste image data through the internal multi-module fusion learning framework. The predicted calorific value is output in real time and displayed on the interface of the monitoring system for real-time monitoring by operators. At the same time, the predicted calorific value is recorded in the log file for subsequent analysis and tracing. The predicted calorific value is synchronously sent to the control system of the waste incineration power plant to provide data support for the real-time adjustment of the incineration parameters. This helps the power plant optimize the combustion process and improve energy efficiency.

[0140] Continuously monitor the performance of the prediction model in real-time applications, and pay attention to the accuracy and stability of the prediction value. If the prediction effect is found to be reduced, the model should be adjusted or retrained in time. According to the real-time prediction results and the actual feedback of the incineration process, the incineration parameters such as furnace temperature and air volume should be adjusted in time to optimize the combustion process and improve the utilization rate of calorific value. Regularly optimize and upgrade the entire system, including the replacement of hardware equipment and the continuous optimization of software algorithms.

[0141] This method uses the long short-term memory network model (LSTM) to extract the time series features of the calorific value of garbage and capture the change pattern of the calorific value over time. In the face of the high-dimensional and complex features in the garbage images entering the furnace, a dense convolutional network (DenseNet) or Transformer model is used as an image feature extractor. The joint features (time series-image) are compressed into a low-dimensional latent space through the variational autoencoder model (VAE) to extract representative features for improving the performance of downstream regression prediction tasks.

[0142] The images of the garbage entering the furnace are obtained through the monitoring video of the garbage storage pool, and according to the garbage transportation process of the waste incineration power plant, the average calorific value within 40-60 minutes after the feeding time is used as the temporal calorific value label of the garbage entering the furnace.

[0143] A deep learning model was established using manually annotated datasets to extract time series and image features respectively. Time series information (real value or predicted value) was introduced using two different strategies, and a planned sampling training method was used to simulate and compare the calorific value prediction performance under different scenarios.

[0144] The test samples are input into the model, and the root mean square error (RMSE), mean absolute error (MAE), and R-squared value are used as quantitative evaluation indicators; a qualitative explanation of the model decision is given in the form of visual images.

[0145] In the following description, a number of different aspects of the present invention will be described. However, for those of ordinary skill in the art, the present invention may be implemented using only some or all of the structures or processes of the present invention. For clarity of explanation, specific numbers, configurations, and sequences are described, but it is clear that the present invention can be implemented without these specific details. In other cases, in order not to confuse the present invention, some well-known methods will not be described in detail.

[0146] Based on the monitoring video of the garbage storage pool and the corresponding calorific value data after incineration (calculated by the steam energy generated by the furnace), the image information (224×224 pixels) of the garbage surface just below the grabbing action was intercepted by manual annotation as the garbage image entering the furnace, and the corresponding time, furnace number, weighing and calorific value data (based on the garbage transportation process, the average calorific value 40-60 minutes after delivery) was used as the time series calorific value label. After screening, the data set (such as Figure 2 shown).

[0147] Data cleaning methods are used on the data set to remove samples with abnormal heat values, and smoothing denoising, histogram equalization and other methods are used to improve the quality of the image. In addition, the number of samples is expanded through random rotation and scaling.

[0148] The prediction model mainly adopts a multi-module fusion method based on deep learning, which consists of three modules (such as Figure 3 shown):

[0149] The long short-term memory model (LSTM) is used to extract features from the time series heat value data. The framework uses a 4-layer LSTM, with a hidden layer size of 64 and an input layer size of 2, and outputs the features of the last time step. Two strategies are used to introduce time series information: the first is t-LSTM, which means that the real heat value from the previous time step is used as the input for the next step, simulating an ideal scenario; the second is p-LSTM, which means that the predicted heat value from the previous time step is used as the input for the next step, simulating a real scenario.

[0150] During the training process, in order to alleviate the performance degradation problem that may be caused by error accumulation in the p-LSTM strategy, the present invention adopts a planned sampling technique. Specifically, a linear decay schedule is used to regulate the probability p(t), which determines the possibility of using the predicted value from the previous time step instead of the true value as input. Initially, the probability is set to p(0) = 1.0, and gradually decreases by 0.005 in each epoch until it reaches a minimum value of p(t-final) = 0.5. Through this gradual control transition, the model can gradually simulate and adapt to actual recursive prediction scenarios, thereby improving its robustness.

[0151] In view of the high-dimensional and complex features of the garbage images entering the furnace, the present invention uses two different deep learning models for feature extraction to provide richer and more accurate visual information for subsequent calorific value prediction. The first is the DenseNet network, which can avoid possible gradient vanishing and feature loss problems through dense connections between layers, and can fully extract and fuse image features at different levels; the second is the Transformer model, which associates the features of different regions in the garbage image under the background of complex features and effectively extracts representative global features.

[0152] Since the combined features of the upstream output contain high-dimensional image features, it is difficult to directly use complex patterns as the input of the regression model. Therefore, this module adopts the variational autoencoder model (VAE) by improving the encoder and decoder The structure compresses high-dimensional features into a low-dimensional latent space z = μ(x) + σ(x) ⊙∈, and then comprehensively processes the combined features. This improvement can more effectively capture representative features, thereby improving the prediction performance of downstream regression tasks. During the training process, the minimization of reconstruction error and KL divergence loss are incorporated into the loss function to optimize the model learning process.

[0153] The present invention uses multiple quantitative evaluation indicators such as root mean square error (RMSE), mean absolute error (MAE), and R-squared value to comprehensively measure the performance of the model in the calorific value prediction task. RMSE is used to measure the difference between the predicted value and the actual value. The smaller the value, the lower the model error; MAE is used to represent the average prediction error, reflecting the stability and overall error level of the model; R-squared value is used to evaluate the explanatory power of the model, indicating the correlation between the predicted result and the actual result. The closer the value is to 1, the better the model fitting effect.

[0154] The present invention combines the class activation mapping (GradCAM++) technology to display the decision-making process of the model in a more intuitive way. In the present invention, GradCAM++ is applied to the analysis of the visual feature extraction module to display the key areas of interest of the model when processing the images of the garbage entering the furnace, helping users understand the decision basis of the model. Through this visualization method, the prediction results of the model can be qualitatively analyzed, and the model's areas of interest in the image can be intuitively displayed to verify whether the image features it focuses on are highly correlated with the calorific value of incineration.

[0155] In addition, the reliability and interpretability of the model can be further verified by combining the above quantitative evaluation indicators with the visualization results. By comparing the visualization images under different time steps or different input strategies, users can identify the sensitivity of the model to the input information and the prediction behavior under different conditions to support the practical application of intelligent control of the calorific value of waste incineration.

[0156] The embodiment adopts the method proposed by the present invention to analyze randomly selected test samples. Figure 2 Images and time-series thermal value labels of some test samples are shown.

[0157] The proposed multi-module fusion learning framework is used to process the time series heat value labels and images. For 200 test samples, the quantitative evaluation index results obtained by the experiment are shown in Table 1, and the corresponding prediction result scatter plot is shown in Figure 3 As shown in the figure, under two different strategies of the temporal feature extraction module, the performance of the proposed method is significantly improved compared with the benchmark model. When the visual feature extraction module adopts the DenseNet161 network, the prediction accuracy of the model is the best (RMSE is 138.89, MAE is 109.10, R 2 is 0.787), indicating that the model proposed in the present invention has excellent accuracy and robustness in the calorific value prediction task.

[0158]

[0159] Table 1: Quantitative metrics for model performance evaluation

[0160] also, Figure 4 The visualization analysis results of some sample samples are shown. The thermal map generated by the GradCAM++ method clearly shows that the model proposed in this invention can focus on key areas related to high calorific value features when processing the images of the garbage entering the furnace. These areas usually have a specific visual appearance and can reflect the typical characteristics of high calorific value garbage. In contrast, the model pays significantly less attention to non-relevant areas (such as the walls of the garbage storage pool and the garbage grab area). The visualization analysis results further verify the effectiveness and reliability of the model in distinguishing calorific value-related and irrelevant features.

[0161] In summary, the method proposed in this invention verifies its high interpretability and reliability in practical applications through quantitative evaluation indicators and visual analysis, providing strong support for the intelligent control of waste incineration power plants. The overall flow chart of the image-based urban solid waste calorific value prediction method proposed in this patent is as follows: Figure 5 shown.

Claims

1. An image-based method for predicting the calorific value of municipal solid waste, characterized in that: include: Step S101, based on the monitoring video of the garbage storage pool, obtain the image of the garbage entering the furnace, collect the time series calorific value data for time tagging, take the image of the garbage entering the furnace in the monitoring video of the garbage storage pool as the starting time, and retrieve the average calorific value within 40 to 60 minutes after the starting time in the time series calorific value data as the time series calorific value label of the image of the garbage entering the furnace, so as to obtain the data set to be processed; Step S102, constructing a deep learning model, using the data set to be processed to extract time series features and image features in the data set to be processed, and adopting a planned sampling training strategy to simulate different application scenarios, using a supervised learning method to train the model, and gradually adjusting the model hyperparameters and structure to further improve the prediction accuracy; Step S103, after the training is completed, the test set is input into the model for performance evaluation, and the prediction effect of the model on the test set is evaluated by calculating the mean square error (MSE) and mean absolute error (MAE) indicators, and combining the model decision visualization technology; After the test is completed, the solid waste calorific value prediction model is finally obtained; Step S104, performing real-time analysis on the monitoring video of the garbage storage pool, extracting real-time images and time information data of the garbage entering the furnace, and inputting them into the solid waste calorific value prediction model to obtain the predicted calorific value of the solid waste.

2. The method for predicting the calorific value of urban solid waste based on images according to claim 1, characterized in that: The step S101 includes: Obtain real-time monitoring video streams from the monitoring system of the garbage storage pool, manually filter out garbage images entering the furnace from the monitoring video, and the garbage images represent garbage samples that are about to enter the incinerator. Time-stamp each garbage image entering the furnace and record the time point when the image was captured; Obtain the time series calorific value data synchronously recorded by the monitoring video, which includes the calorific value of garbage incineration at different time points; For each image of garbage entering the furnace, the time point when each image of garbage entering the furnace is captured is used as the starting time, and a time window is set. The range of the time window is 40 to 60 minutes from the starting time. The selection of the time window is set according to the transmission time of the garbage entering the furnace before incineration; In a certain time window, the corresponding calorific value data points are extracted from the time series calorific value data, and the average value of the calorific value data points is calculated as the time series calorific value label of the corresponding waste image entering the furnace; Pair each waste image with the corresponding time series calorific value label to form a data pair including the image and the label, and sort the pre-processed waste images and the corresponding time series calorific value labels to obtain the data set to be processed; The preprocessed data is divided into training set, validation set and test set. The training set and validation set are used for model training, and the test set is used to evaluate the performance of the model.

3. The method for predicting the calorific value of urban solid waste based on images according to claim 1, characterized in that: The step S102 includes: Constructing a multi-module fusion deep learning model including a temporal feature extraction module, a visual feature extraction module, and an encoding and decoding module; For time series data, the LSTM layer in the time series feature extraction module is used to capture the time dependency and sequence features in the data; For image data, the convolutional layer in the visual feature extraction module is used to extract the texture and shape features of the image; The spliced ​​temporal features and image features are input into the downstream encoding and decoding module for processing; The training samples are selected from the data set to be processed by planned sampling, and the deep learning model is trained using the selected training samples. During the training process, the prediction value is calculated by forward propagation, and the model parameters are updated by back propagation to minimize the loss function. Set appropriate training rounds, learning rate, and batch size hyperparameters, and monitor the loss and accuracy indicators during the training process to evaluate the training effect of the model. Adjust the model hyperparameters or optimization strategy based on the performance on the validation set, and repeat the training and validation process until the performance of the model on the validation set meets the expected requirements. When the performance indicators of the validation set are stable and meet the requirements, save the trained model.

4. The method for predicting the calorific value of urban solid waste based on images according to claim 1, characterized in that: The step S103 includes: After the training is completed, the image data and time data in the test set are input into the trained model to independently evaluate the generalization performance of the model. During the test, the mean square error (MSE), root mean square error (RMSE) and indicators on the test set are calculated. The test results are used as the final evaluation basis for the model performance; The class activation map visualization technology is used to display the areas that the model focuses on when processing the images of garbage entering the furnace. The prediction results of the model are further qualitatively analyzed to verify whether the image features of the areas of concern are highly correlated with the calorific value of incineration.

5. The method for predicting the calorific value of urban solid waste based on images according to claim 1, characterized in that: The step S104 includes: Conduct real-time analysis on the monitoring video of the garbage storage pool, capture the image data of the garbage entering the furnace, integrate the time series calorific value data of the previous time step and the real-time image data of the garbage entering the furnace, substitute them into the solid waste calorific value prediction model, trigger the solid waste calorific value prediction model to make a prediction, and the solid waste calorific value prediction model will output the predicted calorific value of the solid waste in real time based on the input data, display the predicted calorific value on the interface of the monitoring system, record it in the log file and send it to the control system synchronously.

Images