A sludge moisture content detection method based on convolutional autoencoder and random forest algorithm

Abstract

This application provides a method for detecting sludge moisture content based on convolutional autoencoders and random forest algorithms, comprising: acquiring sludge samples and their images; detecting the moisture content of the sludge samples; and constructing a sludge sample image database; performing feature extraction and noise reduction processing on the images in the sludge sample image database based on a convolutional autoencoder to obtain a sample dataset of the sludge sample images; constructing a sludge moisture content detection model based on the sample dataset using a random forest algorithm; and detecting the moisture content of new sludge samples based on the sludge moisture content detection model. This application constructs a wastewater moisture content detection method based on convolutional autoencoders and random forest algorithms, which features fast detection speed, high accuracy, and flexible deployment on mobile devices; it also has low requirements for the professional expertise and experience of users, and compared with traditional experimental methods, it can quickly assess the moisture content of sludge samples.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of environmental protection, and relates to a sludge moisture content detection method, in particular to a sludge moisture content detection method based on a convolutional autoencoder and a random forest algorithm. BACKGROUND

[0002] With the improvement of the scale of urban sewage treatment in China, the municipal sludge output also increases year by year accordingly, and sludge has become a major problem restricting the urban water environment governance and protection work. Whether the sludge can be properly treated and disposed has gradually attracted widespread attention from the whole society. Common technical process routes in sludge treatment and disposal mainly include anaerobic digestion, aerobic composting, drying incineration and land use, etc. A series of treatment and disposal standards issued by the state all put forward regulations on the moisture content of sludge. The moisture content that is too high is not conducive to the operation of the subsequent treatment and disposal process, and also cannot meet the requirements of sludge reduction treatment and disposal.

[0003] The existing sludge moisture content measurement method mainly refers to the “Sludge Test Method for Urban Sewage Treatment Plant” (CJ / T 211-2005), which uses the method of heating and drying to evaporate the water in the sludge, and the sludge before and after drying is weighed to calculate the moisture content of the sludge. However, this method is relatively complex and tedious, and requires heating and drying, weighing and comparison, etc. It also needs professional technicians to perform in a laboratory equipped with certain hardware conditions, and the measurement process takes a long time, which leads to low detection efficiency and high labor intensity due to repetitive work, and is not conducive to rapid detection of sludge moisture content. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a sludge moisture content detection method based on a convolutional autoencoder and a random forest algorithm, which is used to solve the problems of low detection efficiency and high labor intensity due to the need for heating and drying, weighing and comparison, etc. for sludge moisture content detection, and the limitation of hardware conditions and long measurement time in the process of implementing the sludge moisture content detection technology of the prior art.

[0005] To achieve the above-mentioned purposes and other related purposes, in a first aspect, the present application provides a sludge moisture content detection method based on a convolutional autoencoder and a random forest algorithm, comprising the following steps: obtaining a sludge sample and a sludge sample image, detecting the moisture content of the sludge sample, and constructing a sludge sample image database; performing feature extraction and noise reduction processing on the sludge sample images in the sludge sample image database based on a convolutional autoencoder to obtain a sample data set of the sludge sample images; constructing a sludge moisture content detection model based on the random forest algorithm based on the sample data set; and detecting the moisture content of the new sludge sample based on the sludge moisture content detection model.

[0006] In one implementation of the first aspect, acquiring sludge samples and sludge sample images, detecting the moisture content of sludge samples, and constructing a sludge sample image database includes the following steps: acquiring sludge sample images with different moisture contents, and simultaneously detecting the moisture content of the sludge samples; preprocessing the sludge sample images to obtain grayscale image data, in order to construct a sludge sample image database to be detected.

[0007] In one implementation of the first aspect, preprocessing the sludge sample image includes: format conversion, pixel adjustment, and grayscale processing.

[0008] In one implementation of the first aspect, the calculation formula for grayscale processing is:

[0009]

[0010] Where Gray represents the grayscale value of the sludge sample grayscale image obtained after grayscale processing; R, G, and B represent the color values ​​of the three primary colors in the PNG format of the sludge sample image, respectively.

[0011] In one implementation of the first aspect, the convolutional autoencoder includes an encoder and a decoder; performing feature extraction and noise reduction processing on sludge sample images in the sludge sample image database based on the convolutional autoencoder to obtain a sample dataset of the sludge sample images includes the following steps: encoding the sludge sample images in the sludge sample image database based on the encoder to obtain feature-encoded data of the sludge sample images; decoding the feature-encoded data based on the decoder to obtain a reconstructed image matrix; and constructing a sample dataset of the sludge sample images based on the reconstructed image matrix.

[0012] In one implementation of the first aspect, the encoder includes several convolutional layers, batch regularization layers, pooling layers, flattening layers, and fully connected layers: the convolutional layers are connected to each other through the batch regularization layers and activation functions to extract image features; the pooling layers are connected to the convolutional layers to reduce the dimensionality of image features to generate image feature data; the flattening layers are used to flatten the image feature data; and the fully connected layers are used to generate image feature encoding data.

[0013] In one implementation of the first aspect, the decoder includes several convolutional layers, batch regularization layers, pooling layers, flattening layers, and fully connected layers; the decoder has the same internal architecture as the encoder, but the connection order is reversed.

[0014] In one implementation of the first aspect, the image feature encoding data is input into the decoder to generate a reconstructed image matrix that is completely consistent with the dimensional information in the sludge sample image database, thereby completing the denoising process of the sludge sample image.

[0015] In one implementation of the first aspect, the sample dataset includes a training dataset and a test dataset; the training dataset is used to train the sludge moisture content detection model; and the test dataset is used to evaluate the performance of the sludge moisture content detection model.

[0016] In one implementation of the first aspect, the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm further includes: unifying the sludge moisture content detection model with sludge sample image preprocessing, sludge sample image feature extraction and noise reduction models to obtain an encapsulated model; and inputting a new sludge sample image into the encapsulated model to obtain a predicted value of the new sludge sample moisture content.

[0017] Secondly, this application provides a sludge moisture content detection system based on a convolutional autoencoder and a random forest algorithm, comprising: an acquisition module for acquiring sludge sample images and constructing a sludge sample image database; a feature extraction module for performing feature extraction and noise reduction processing on the sludge sample images in the sludge sample image database based on a convolutional autoencoder to obtain a sample dataset of the sludge sample images; a prediction module for constructing a sludge moisture content detection model based on the sample dataset using a random forest algorithm; and a detection module for detecting the moisture content of new sludge samples based on the sludge moisture content detection model.

[0018] Finally, this application provides a sludge moisture content detection device based on convolutional autoencoder and random forest algorithm, comprising: a processor and a memory. The memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the sludge moisture content detection device based on convolutional autoencoder and random forest algorithm performs the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm.

[0019] As described above, the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention has the following beneficial effects:

[0020] (1) This application provides a sludge moisture content detection method based on convolutional autoencoder and random forest algorithm. Compared with traditional sludge moisture content detection technology, this invention can realize rapid analysis of the sample to be tested, avoid the dependence of traditional methods on laboratory hardware conditions and professional testing personnel capabilities, can be flexibly deployed, and can detect sludge moisture content at low cost and high efficiency.

[0021] (2) This application fully considers the influence of different photographing equipment and shooting conditions on the image of the sample to be tested, and proposes a grayscale-based preprocessing method. By performing format conversion, pixel adjustment and grayscale processing on the image of the sludge sample, the influence of image clarity, brightness, saturation and other factors on the characteristics related to sludge moisture content is reduced.

[0022] (3) This application uses a convolutional autoencoder to further extract key feature information from sludge sample images. Compared with directly using grayscale image matrices for model training and prediction, the reconstructed image matrix generated by the convolutional autoencoder can reduce the influence of noise signals in the image during model training, and further improve the training speed and accuracy of the model.

[0023] (4) This application uses the random forest regression algorithm to train the model. By using the average decision tree, the risk of overfitting can be reduced. In addition, the trained model is very stable. A small number of outliers in the dataset will not have much impact on the model. Moreover, the reconstructed image matrix generated by the convolutional autoencoder can reduce the noise signal in the samples in the dataset, which helps to reduce the risk of overfitting due to noise signal during the training of random forest. Attached Figure Description

[0024] Figure 1 The diagram shown is a flowchart of an embodiment of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0025] Figure 2 The diagram shows an application scenario of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention in one embodiment.

[0026] Figure 3A The diagram shown is a flowchart of step S11 in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0027] Figure 3B The images shown are original images of sludge samples with different moisture contents in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0028] Figure 3C The image shown is a grayscale image of the sludge sample after preprocessing, as described in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of this invention.

[0029] Figure 4 The diagram shown is a flowchart of step S12 in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0030] Figure 5 The diagram shows the structure of the sludge image feature extraction and noise reduction model based on a convolutional autoencoder according to the present invention.

[0031] Figure 6 The image shown is a reconstructed image of a sludge sample in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0032] Figure 7 The diagram shows the structure of the sludge moisture content detection model based on the random forest algorithm in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0033] Figure 8 The diagram shown is a schematic representation of the principle structure of the sludge moisture content detection system based on convolutional autoencoder and random forest algorithm of the present invention in one embodiment.

[0034] Figure 9 The diagram shown is a schematic representation of the principle structure of the sludge moisture content detection device based on convolutional autoencoder and random forest algorithm of the present invention in one embodiment.

[0035] Component designation explanation

[0036] 81 Acquisition Module

[0037] 82 Feature Extraction Module

[0038] 83 Prediction Module

[0039] 84 Detection Module

[0040] 91 processor

[0041] 92 Memory

[0042] Steps S11 to S14 Detailed Implementation

[0043] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0044] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0045] The sludge moisture content detection method based on convolutional autoencoder and random forest algorithm provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0046] Please see Figure 1 and Figure 2 The figures show a flowchart of an embodiment of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention, and an application scenario diagram of an embodiment of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention. Figure 1 and Figure 2 As shown in the figure, this embodiment provides a method for detecting sludge moisture content based on convolutional autoencoder and random forest algorithm.

[0047] The sludge moisture content detection method based on convolutional autoencoder and random forest algorithm specifically includes the following steps:

[0048] S11: Acquire sludge samples and images, detect the moisture content of the sludge samples, and construct a sludge sample image database. (See also...) Figure 3A The diagram shows a flowchart of step S11 in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention. Figure 3A As shown, step S11 includes the following steps:

[0049] S111, detect the moisture content of sludge samples and acquire images of sludge samples with different moisture contents.

[0050] Please see Figure 3B The image shows the original images of sludge samples with different moisture contents in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0051] In this embodiment, the moisture content of the sludge sample is detected using a heating and weighing method; that is, the sludge sample is heated, and the weight of water lost during the heating process is measured to calculate the sludge moisture content. Simultaneously, images of the sludge sample are taken using a portable photographic device, such as a mobile phone camera.

[0052] Specifically, sludge samples with different moisture contents are collected from wastewater or sludge treatment plants, and photographs are taken to obtain sludge sample images. The sludge samples are then sent to the plant's laboratory to test their moisture content, and the images are labeled based on the moisture content test results. The moisture content of the sludge samples ranges from 48% to 75%. Sludge with a moisture content of 48% has a higher degree of drying, while sludge with a moisture content of 75% is freshly dewatered sludge produced in centrifuges or belt dewatering machines.

[0053] It is important to note that when photographing sludge samples, the image should be taken from 0.5m-1m above the surface of the sludge to be tested or sampled using a portable photographic device (such as a mobile phone camera or digital camera). Before taking the photo, it should be checked whether the sludge has been stored for a long time, causing surface moisture evaporation and resulting in a surface moisture content lower than the actual moisture content. For newly produced sludge from drying or dewatering equipment, samples and images can be obtained directly. However, for sludge that has been stored or aged for a long time, the top 1cm-2cm of relatively dry sludge needs to be removed before sampling and photographing the interior to obtain the final sludge sample image.

[0054] S112, the sludge sample image is preprocessed to obtain grayscale image data, thereby constructing a database of sludge sample images to be detected. Please refer to [link / reference]. Figure 3C The image shown is a grayscale image of the sludge sample after preprocessing of the original image in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0055] To enhance the accuracy of the model and reduce the risk of overfitting, sludge sample images need to be preprocessed.

[0056] In this embodiment, the preprocessing of the sludge sample image includes: format conversion, pixel adjustment, and grayscale processing. Specifically, the original sludge sample image matrix data is converted into a PNG format image, the image pixels are adjusted to 800*600, and then a grayscale algorithm is used to convert the sludge sample image into a grayscale image.

[0057] Specifically, the format conversion of sludge sample images is necessary because the formats of sludge sample images acquired by different devices may vary significantly. Therefore, it is necessary to convert sludge sample images of different formats to a unified PNG format.

[0058] The format conversion of sludge sample images is primarily due to the large number of images in the database. Therefore, tools from the PIL image processing library in Python can be used for batch processing, adjusting the image format to PNG and the image resolution to 800*600 pixels.

[0059] Then, the sludge sample images were converted to grayscale using a grayscale calculation formula. The grayscale calculation formula is as follows:

[0060]

[0061] Where Gray represents the grayscale value of the sludge sample grayscale image obtained after grayscale processing; R, G, and B represent the color values ​​of the three primary colors in the PNG format of the sludge sample image, respectively.

[0062] Finally, a sludge sample image database was constructed based on the obtained grayscale images of the sludge samples.

[0063] S12, feature extraction and noise reduction processing are performed on the sludge sample images in the sludge sample image database based on a convolutional autoencoder to obtain a sample dataset of the sludge sample images. (See also...) Figure 4 The diagram shows a flowchart of step S12 in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention. Figure 4 As shown, step S12 includes the following steps:

[0064] S121, The encoder is used to process the sludge sample image database to extract image features from the sludge sample images. (See also...) Figure 5 The diagram shows a schematic representation of the sludge image feature extraction and noise reduction model based on a convolutional autoencoder, as described in this invention. Figure 5 As shown, in this embodiment, the convolutional autoencoder includes an encoder and a decoder.

[0065] Obtaining a sample dataset of sludge sample images based on a convolutional autoencoder and the sludge sample image database includes the following steps: The encoder comprises several convolutional layers, a batch regularization layer, a pooling layer, a flattening layer, and a fully connected layer. The convolutional layers are connected to each other through the batch regularization layer and an activation function to extract image features; the pooling layer is connected to the convolutional layers to reduce the dimensionality of image features to generate image feature data; the flattening layer is used to flatten the image feature data; and the fully connected layer is used to generate image feature encoding data.

[0066] Specifically, a convolutional autoencoder is further used to extract important features from grayscale images and reduce the impact of noise signals such as illumination differences and vignetting that may be introduced during the photography process. Therefore, a convolutional autoencoder is built based on PyTorch, which includes an encoder and a decoder.

[0067] The encoder processes the grayscale image data of the sludge sample generated in step S11. The encoder contains 3 convolutional layers (Conv), 3 batch regularization layers (BatchNorm), 3 pooling layers (Pooling), 1 flattening layer (Flatten), and 1 fully connected layer (FC).

[0068] In the encoder, the matrix of grayscale values ​​from the grayscale image of the sludge sample is successively passed through a feature extraction layer consisting of three convolutional layers (Conv), a pooling layer, and a batch regularization layer (BatchNorm). This allows for better extraction of features related to the sludge's moisture content, such as particle size, flow regime, and texture. The generated feature image data is then processed through a flattening layer (Flatten) and a fully connected layer (FC) to generate image feature encoding data. Convolutional layers Conv1 to Conv3 are primarily used to extract specific features from the grayscale image, effectively capturing image features reflecting the sludge's moisture content. Each convolutional layer is connected by a batch regularization layer and a ReLU activation function. The batch regularization layers (BatchNorm1 to BatchNorm3) effectively improve the stability of the neural network structure within the convolutional autoencoder during training and enhance the extraction capability of convolutional layers Conv1 to Conv3 for moisture content-related features.

[0069] In this embodiment, a pooling layer is used after each convolutional layer of the encoder to reduce the input dimensionality and the risk of overfitting. The feature image data generated after the Pooling3 layer is then flattened by the Flatten layer, and finally passed through a fully connected layer (FC) to generate image feature encoding data.

[0070] S122, the feature encoding data is flattened based on the decoder to obtain a reconstructed image matrix.

[0071] In this embodiment, the decoder has the same architecture as the encoder, but with the connection order reversed. The decoder's input is the image feature encoding data ultimately generated by the encoder. The decoder then generates a reconstructed image matrix based on the input, which is completely identical to the dimensional information of the original grayscale image matrix in the image database. Compared to the original grayscale image matrix, this reconstructed image data has noise removed while retaining key features.

[0072] Specifically, in the decoder, based on the encoding generated by the encoder, a reconstructed image with the exact same dimensions as the grayscale image is generated following the reverse process. That is, the image feature encoding data is input into the decoder to generate a reconstructed image matrix with dimensional information completely consistent with the sludge sample image database, thus completing feature extraction and denoising of the sludge sample image. This process is achieved through a sludge image feature extraction and denoising model trained based on a convolutional autoencoder.

[0073] For the sludge image feature extraction and denoising model, the mean square error (MSE) of the original grayscale image matrix and the reconstructed image matrix is ​​used as the loss function during the training process, and the Adam optimizer is used in the optimization process.

[0074] S123, Based on the reconstructed image matrix, construct a sample dataset of the sludge sample images. (See also...) Figure 6 The image shown is a reconstructed image of a sludge sample in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0075] In this embodiment, the sludge image feature extraction and denoising model processes the original sludge sample image database to generate a reconstructed image database. Each reconstructed image matrix in this database is combined with the corresponding moisture content value of the sludge sample to form a sample. The sludge sample image database can be transformed into a sample dataset through this model. The sample dataset includes a training dataset and a test dataset.

[0076] S13, Obtain a sludge moisture content detection model based on the aforementioned sample dataset. (See also...) Figure 7 The diagram shows a schematic of the sludge moisture content detection model based on the random forest algorithm in the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm of the present invention.

[0077] In this embodiment, a random forest regression algorithm is used to build a sludge moisture content detection model based on a sample dataset. The sample dataset includes a training dataset and a test dataset; the training dataset is used to train the sludge moisture content detection model; and the test dataset is used to evaluate the performance of the sludge moisture content detection model.

[0078] Specifically, the generated reconstructed images, combined with sludge moisture content labels, constitute a sample dataset. Using a random forest regression algorithm, a sludge moisture content detection model can be trained based on this dataset. To verify the model's performance on new samples, the sample dataset is divided into a training dataset and a test dataset in a 7:3 ratio before training. The training dataset is used for model training, and the test dataset is used to test the model's accuracy on new data. To ensure effective training, the number of samples in the training dataset is kept above 100. The number of decision trees in the random forest algorithm is related to the number of samples in the training dataset. When the number of samples in the training dataset is around 100-200, the number of decision trees is 100. When the number of samples in the training dataset is above 200, the number of decision trees can be set to 50% of the number of samples, but should not exceed 250 to avoid excessively long model training time. Finally, the sludge sample moisture content detection model is obtained.

[0079] For example: Please continue reading Figure 7 The reconstructed image is input into different decision trees. When input into decision tree 1, a predicted value is obtained; when input into decision tree 2, another predicted value is obtained. This process continues until several different predicted values ​​are obtained. These predicted values ​​are then subjected to numerical processing (e.g., calculating the mean or median, etc., and this method is not limited to a single method or combination thereof) to ultimately obtain a moisture content data that closely approximates the true value.

[0080] S14, The new sludge moisture content is detected based on the sludge moisture content detection model.

[0081] Based on the sludge moisture content detection model, it is encapsulated in a unified manner with the sludge sample image preprocessing, sludge sample image feature extraction and noise reduction model to obtain the encapsulated model; the new sludge sample image is input into the encapsulated model to obtain the predicted value of the new sludge sample moisture content.

[0082] Specifically, after the sludge moisture content detection model is trained, it can be packaged together with the preprocessing model, feature extraction and denoising model, and sludge moisture content prediction model, and flexibly deployed on mobile or web platforms. When using it, a new sludge sample is acquired and photographed; the resulting image is input into the packaged model to obtain a predicted value for the moisture content of the current sludge sample.

[0083] It should be noted that the staff developed an image preprocessing model based on the PIL image processing library in Python, built a convolutional autoencoder based on the PyTorch library to generate a feature extraction and noise reduction model, and built a sludge moisture content prediction model based on the Scikit-learn library.

[0084] The sludge moisture content detection method based on convolutional autoencoder and random forest algorithm provided in this application has the characteristics of fast detection speed, high detection accuracy, and flexible deployment on mobile devices. It has low requirements for the professional and experience requirements of users and does not require related experimental instruments. Compared with traditional experimental methods, it can quickly assess the moisture content of sludge samples.

[0085] The scope of protection for the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0086] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following... Figure 1 The method for detecting sludge moisture content based on convolutional autoencoder and random forest algorithm.

[0087] At any possible level of technical detail, this application can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0088] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0089] The computer-readable program described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards these instructions to a computer-readable storage medium in the respective computing / processing device. The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as "C" or similar programming languages. Computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of this application.

[0090] This application also provides a sludge moisture content detection system based on convolutional autoencoder and random forest algorithm. The sludge moisture content detection system based on convolutional autoencoder and random forest algorithm can implement the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm described in this application. However, the implementation device of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm described in this application includes, but is not limited to, the structure of the sludge moisture content detection system based on convolutional autoencoder and random forest algorithm listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of this application are included within the protection scope of this application.

[0091] The following will describe in detail the sludge moisture content detection system based on convolutional autoencoder and random forest algorithm provided in this embodiment, with reference to the illustrations.

[0092] This embodiment provides a sludge moisture content detection system based on a convolutional autoencoder and a random forest algorithm, including:

[0093] Please see Figure 8 The diagram shows a schematic representation of the principle structure of the sludge moisture content detection system based on convolutional autoencoder and random forest algorithm in one embodiment of the present invention. Figure 8 As shown, the sludge moisture content detection system based on convolutional autoencoder and random forest algorithm includes: acquisition module 81, model building module 82 and detection module 83.

[0094] The acquisition module 81 is used to acquire sludge samples and their images, detect the moisture content of sludge samples, and construct a sludge sample image database.

[0095] The moisture content of sludge samples was detected, and images of sludge samples with different moisture contents were acquired.

[0096] In this embodiment, the moisture content of the sludge sample is detected using a heating and weighing method; that is, the moisture content of the sludge is calculated by heating the sludge sample and measuring the weight of water lost during the heating process. Simultaneously, images of the sludge sample are acquired using a portable photographic device, such as a mobile phone camera.

[0097] Specifically, sludge samples with different moisture contents are collected from wastewater or sludge treatment plants, and photographs are taken to obtain sludge sample images. The sludge samples are then sent to the plant's laboratory to test their moisture content, and the images are labeled based on the moisture content test results. The moisture content of the sludge samples ranges from 48% to 75%. Sludge with a moisture content of 48% is considered to have a higher degree of drying, while sludge with a moisture content of 75% is freshly dewatered sludge produced in centrifuges or belt dewatering machines. For freshly produced sludge from drying or dewatering equipment, samples and images can be directly obtained through sampling and photography. However, for sludge that has been stored or aged for a long time, the top 1-2 cm of relatively dry sludge needs to be removed before sampling and photographing the interior to obtain sludge sample images.

[0098] Meanwhile, in order to enhance the accuracy of the model and reduce the risk of overfitting, the sludge sample images were preprocessed to obtain grayscale image data in order to construct a database of sludge sample images to be detected.

[0099] In this embodiment, the preprocessing of the sludge sample image includes: format conversion, pixel adjustment, and grayscale processing. Specifically, the original sludge sample image matrix data is converted into a PNG format image, the image pixels are adjusted to 800*600, and then a grayscale algorithm is used to convert the sludge sample image into a grayscale image.

[0100] Specifically, the format conversion of sludge sample images is necessary because the formats of sludge sample images acquired by different devices may vary significantly. Therefore, it is necessary to convert sludge sample images of different formats to a unified PNG format.

[0101] The format conversion of sludge sample images is primarily due to the large number of images in the database. Therefore, tools from the PIL image processing library in Python can be used for batch processing, adjusting the image format to PNG and the image resolution to 800*600 pixels.

[0102] Then, the color values ​​of the three primary colors in the PNG format of the sludge sample image are calculated using the grayscale calculation formula.

[0103] Finally, a sludge sample image database was constructed based on the obtained grayscale images of the sludge samples.

[0104] The feature extraction module 82 is connected to the acquisition module 81 and is used to perform feature extraction and noise reduction processing on the sludge sample images in the sludge sample image database based on a convolutional autoencoder, so as to obtain a sample dataset of the sludge sample images.

[0105] First, a sample dataset of sludge sample images is obtained based on the convolutional autoencoder and the sludge sample image database, resulting in a sludge sample image feature extraction and noise reduction model.

[0106] In this embodiment, the convolutional autoencoder includes an encoder and a decoder. The encoder processes the sludge sample image database to extract image features from the sludge sample images; the image features are then flattened to generate image feature encoded data.

[0107] The encoder consists of several convolutional layers, batch regularization layers, pooling layers, flattening layers, and fully connected layers. The convolutional layers are connected via batch regularization layers and activation functions to extract image features; the pooling layers are connected to the convolutional layers to reduce the dimensionality of image features to generate image feature data; the flattening layers flatten the image feature data; and the fully connected layers generate the encoded image feature data.

[0108] Specifically, a convolutional autoencoder is further used to extract important features from grayscale images and reduce the impact of noise signals such as illumination differences and vignetting that may be introduced during the photography process. Therefore, a convolutional autoencoder is built based on PyTorch.

[0109] The encoder processes the grayscale image data of the sludge sample generated in step S11. The encoder contains 3 convolutional layers (Conv), 3 batch regularization layers (BatchNorm), 3 pooling layers (Pooling), 1 flattening layer (Flatten), and 1 fully connected layer (FC).

[0110] In the encoder, the matrix of grayscale values ​​from the grayscale image of the sludge sample is successively passed through a feature extraction layer consisting of three convolutional layers (Conv), pooling layers, and a batch regularization layer (BatchNorm). This allows for better extraction of features related to sludge moisture content, such as particle size, flow regime, and texture. The generated feature image data is then processed by a flattening layer (Flatten) and a fully connected layer (FC) to produce image feature encoding data. Convolutional layers Conv1 to Conv3 are primarily used to extract specific features from the grayscale image, effectively capturing image features reflecting sludge moisture content. Each convolutional layer is connected by a batch regularization layer and a ReLU activation function. The batch regularization layers (BatchNorm1 to BatchNorm3) effectively improve the stability of the neural network structure within the convolutional autoencoder during training and enhance the extraction capability of convolutional layers Conv1 to Conv3 for moisture content-related features. A pooling layer is used after each convolutional layer in the encoder to reduce input dimensionality and the risk of overfitting. The feature image data generated after the pooling layer (Pooling3) is then flattened by the flatten layer (Flatten), and finally passed through a fully connected layer (FC) to generate image feature encoding data.

[0111] The image feature encoding data is flattened based on the decoder to obtain the reconstructed image matrix.

[0112] In this embodiment, the decoder has the same architecture as the encoder, but with the connection order reversed. The decoder's input is the image feature encoding data ultimately generated by the encoder. The decoder then generates a reconstructed image matrix based on the input, which is completely identical to the dimensional information of the original grayscale image matrix in the image database. Compared to the original grayscale image matrix, this reconstructed image data has noise removed while retaining key features.

[0113] Specifically, in the decoder, based on the encoding generated by the encoder, a reconstructed image with the exact same dimensions as the grayscale image is generated following the reverse process. That is, the image feature encoding data is input into the decoder to generate a reconstructed image matrix with dimensional information completely consistent with the sludge sample image database, thus completing feature extraction and denoising of the sludge sample image. This process is achieved through a sludge image feature extraction and denoising model trained based on a convolutional autoencoder.

[0114] For the sludge image feature extraction and denoising model, the mean square error (MSE) of the original grayscale image matrix and the reconstructed image matrix is ​​used as the loss function during the training process, and the Adam optimizer is used in the optimization process.

[0115] A sample dataset of sludge sample images is constructed based on the reconstructed image matrix.

[0116] In this embodiment, the sludge image feature extraction and denoising model processes the original sludge sample image database to generate a reconstructed image database. Each reconstructed image matrix in this database is combined with the corresponding moisture content value of the sludge sample to form a sample. The sludge sample image database can be transformed into a sample dataset through this model. The sample dataset includes a training dataset and a test dataset.

[0117] Prediction module 83 is used to obtain a sludge moisture content detection model based on the sample dataset.

[0118] In this embodiment, a random forest regression algorithm is used to build a sludge moisture content detection model based on a sample dataset. The sample dataset includes a training dataset and a test dataset; the training dataset is used to train the sludge moisture content detection model; and the test dataset is used to evaluate the performance of the sludge moisture content detection model.

[0119] Specifically, the generated reconstructed images, combined with sludge moisture content labels, constitute a sample dataset. Using a random forest regression algorithm, a sludge moisture content detection model can be trained based on this dataset. To verify the model's performance on new samples, the sample dataset is divided into a training dataset and a test dataset in a 7:3 ratio before training. The training dataset is used for model training, and the test dataset is used to test the model's accuracy on new data. To ensure effective training, the number of samples in the training dataset is kept above 100. The number of decision trees in the random forest algorithm is related to the number of samples in the training dataset. When the number of samples in the training dataset is around 100-200, the number of decision trees is 100. When the number of samples in the training dataset is above 200, the number of decision trees can be set to 50% of the number of samples, but should not exceed 250 to avoid excessively long model training time. Finally, the sludge sample moisture content detection model is obtained.

[0120] The detection module 84 is used to detect the moisture content of new sludge samples based on the sludge moisture content detection model.

[0121] Based on the sludge moisture content detection model, it is encapsulated in a unified manner with the sludge sample image preprocessing, sludge sample image feature extraction and noise reduction model to obtain the encapsulated model; the new sludge sample image is input into the encapsulated model to obtain the predicted value of the new sludge sample moisture content.

[0122] Specifically, after the sludge moisture content detection model is trained, it can be packaged together with the preprocessing model, feature extraction and denoising model, and sludge moisture content prediction model, and flexibly deployed on mobile or web platforms. When using it, a new sludge sample is acquired and photographed; the resulting image is input into the packaged model to obtain a predicted value for the moisture content of the current sludge sample.

[0123] It should be noted that the staff developed an image preprocessing model based on the PIL image processing library in Python, built a convolutional autoencoder based on the PyTorch library to generate a feature extraction and noise reduction model, and built a sludge moisture content prediction model based on the Scikit-learn library.

[0124] A sludge moisture content detection model based on convolutional autoencoders and random forest algorithms has been developed. This system improves the detection speed and accuracy of sludge moisture content. Furthermore, it can be flexibly deployed on mobile devices, requires minimal user expertise and experience, eliminates the need for specialized experimental equipment, and enables rapid assessment of sludge sample moisture content.

[0125] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, module x can be a separate processing element, or it can be integrated into a chip within the system. Alternatively, it can be stored as program code in the system's memory, and its function can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0126] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0127] Please see Figure 9 The diagram shows a schematic representation of the principle structure of the sludge moisture content detection device based on a convolutional autoencoder and random forest algorithm in one embodiment of the present invention. Figure 9 As shown, this embodiment provides a sludge moisture content detection device based on convolutional autoencoder and random forest algorithm. The sludge moisture content detection device based on convolutional autoencoder and random forest algorithm includes: a processor 91 and a memory 92; the memory 92 is used to store computer programs; the processor 91 is connected to the memory 92 and is used to execute the computer program stored in the memory 92, so that the sludge moisture content detection device based on convolutional autoencoder and random forest algorithm performs the various steps of the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm as described above.

[0128] Preferably, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0129] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0130] In summary, the sludge moisture content detection method based on convolutional autoencoder and random forest algorithm provided in this application has the following beneficial effects:

[0131] The sludge moisture content detection method based on convolutional autoencoders and random forest algorithms provided in this application enables rapid analysis of samples, avoiding the dependence on laboratory hardware and professional testing personnel required by traditional methods. It allows for flexible deployment and enables low-cost, high-efficiency detection of sludge moisture content. While fully considering the impact of different imaging devices and shooting conditions on the images of the samples, it also reduces the influence of noise signals in the images during model training, further improving the training speed and accuracy of the model. Furthermore, it further reduces the risk of model overfitting. The sludge moisture content detection model based on convolutional autoencoders and random forest algorithms has a simple structure and features fast detection speed, high detection accuracy, and flexible deployment on mobile devices. It has low requirements for the professional expertise and experience of users, requires no related experimental instruments, and can quickly assess the moisture content of sludge samples.

[0132] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A sludge moisture content detection method based on a convolutional autoencoder and a random forest algorithm, characterized by, The method comprises the following steps: Obtaining a sludge sample and a sludge sample image, detecting the water content of the sludge sample, and constructing a sludge sample image database; including: obtaining sludge sample images with different water contents, and simultaneously detecting the water content of the sludge sample; preprocessing the sludge sample image to obtain gray image data to construct a sludge sample image database to be detected; wherein the preprocessing includes: format conversion, pixel adjustment and grayscale processing; the calculation formula of the grayscale processing is: ， wherein, represents a gray value of a gray scale image of the sludge sample obtained after the gray scale processing; respectively represent color values of three primary colors in the PNG format of the sludge sample image; Based on the convolutional autoencoder, the sludge sample image in the sludge sample image database is subjected to feature extraction and noise reduction processing to obtain a sample data set of the sludge sample image; the convolutional autoencoder includes an encoder and a decoder; including: based on the encoder, the sludge sample image in the sludge sample image database is subjected to encoding processing to obtain feature encoding data of the sludge sample image; based on the decoder, the feature encoding data is subjected to decoding processing to obtain a reconstructed image matrix; based on the reconstructed image matrix, a sample data set of the sludge sample image is constructed; wherein, The encoder includes a plurality of convolutional layers, batch normalization layers, pooling layers, flattening layers and fully connected layers; wherein the convolutional layers are connected through the batch normalization layers and activation functions, and are used for extracting image features; the pooling layer is connected with the convolutional layer, and is used for reducing the dimension of the image feature to generate image feature data; the flattening layer is used for flattening the image feature data; the fully connected layer is used for generating image feature encoding data; The decoder includes a plurality of convolutional layers, batch normalization layers, pooling layers, flattening layers and fully connected layers; the decoder has the same internal architecture as the encoder, but the connection order is reversed; Based on the sample data set, a random forest algorithm is used to construct a sludge water content detection model; Based on the sludge water content detection model, the water content of the new sludge sample is detected; Based on the sludge water content detection model, the sludge sample image preprocessing, sludge sample image feature extraction and noise reduction model are uniformly packaged to obtain a packaged model; The new sludge sample image is input into the packaged model to obtain a predicted value of the water content of the new sludge sample. 2.The sludge moisture content detection method based on convolutional auto-encoder and random forest algorithm according to claim 1, characterized in that, The image feature encoding data is input into the decoder to generate a reconstructed image matrix consistent with the dimension information in the sludge sample image database, and the denoising process of the sludge sample image is completed. 3.The sludge moisture content detection method based on convolutional auto-encoder and random forest algorithm according to claim 1, characterized in that, The sample data set includes a training data set and a test data set; the training data set is used to train the sludge water content detection model; the test data set is used to evaluate the performance of the sludge water content detection model.

4. A system for performing the sludge moisture content detection method based on a convolutional autoencoder and random forest algorithm according to any one of claims 1 to 3, characterized in that, It comprises: The acquisition module is used for acquiring sludge samples and images thereof, detecting water content of the sludge samples, and constructing a sludge sample image database; comprising: acquiring sludge sample images with different water contents, and simultaneously detecting water content of the sludge samples; preprocessing the sludge sample images to obtain gray image data, so as to construct a sludge sample image database to be detected; wherein the preprocessing comprises: format conversion, pixel adjustment, and gray scale processing; and a calculation formula of the gray scale processing is: ， wherein, represents a gray value of a gray scale image of the sludge sample obtained after the gray scale processing; respectively represent color values of three primary colors in the PNG format of the sludge sample image; The feature extraction module is used for performing feature extraction and noise reduction processing on the sludge sample images in the sludge sample image database based on a convolutional autoencoder, so as to obtain a sample data set of the sludge sample images; the convolutional autoencoder comprises: an encoder and a decoder; comprising: performing encoding processing on the sludge sample images in the sludge sample image database based on the encoder, so as to obtain feature encoding data of the sludge sample images; performing decoding processing on the feature encoding data based on the decoder, so as to obtain a reconstructed image matrix; and constructing the sample data set of the sludge sample images based on the reconstructed image matrix; wherein, The encoder comprises a plurality of convolutional layers, batch normalization layers, pooling layers, flattening layers, and fully connected layers; wherein the convolutional layers are connected through the batch normalization layers and activation functions, and are used for extracting image features; the pooling layers are connected with the convolutional layers, and are used for reducing the dimension of the image features to generate image feature data; the flattening layers are used for flattening the image feature data; and the fully connected layers are used for generating image feature encoding data; The decoder comprises a plurality of convolutional layers, batch normalization layers, pooling layers, flattening layers, and fully connected layers; the decoder has the same internal architecture as the encoder, but the connection order is reversed; The prediction module is used for constructing a sludge water content detection model based on the sample data set and using a random forest algorithm; The detection module is used for detecting water content of a new sludge sample based on the sludge water content detection model; based on the sludge water content detection model, a sludge sample image preprocessing model, a sludge sample image feature extraction and noise reduction model are uniformly packaged to obtain a packaged model; and the new sludge sample image is input into the packaged model to obtain a predicted value of water content of the new sludge sample.

5. A sludge moisture content detection device based on convolutional autoencoder and random forest algorithm, characterized in that, comprising: a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory, and is used for executing the computer program stored in the memory, so that the sludge water content detection device based on the convolutional autoencoder and the random forest algorithm executes the sludge water content detection method based on the convolutional autoencoder and the random forest algorithm in any one of claims 1 to 3.

Images