Fine aggregate moisture content detection method and device based on depth image and microwave spectrum

Abstract

The present invention relates to the technical field of concrete production, and specifically to a method and device for detecting moisture content of fine aggregate based on depth image and microwave spectrum, combining RGB-D imaging technology with microwave sensing technology to obtain material thickness curve data and microwave spectrum curve of fine aggregate; inputting material thickness curve data and microwave spectrum curve into a trained multimodal moisture content prediction model, the trained multimodal moisture content prediction model includes a material thickness feature extraction network, a microwave spectrum feature extraction network and a feature fusion network, the material thickness curve data passes through the material thickness feature extraction network to extract material thickness features, the microwave spectrum curve passes through the microwave spectrum feature extraction network to extract microwave spectrum features, the material thickness features and the microwave spectrum features are input into a feature fusion network to predict the predicted value of moisture content. The present invention solves the problems of inaccurate detection of moisture content of fine aggregate and high delay in the prior art, effectively filters out material thickness fluctuations, and improves the accuracy and stability of moisture content measurement.

Description

Technical Field

[0001] The invention relates to the technical field of concrete production, and in particular to a method and a device for detecting moisture content of fine aggregate based on depth images and microwave spectrum. Background Art

[0002] Concrete is a composite material formed by mixing sand and gravel aggregate, cement, water, admixtures and additives in a certain proportion. With the continuous development of urban infrastructure and basic facilities, concrete has become the most widely used and largest man-made material in construction. In the process of concrete preparation, the determination of the moisture content of fine aggregate is very important, which directly affects the control of the water-cement ratio in the concrete mix. This affects the amount of cement, and indirectly affects the emission of carbon dioxide. Therefore, accurately measuring and controlling the moisture content of fine aggregate is not only crucial to ensuring the quality of building materials, but also of great significance to reducing carbon emissions.

[0003] At present, the methods for testing the moisture content of fine aggregate include drying method, infrared detection method, resistance method, capacitance method, neutron method and microwave method. Most concrete mixing plants still use the oven drying method to test the moisture content of fine aggregate. This method is time-consuming, but has the advantage of high measurement accuracy and can be used as a test standard for inspection. The moisture content formula is:

[0004] ;

[0005] In the formula, Indicates the mass of the sample after drying; Indicates the mass of the sample before drying.

[0006] For non-destructive testing of moisture content in fine aggregate, moisture detection sensors are often used. For example, the principle of the resistance method is to use the change in resistance or conductivity of the sample to be tested for measurement; the capacitance method is based on the relationship between the composite dielectric parameters and moisture content of the sample to be tested. When the moisture content in the sample fluctuates, the composite dielectric parameters will change, resulting in fluctuations in the capacitance value in the capacitor; the infrared method refers to a method of detecting the moisture content of the sample to be tested by using the difference in the infrared diffuse reflection spectrum absorbed by the sample to be tested, the correlation between the attenuation degree of the reflection of the infrared radiation sample to be tested and the moisture content of the sample. Microwave moisture content detection is a method of rapid moisture content determination that has emerged in recent years. The microwave method is used to measure moisture content, which has high sensitivity, fast speed, and can facilitate dynamic detection and real-time control. There are two main methods: transmission method and reflection method. The frequency of microwaves is between 30MHz and 300GHz. Under the condition of 20℃, the dielectric constant of water is about 80, and the dielectric constant of fine aggregate is about 2~5. The microwave method for measuring moisture content uses the principle that various parameters such as water's absorption and reflection of microwave energy or the resonant frequency of the microwave resonant cavity will change with the change of moisture content.

[0007] However, there are many problems with the existing fine aggregate moisture content measurement technology. The traditional fine aggregate moisture content detection method is measured by drying method, which has problems such as long detection time and sampling limitations, and cannot be applied to real-time detection of moisture content in production; including contact measurement methods such as resistance and capacitance methods, which can only detect the local material water distribution around the probe, resulting in large measurement errors and poor repeatability. In addition, the probe is prone to wear and tear after long-term use, which destroys the accuracy of moisture content detection; the accumulation thickness of fine aggregate in the production process is uneven, which is easy to affect the moisture content detection results. Material thickness fluctuations and deviations will not only affect the accuracy of moisture content measurement, but may also lead to waste of raw materials and increased energy consumption, further increasing carbon emissions.

[0008] In this context, the present invention proposes a method and device for detecting moisture content of fine aggregate based on depth image and microwave spectrum, so as to improve the accuracy and stability of moisture content measurement. Summary of the invention

[0009] The purpose of this application is to propose a method and device for detecting moisture content of fine aggregate based on depth image and microwave spectrum in response to the above-mentioned technical problems.

[0010] In a first aspect, the present invention provides a method for detecting moisture content of fine aggregate based on depth image and microwave spectrum, comprising the following steps:

[0011] An RGB-D depth camera is used to collect the material depth image of fine aggregate on the conveyor belt, and a multi-frequency microwave transmitting probe and a microwave receiving probe are used to collect the microwave signal of the fine aggregate at the same location on the conveyor belt. The material thickness of the fine aggregate at that location is calculated based on the material depth image, and the material thickness curve data is obtained. A microwave spectrum curve is generated based on the microwave signal.

[0012] Constructing and training a multimodal moisture content prediction model to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network;

[0013] The microwave spectrum curve and material thickness curve data are input into the trained multimodal moisture content prediction model. The microwave spectrum curve passes through the microwave spectrum feature extraction network to extract the microwave spectrum features. The material thickness curve data passes through the material thickness feature extraction network to extract the material thickness features. The microwave spectrum features and material thickness features are input into the feature fusion network to predict the moisture content.

[0014] Preferably, the material thickness of the fine aggregate at the location is calculated according to the material depth image to obtain a material thickness curve, which specifically includes:

[0015] An RGB-D depth camera is used to collect a depth image of an empty conveyor belt to obtain a belt depth image. The depth value of each pixel point in the belt depth image represents the actual distance between the camera and the surface of the conveyor belt.

[0016] The depth value of each pixel in the material depth image represents the actual distance between the camera and the fine aggregate surface. The difference between the depth value of the same pixel in the belt depth image and the depth value in the material depth image is calculated to obtain the material thickness data of the pixel.

[0017] Calculate the material thickness data of all pixels in the material depth image to obtain the material thickness curve data.

[0018] Preferably, the training data used in the training process of the multimodal moisture content prediction model is obtained by the following steps:

[0019] Draw a material thickness curve of fine aggregate on a pixel point-material thickness plane according to the material thickness curve data of fine aggregate;

[0020] According to a plurality of material thickness curves, the pixel point-material thickness plane is divided into a plurality of material thickness ranges, the area between every two material thickness curves is a material thickness range, a material thickness baseline is defined for each material thickness range, and a moisture content regression model is constructed and trained to obtain a trained moisture content regression model;

[0021] Calculate the envelope area of ​​the material thickness curve of fine aggregate and all material thickness baselines respectively, and select the material thickness baseline with the smallest envelope area and its corresponding trained moisture content regression model;

[0022] Input the microwave spectrum curve of the fine aggregate into the selected trained moisture content regression model, and output the moisture content label of the fine aggregate;

[0023] The training data is constructed based on the material thickness curve data, microwave spectrum curve and corresponding moisture content labels of fine aggregate.

[0024] Preferably, a moisture content regression model is constructed and trained to obtain a trained moisture content regression model, which specifically includes:

[0025] A moisture content regression model is constructed for each material thickness range. The moisture content regression model includes one of a random forest regression model and a support vector regression model. The microwave spectrum curve of the fine aggregate is input into the moisture content regression model to obtain the moisture content label of the fine aggregate.

[0026] Compare the moisture content label of the fine aggregate with the actual moisture content of the fine aggregate to obtain a comparison result;

[0027] The parameters of the moisture content regression model are adjusted according to the comparison results to obtain a trained moisture content regression model.

[0028] Preferably, the microwave spectrum feature extraction network includes a multi-core one-dimensional separable convolution module, a first separable convolution layer, a multi-head attention module, a second separable convolution layer and a first convolution layer connected in sequence, wherein the convolution kernel sizes of the first separable convolution layer and the second separable convolution layer are both , the convolution kernel size of the first convolutional layer is The microwave spectrum curve passes through the microwave spectrum feature extraction network, and the specific process of extracting the microwave spectrum features is as follows:

[0029] Input the microwave spectrum curve into a multi-core one-dimensional separable convolution module for multi-scale feature extraction. The multi-core one-dimensional separable convolution module includes a plurality of one-dimensional separable convolution layers with different convolution kernel sizes. The microwave spectrum curve is convolved through the plurality of one-dimensional separable convolution layers with different convolution kernel sizes to obtain a first multi-scale feature. After the first multi-scale feature is spliced ​​in the depth dimension, a second multi-scale feature is obtained.

[0030] The second multi-scale feature passes through the first separable convolution layer and the multi-head attention module in sequence to obtain the first global feature;

[0031] The first global feature is sequentially passed through the second separable convolution layer and the first convolution layer to obtain a microwave spectrum feature;

[0032] The material thickness feature extraction network includes five convolutional modules, a flattening layer, and three first fully connected layers connected in sequence. The material thickness curve data passes through the material thickness feature extraction network, and the specific process of extracting the material thickness feature is as follows:

[0033] The material thickness curve data is input into five convolution modules in sequence to obtain local features. Each convolution module includes a convolution kernel size of One-dimensional convolution layer, ReLU activation function layer and maximum pooling layer;

[0034] The local features are input into the flattening layer and flattened into a one-dimensional vector to obtain the flattened local features;

[0035] The flattened local features are sequentially integrated and extracted through three first fully connected layers to obtain material thickness features. The first fully connected layers all use the ReLU activation function.

[0036] The feature fusion network includes a summation module, a second fully connected layer, a third fully connected layer, a second convolutional layer, and a fourth fully connected layer connected in sequence, wherein the convolution kernel size of the second convolutional layer is The output dimension of the second and third fully connected layers is 2D. The microwave spectrum features and material thickness features are input into the feature fusion network to predict the specific process of obtaining the predicted value of moisture content as follows:

[0037] Input the microwave spectrum feature and the material thickness feature into a summing module, perform element-by-element summing operation at corresponding feature positions, and obtain a first mixed feature;

[0038] The first mixed feature is input into the second fully connected layer, and activated by linear transformation and nonlinear activation function to obtain the second mixed feature;

[0039] The second mixed feature is input into the third fully connected layer, and the first weight and the second weight are output. The first weight and the second weight are subjected to softmax normalization operation and then element-wise multiplied with the microwave spectrum feature and the material thickness feature to obtain weighted microwave spectrum feature and weighted material thickness feature. The weighted microwave spectrum feature and the weighted material thickness feature are summed to obtain a fusion feature.

[0040] The fused features pass through the second convolutional layer and the fourth fully connected layer in sequence to obtain the moisture content prediction value.

[0041] As a preferred method, the method further includes: determining a center line extending along the conveying direction of the conveying belt, and dividing the conveying belt into a left area and a right area based on the center line;

[0042] According to the material thickness curve data, the average material thickness of the left area and the average material thickness of the right area are calculated respectively, and the absolute value of the material thickness of the left area and the average material thickness of the right area is obtained by subtracting the absolute value of the material thickness of the left area and the right area. ;

[0043] The absolute value of the material thickness The material deviation is judged by the set threshold. If it exceeds the set threshold, the distribution of fine aggregate in the left and right areas is adjusted until the absolute value of the material thickness reaches Less than the set threshold.

[0044] In a second aspect, the present invention provides a fine aggregate moisture content detection device based on depth image and microwave spectrum, comprising:

[0045] The data acquisition module is configured to use an RGB-D depth camera to collect a material depth image of fine aggregate on the conveyor belt, use a multi-frequency microwave transmitting probe and a microwave receiving probe to collect microwave signals of fine aggregate at the same location on the conveyor belt, calculate the material thickness of the fine aggregate at the location according to the material depth image, obtain material thickness curve data, and generate a microwave spectrum curve according to the microwave signal;

[0046] A model building and training module is configured to build and train a multimodal moisture content prediction model to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network;

[0047] The moisture content prediction module is configured to input the microwave spectrum curve and material thickness curve data into the trained multimodal moisture content prediction model, the microwave spectrum curve passes through the microwave spectrum feature extraction network to extract the microwave spectrum features, the material thickness curve data passes through the material thickness feature extraction network to extract the material thickness features, the microwave spectrum features and the material thickness features are input into the feature fusion network to predict the predicted value of the moisture content.

[0048] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0050] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) The fine aggregate moisture content detection method based on depth image and microwave spectrum proposed in the present invention combines RGB-D imaging technology with microwave sensing technology, which can detect the moisture content of fine aggregate in real time online, effectively solving the problems of inaccurate and high delay in fine aggregate moisture content detection in the prior art, effectively filtering out material thickness fluctuations, and improving the accuracy and stability of moisture content measurement.

[0053] (2) The fine aggregate moisture content detection method based on depth image and microwave spectrum proposed in the present invention fuses the material thickness characteristics and microwave spectrum characteristics through a multimodal moisture content prediction model, and uses a deep learning model to predict the moisture content of fine aggregate. Compared with the traditional single modal method, multimodal fusion improves the detection accuracy and enhances the robustness of the system, and can better cope with the challenges under different working conditions.

[0054] (3) The fine aggregate moisture content detection method based on depth image and microwave spectrum proposed in the present invention calibrates the fine aggregate thickness curve by segmented thickness and uses the actual moisture content for calibration, thereby establishing an accurate relationship between the thickness curve data and the moisture content, which can provide reliable model support for subsequent moisture content measurement.

[0055] (4) The fine aggregate moisture content detection method based on depth image and microwave spectrum proposed in the present invention can accurately determine whether there is material deviation in the fine aggregate during the transmission process by comparing the material thickness values ​​of the left area and the right area, so as to timely discover the material deviation and guide the transmission system to make adjustments in a quantitative manner to ensure the uniformity of the material layer, thereby improving the accuracy of moisture content measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic flow chart of a method for detecting moisture content of fine aggregate based on depth image and microwave spectrum according to an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the material thickness range division of an embodiment of the present application;

[0059] Figure 3 A schematic diagram of the structure of a microwave spectrum feature extraction network of a fine aggregate moisture content detection method based on a depth image and a microwave spectrum according to an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of a material thickness feature extraction network of a fine aggregate moisture content detection method based on depth image and microwave spectrum according to an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of a feature fusion network of a fine aggregate moisture content detection method based on a depth image and a microwave spectrum according to an embodiment of the present application;

[0062] Figure 6 It is a structural schematic diagram of a fine aggregate moisture content detection system according to an embodiment of the present application;

[0063] Figure 7 It is a structural schematic diagram of a fine aggregate accumulation material thickness collection platform of a fine aggregate moisture content detection system of an embodiment of the present application;

[0064] Figure 8 A schematic structural diagram of a fine aggregate microwave collection platform of a fine aggregate moisture content detection system of an embodiment of the present application;

[0065] Fig. 9 A schematic diagram of a fine aggregate moisture content detection device based on depth image and microwave spectrum according to an embodiment of the present application;

[0066] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention.

[0067] Figure numerals: 11, conveyor belt; 12, industrial computer; 13, RGB-D mounting bracket; 14, RGB-D depth camera; 15, fine aggregate; 16, microwave sensor mounting bracket; 17, multi-frequency microwave transmitting probe; 18, microwave receiving probe. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Figure 1 A method for detecting moisture content of fine aggregate based on depth image and microwave spectrum provided in an embodiment of the present application is shown, comprising the following steps:

[0070] S1, uses an RGB-D depth camera to collect the material depth image of fine aggregate on the conveyor belt, uses a multi-frequency microwave transmitting probe and a microwave receiving probe to collect the microwave signal of the fine aggregate at the same location on the conveyor belt, and calculates the material thickness of the fine aggregate at that location based on the material depth image to obtain the material thickness curve data, and generates a microwave spectrum curve based on the microwave signal.

[0071] In a specific embodiment, the material thickness of the fine aggregate at the location is calculated according to the material depth image to obtain material thickness curve data, which specifically includes:

[0072] An RGB-D depth camera is used to collect a depth image of an empty conveyor belt to obtain a belt depth image. The depth value of each pixel point in the belt depth image represents the actual distance between the camera and the surface of the conveyor belt.

[0073] The depth value of each pixel in the material depth image represents the actual distance between the camera and the fine aggregate surface. The difference between the depth value of the same pixel in the belt depth image and the depth value in the material depth image is calculated to obtain the material thickness data of the pixel.

[0074] Calculate the material thickness data of all pixels in the material depth image to obtain the material thickness curve data.

[0075] S2, constructing and training a multimodal moisture content prediction model to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network.

[0076] In a specific embodiment, the training data used in the training process of the multimodal moisture content prediction model is obtained by the following steps:

[0077] Draw a material thickness curve of fine aggregate on a pixel point-material thickness plane according to the material thickness curve data of fine aggregate;

[0078] According to a plurality of material thickness curves, the pixel point-material thickness plane is divided into a plurality of material thickness ranges, the area between every two material thickness curves is a material thickness range, a material thickness baseline is defined for each material thickness range, and a moisture content regression model is constructed and trained to obtain a trained moisture content regression model;

[0079] Calculate the envelope area of ​​the material thickness curve of fine aggregate and all material thickness baselines respectively, and select the material thickness baseline with the smallest envelope area and its corresponding trained moisture content regression model;

[0080] Input the microwave spectrum curve of the fine aggregate into the selected trained moisture content regression model, and output the moisture content label of the fine aggregate;

[0081] The training data is constructed based on the material thickness curve data, microwave spectrum curve and corresponding moisture content labels of fine aggregate.

[0082] In a specific embodiment, constructing and training a moisture content regression model to obtain a trained moisture content regression model specifically includes:

[0083] A moisture content regression model is constructed for each material thickness range. The moisture content regression model includes one of a random forest regression model and a support vector regression model. The microwave spectrum curve of the fine aggregate is input into the moisture content regression model to obtain the moisture content label of the fine aggregate.

[0084] Compare the moisture content label of the fine aggregate with the actual moisture content of the fine aggregate to obtain a comparison result;

[0085] The parameters of the moisture content regression model are adjusted according to the comparison results to obtain a trained moisture content regression model.

[0086] Specifically, the present invention proposes a material thickness segmentation calibration algorithm to calibrate the moisture content of fine aggregate. The specific implementation process of the material thickness segmentation calibration algorithm is as follows:

[0087] The material depth image corresponding to the fine aggregate 15 is continuously collected by the RGB-D depth camera 14, and the material thickness curve is drawn on the pixel point-material thickness plane according to the material thickness curve data corresponding to all the pixel points in the material depth image; a plurality of material thickness curves are analyzed by using a data analysis method, and the pixel point-material thickness plane is divided into a plurality of material thickness ranges according to the plurality of material thickness curves, a material thickness baseline is defined for each material thickness range, and a moisture content regression model is constructed, and the corresponding moisture content regression model is trained by experimental data, referring to Figure 2 The area between each two material thickness curves is a material thickness range. Corresponding to moisture content regression model Y1, material thickness range Corresponding to moisture content regression model Y2, material thickness range Corresponding to the moisture content regression model Y3; compare the material thickness curve with each material thickness baseline, and calculate the envelope area between the material thickness curve and the material thickness baseline. The smaller the envelope area, the higher the matching degree of the two curves. According to the size of the envelope area, select the material thickness baseline closest to the material thickness curve and its corresponding moisture content regression model for the calculation of the current fine aggregate moisture content label to reduce the impact of material thickness fluctuations on moisture content detection. According to the moisture content calibration method of material thickness segmented calibration, obtain the material thickness curve data of fine aggregate and the moisture content label corresponding to the microwave spectrum curve, which are applied to the subsequent multi-modal moisture content prediction model.

[0088] In the training process of the moisture content regression model, the material thickness baseline closest to the material thickness curve of the new sample fine aggregate and its corresponding moisture content regression model are selected, the microwave spectrum curve of the new sample fine aggregate is input into the corresponding moisture content regression model, its moisture content label is calculated, and the calculated moisture content label is compared with the actual moisture content to verify and optimize the moisture content regression model. The accuracy of the moisture content regression model is verified using the new sample data, and the model parameters are optimized according to the verification results, the material thickness range is adjusted, a new material thickness baseline is obtained, and the optimized moisture content regression model is fitted to improve the prediction accuracy.

[0089] S3, input the microwave spectrum curve and material thickness curve data into the trained multimodal moisture content prediction model, the microwave spectrum curve passes through the microwave spectrum feature extraction network to extract the microwave spectrum features, the material thickness curve data passes through the material thickness feature extraction network to extract the material thickness features, the microwave spectrum features and material thickness features are input into the feature fusion network to predict the moisture content.

[0090] In a specific embodiment, the microwave spectrum feature extraction network includes a multi-core one-dimensional separable convolution module, a first separable convolution layer, a multi-head attention module, a second separable convolution layer and a first convolution layer connected in sequence, wherein the convolution kernel sizes of the first separable convolution layer and the second separable convolution layer are both , the convolution kernel size of the first convolutional layer is The microwave spectrum curve passes through the microwave spectrum feature extraction network, and the specific process of extracting the microwave spectrum features is as follows:

[0091] Input the microwave spectrum curve into a multi-core one-dimensional separable convolution module for multi-scale feature extraction. The multi-core one-dimensional separable convolution module includes a plurality of one-dimensional separable convolution layers with different convolution kernel sizes. The microwave spectrum curve is convolved through the plurality of one-dimensional separable convolution layers with different convolution kernel sizes to obtain a first multi-scale feature. After the first multi-scale feature is spliced ​​in the depth dimension, a second multi-scale feature is obtained.

[0092] The second multi-scale feature passes through the first separable convolution layer and the multi-head attention module in sequence to obtain the first global feature;

[0093] The first global feature is sequentially passed through the second separable convolution layer and the first convolution layer to obtain a microwave spectrum feature;

[0094] The material thickness feature extraction network includes five convolutional modules, a flattening layer, and three first fully connected layers connected in sequence. The material thickness curve data passes through the material thickness feature extraction network, and the specific process of extracting the material thickness feature is as follows:

[0095] The material thickness curve data is sequentially input into five convolution modules to obtain local features. Each convolution module includes a convolution kernel size of One-dimensional convolution layer, ReLU activation function layer and maximum pooling layer;

[0096] The local features are input into the flattening layer and flattened into a one-dimensional vector to obtain the flattened local features;

[0097] The flattened local features are sequentially integrated and extracted through three first fully connected layers to obtain material thickness features. The first fully connected layers all use the ReLU activation function.

[0098] The feature fusion network includes a summation module, a second fully connected layer, a third fully connected layer, a second convolutional layer, and a fourth fully connected layer connected in sequence, wherein the convolution kernel size of the second convolutional layer is The output dimension of the second and third fully connected layers is 2D. The material thickness feature and microwave spectrum feature are input into the feature fusion network to predict the specific process of obtaining the predicted value of moisture content as follows:

[0099] Input the microwave spectrum feature and the material thickness feature into a summing module, perform element-by-element summing operation at corresponding feature positions, and obtain a first mixed feature;

[0100] The first mixed feature is input into the second fully connected layer, and activated by linear transformation and nonlinear activation function to obtain the second mixed feature;

[0101] The second mixed feature is input into the third fully connected layer, and the first weight and the second weight are output. The first weight and the second weight are subjected to softmax normalization operation and then element-wise multiplied with the microwave spectrum feature and the material thickness feature to obtain weighted microwave spectrum feature and weighted material thickness feature. The weighted microwave spectrum feature and the weighted material thickness feature are summed to obtain a fusion feature.

[0102] The fused features pass through the second convolutional layer and the fourth fully connected layer in sequence to obtain the moisture content prediction value.

[0103] Specifically, in the process of testing the moisture content of fine aggregate, the fluctuation of material thickness is an important influencing factor. Materials of different thicknesses will cause changes in the absorption and reflection characteristics of microwave signals, thereby affecting the microwave spectrum curve. The moisture content measurement results without material thickness compensation may produce significant deviations due to changes in material thickness. Therefore, a multimodal moisture content prediction model is used to adaptively compensate for the thickness of fine aggregate.

[0104] Specifically, refer to Figure 3 In the microwave spectrum feature extraction network, the microwave spectrum curve is first subjected to one-dimensional separable convolution operations through several one-dimensional separable convolution layers with different convolution kernel sizes. For example, one-dimensional separable convolution layers with convolution kernel sizes of 3, 7, 11, and 15 are used to perform one-dimensional separable convolution operations on the microwave spectrum curve to capture different scale features in the microwave spectrum curve and obtain the first multi-scale feature; the different scale features are spliced ​​in the depth dimension to fuse the multi-scale feature information and obtain the second multi-scale feature. The second multi-scale feature is subjected to a convolution kernel size of The first separable convolution layer performs a one-dimensional separable convolution operation to further refine the fused multi-scale features to reduce redundancy and noise and enhance the representation ability of the features. In order to capture the intrinsic connection between microwave spectrum features, a multi-head attention module (MHSA) is introduced after the first separable convolution layer to obtain the first global feature. MHSA can enhance the connection between different parts of the microwave signal by calculating the mutual correlation between microwave spectrum features, thereby better representing the global features in the microwave signal. The first global feature obtained after MHSA is further passed through a convolution kernel size of The second separable convolution layer performs a one-dimensional separable convolution operation to further reduce redundancy and noise. Finally, a convolution kernel size of The first convolutional layer compresses the high-dimensional features to the target feature dimension to obtain the final microwave spectrum features.

[0105] refer to Figure 4 In the material thickness feature extraction network, the material thickness curve data is subjected to five layers of continuous convolution operations through five convolution modules, and each convolution operation adopts One-dimensional convolution kernels of different sizes are used to capture local patterns and trends in the material thickness curve data. Through layer-by-layer convolution, the material thickness feature extraction network can identify higher-level and more abstract features. Each convolution operation is followed by a ReLU activation function layer. The ReLU function helps introduce nonlinearity while maintaining network sparsity and reducing computational complexity, so that the network can express more complex functional relationships. After each layer of convolution and activation operation, a maximum pooling layer is added. Its function is to reduce the spatial dimension of the feature by downsampling, thereby reducing the number of parameters and computational burden, while retaining important information in the material thickness curve data. After completing the convolution and pooling operations, local features are obtained. The local features are flattened into one-dimensional vectors through the flattening layer to obtain the flattened local features. Then, the flattened local features are further processed through three first fully connected layers. Each first fully connected layer uses the ReLU activation function to further introduce nonlinearity and improve the fitting ability of the network. The function of the first fully connected layer is to integrate the local features extracted from the convolution into global features, and then extract comprehensive material thickness features.

[0106] refer to Figure 5 In the feature fusion network, the microwave spectrum features and material thickness features extracted above are input into the summation module, and the element-by-element summation operation is performed at the corresponding feature positions to obtain a first mixed feature M. The first mixed feature M is input into the second fully connected layer, and a more expressive second mixed feature N is extracted through linear transformation and nonlinear activation function. The dimension of the second mixed feature N is 2D. The second mixed feature N is input into the third fully connected layer, and the second mixed feature N is linearly changed in the third fully connected layer. The dimension of the output of the third fully connected layer is 2D, and the first weight W1 and the second weight W2 corresponding to the microwave spectrum feature and the material thickness feature are generated respectively. Since the information and dimensions of the microwave spectrum feature and the material thickness feature are different, the first weight W1 and the second weight W2 obtained by the output of the second mixed feature N through the third fully connected layer correspond to the microwave spectrum feature and the material thickness feature respectively. The two weights are normalized by softmax to ensure that they can represent the relative importance of the two features to the final prediction. The first normalized weight and the second normalized weight are respectively multiplied element-by-element with the initial microwave spectrum feature and material thickness feature to generate weighted microwave spectrum feature and weighted material thickness feature. The weighted microwave spectrum feature and weighted material thickness feature are summed to obtain the final fusion feature. In order to adjust the dimension of the fusion feature to make it suitable for subsequent processing, a convolution kernel size of The second convolutional layer performs a convolution operation on it and then inputs it into the final fourth fully connected layer to predict the moisture content, thereby obtaining the predicted value of the moisture content.

[0107] In a specific embodiment, the method further includes: determining a center line extending along the conveying direction of the conveyor belt, and dividing the conveyor belt into a left area and a right area based on the center line;

[0108] According to the material thickness curve data, the average material thickness of the left area and the average material thickness of the right area are calculated respectively, and the absolute value of the material thickness of the left area and the average material thickness of the right area is obtained by subtracting the absolute value of the material thickness of the left area and the right area. ;

[0109] The absolute value of the material thickness The material deviation is judged by the set threshold. If it exceeds the set threshold, the distribution of fine aggregate in the left and right areas is adjusted until the absolute value of the material thickness reaches Less than the set threshold.

[0110] Specifically, the above-mentioned material deviation determination algorithm can accurately determine whether there is material deviation in the fine aggregate during the transmission process so as to timely discover the material deviation, and guide the transmission system to make adjustments in a quantitative manner to ensure the uniformity of the material layer, thereby improving the accuracy of moisture content measurement.

[0111] Specifically, compared with the traditional ultrasonic fine aggregate thickness measurement, the present invention uses RGB-D depth camera to measure fine aggregate thickness, which has the following advantages:

[0112] First, for the detection of insufficient material thickness, the RGB-D depth camera has a large detection field of view and can detect all material thickness information in the width direction of the conveyor belt. At the same time, the RGB-D depth camera has a high acquisition frequency to ensure the real-time measurement of material thickness. Therefore, the judgment of insufficient material thickness will be more accurate.

[0113] Second, for the detection of material deviation, the embodiment of the present application provides a material deviation determination algorithm based on the RGB-D depth camera. If the difference in the absolute material thickness between the left area and the right area exceeds a set threshold, it is considered that there is material deviation. This threshold is adjusted according to the actual situation to adapt to different material thicknesses and sand material properties.

[0114] Third, for the detection of fine aggregate types, the RGB-D depth camera can analyze the surface characteristics of fine aggregates, such as texture and color, by combining depth images and RGB images to infer their materials. For example, fine aggregates of different materials (such as quartz sand, river sand, etc.) show different colors and reflective properties in RGB images. The RGB channel can capture the color information of fine aggregates and is used to distinguish fine aggregates of different sources or types. Before the fine aggregates enter the detection process, the RGB-D depth camera can be used to identify their material and color characteristics in advance to determine which type of fine aggregate it is. According to the identified type of fine aggregate, the most suitable moisture regression model is selected. Different types of fine aggregates require different moisture regression models or correction parameters to accurately measure their moisture content or thickness due to different physical properties such as dielectric constant. The accuracy of the measurement results is ensured by matching the appropriate moisture regression model.

[0115] The microwave transmission method uses the characteristic that the dielectric constant of water molecules is much larger than that of fine aggregates, and consumes a lot of energy in high-frequency electric fields. When microwaves transmit through the medium, microwave energy of a specific frequency will be absorbed by the water, causing the microwave signal intensity to change. Since fine aggregates are unevenly distributed on the conveyor belt 11, in order to improve the accuracy of moisture content detection, material thickness compensation is required. The material depth image of fine aggregates is obtained with an RGB-D depth camera, and the material thickness curve data of fine aggregates is calculated based on the material depth image. The microwave signal and material thickness curve data after transmission are input into the multimodal moisture content prediction model to output the predicted moisture content.

[0116] In the embodiments of the present application, Figure 6 The structural schematic diagram of the fine aggregate moisture content detection system is shown, which includes a conveyor belt 11, an industrial computer 12, an RGB-D mounting bracket 13, an RGB-D depth camera 14, fine aggregate 15, a microwave sensor mounting bracket 16, a multi-frequency microwave transmitting probe 17 and a microwave receiving probe 18. The RGB-D depth camera 14 is connected to the industrial computer 12 through a network cable, and the collected material depth image of the fine aggregate is transmitted to the industrial computer 12. The microwave signal received by the microwave receiving probe 18 is transmitted to the industrial computer 12 through a data acquisition card. The transmission direction of the fine aggregate 15 on the conveyor belt 11 is first through the RGB-D depth camera 14, and then through the multi-frequency microwave transmitting probe 17 and the microwave receiving probe 18. In the industrial computer 12, the data processing unit fuses the material depth image and the microwave signal, and finally outputs accurate fine aggregate moisture content data through algorithms such as material thickness fluctuation filtering, material deviation judgment and material thickness segmentation calibration.

[0117] refer to Figure 7, the RGB-D depth camera 14 and the RGB-D mounting bracket 13 constitute a fine aggregate accumulation material thickness collection platform, the RGB-D mounting bracket 13 is a door-type detection bracket, and the RGB-D depth camera 14 is installed on the RGB-D mounting bracket 13 and is located above the midpoint position in the width direction of the conveyor belt 11. The RGB-D depth camera 14 is used to detect the fine aggregate material thickness information on the conveyor belt 11 and save the data as a single-channel depth image. In order to reduce memory, improve detection efficiency and data matching accuracy, the depth image is processed in the data processing unit of the industrial computer 12 to obtain material thickness curve data and draw the material thickness curve.

[0118] refer to Figure 8 The microwave sensor mounting bracket 16, the multi-frequency microwave transmitting probe 17 and the microwave receiving probe 18 constitute a microwave collection platform for fine aggregates. Under the condition that the equipment does not collide with the material, the multi-frequency microwave transmitting probe 17 is installed on the lower beam of the microwave sensor mounting bracket 16, and the microwave receiving probe 18 is installed on the upper beam of the microwave sensor mounting bracket 16. Based on the microwave transmission method, the microwave signals penetrating the fine aggregates 15 with different moisture contents are collected and the data is stored in real time. The multi-frequency microwave transmitting probe 17 transmits the multi-frequency microwave signal, and the microwave signal is received by the microwave receiving probe 18 after penetrating the fine aggregate. The received microwave signal is transmitted to the industrial control computer 12, and its data processing unit generates a microwave spectrum curve according to the microwave spectrum value of the received microwave signal.

[0119] The conveyor belt 11 is used to transport fine aggregate 15 to pass through the fine aggregate accumulation material thickness collection platform and the fine aggregate microwave collection platform in turn to collect material depth images and microwave signals. The material depth image and microwave signal of the fine aggregate should ensure data matching. The data matching method is: the distance between the fine aggregate accumulation material thickness collection platform and the fine aggregate microwave collection platform is L, and the transmission speed of the conveyor belt 11 is v. The time required for the fine aggregate 15 to move from the bottom of the RGB-D depth camera 14 to between the multi-frequency microwave transmitting probe 17 and the microwave receiving probe 18 is When the fine aggregate 15 moves below the RGB-D depth camera 14, the timestamp is recorded as After t time, the fine aggregate 15 reaches between the multi-frequency microwave transmitting probe 17 and the microwave receiving probe 18, and the timestamp is recorded as , then at the timestamp and The collected data are the material depth image and microwave signal corresponding to the same fine aggregate 15. The material thickness of the fine aggregate at that location is calculated according to the material depth image to obtain the material thickness curve data, and the microwave spectrum curve is generated according to the microwave signal. The material thickness curve data and the microwave spectrum curve of the fine aggregate 15 need to be filtered. The material thickness curve data and the microwave spectrum curve are filtered using Gaussian filtering or median filtering methods to remove noise and smooth the data.

[0120] The fine aggregate moisture content detection method based on depth image and microwave spectrum proposed in the present invention is processed by algorithms such as material deviation judgment and material thickness segmentation calibration, and the data of the material depth image and microwave signal are fused after data processing, and the fine aggregate thickness is adaptively compensated, and finally accurate fine aggregate moisture content data is output, which improves the accuracy and stability of moisture content measurement, realizes real-time online detection of fine aggregate moisture content, and solves the problems of inaccurate fine aggregate moisture content detection and high delay in the prior art. This method can optimize the use of fine aggregate, reduce cement consumption, and reduce carbon emissions in the construction industry.

[0121] The above steps S1-S3 do not necessarily represent the order between the steps, but are step symbol representations, and the order between the steps can be adjusted.

[0122] Further references Fig. 9 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a fine aggregate moisture content detection device based on depth image and microwave spectrum. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0123] The embodiment of the present application provides a fine aggregate moisture content detection device based on depth image and microwave spectrum, comprising:

[0124] The data acquisition module 1 is configured to use an RGB-D depth camera to collect a material depth image of fine aggregate on the conveyor belt, use a multi-frequency microwave transmitting probe and a microwave receiving probe to collect microwave signals of fine aggregate at the same location on the conveyor belt, calculate the material thickness of the fine aggregate at the location according to the material depth image, obtain material thickness curve data, and generate a microwave spectrum curve according to the microwave signal;

[0125] The model building and training module 2 is configured to build and train a multimodal moisture content prediction model to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network;

[0126] The moisture content prediction module 3 is configured to input the microwave spectrum curve and the material thickness curve data into the trained multimodal moisture content prediction model, the microwave spectrum curve passes through the microwave spectrum feature extraction network to extract the microwave spectrum feature, the material thickness curve data passes through the material thickness feature extraction network to extract the material thickness feature, the microwave spectrum feature and the material thickness feature are input into the feature fusion network to predict the moisture content.

[0127] Fig.10 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Fig.10As shown, the electronic device of this embodiment includes: a processor 1001 and a memory 1002; wherein the memory 1002 is used to store computer-executable instructions; the processor 1001 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0128] Optionally, the memory 1002 may be independent or integrated with the processor 1001 .

[0129] When the memory 1002 is independently provided, the electronic device further includes a bus 1003 for connecting the memory 1002 and the processor 1001 .

[0130] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 1001 executes the computer execution instructions, the above method is implemented.

[0131] An embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 1001, the above method is implemented.

[0132] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0133] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0134] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned module-composed unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0135] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 1001 to perform some steps of the methods of various embodiments of the present application.

[0136] It should be understood that the processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 1001 may be any conventional processor 1001, etc. The steps of the method disclosed in the invention may be directly embodied in the hardware processor 1001 for execution, or may be executed by a combination of hardware and software modules in the processor 1001.

[0137] The memory 1002 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0138] The bus 1003 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1003 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 1003 in the drawings of the present application is not limited to only one bus 1003 or one type of bus 1003.

[0139] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0140] An exemplary storage medium is coupled to the processor 1001, so that the processor 1001 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 1001. The processor 1001 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 1001 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0141] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting moisture content of fine aggregate based on depth image and microwave spectrum, characterized in that: The following steps are involved: An RGB-D depth camera is used to collect a material depth image of fine aggregate on a conveyor belt, a multi-frequency microwave transmitting probe and a microwave receiving probe are used to collect microwave signals of fine aggregate at the same location on the conveyor belt, and the material thickness of the fine aggregate at the location is calculated based on the material depth image to obtain material thickness curve data, and a microwave spectrum curve is generated based on the microwave signal; A multimodal moisture content prediction model is constructed and trained to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network, wherein the microwave spectrum feature extraction network includes a multi-core one-dimensional separable convolution module, a first separable convolution layer, a multi-head attention module, a second separable convolution layer, and a first convolution layer connected in sequence, wherein the multi-core one-dimensional separable convolution module includes a plurality of one-dimensional separable convolution layers with different convolution kernel sizes, and the convolution kernel sizes of the first separable convolution layer and the second separable convolution layer are both , the convolution kernel size of the first convolutional layer is The material thickness feature extraction network includes five convolution modules, a flattening layer and three first fully connected layers connected in sequence, each of which includes a convolution kernel size of The first fully connected layer adopts the ReLU activation function; the feature fusion network includes a summation module, a second fully connected layer, a third fully connected layer, a second convolutional layer and a fourth fully connected layer connected in sequence, wherein the convolution kernel size of the second convolutional layer is , the output dimension of the second fully connected layer and the third fully connected layer is 2 dimensions; The microwave spectrum curve and material thickness curve data are input into the trained multi-modal moisture content prediction model. The microwave spectrum curve passes through the microwave spectrum feature extraction network to extract microwave spectrum features. The specific process is as follows: Inputting the microwave spectrum curve into the multi-core one-dimensional separable convolution module to extract multi-scale features, respectively performing convolution operations on the microwave spectrum curve through a plurality of one-dimensional separable convolution layers with different convolution kernel sizes to obtain a first multi-scale feature, and concatenating the first multi-scale features in the depth dimension to obtain a second multi-scale feature; The second multi-scale feature passes through the first separable convolutional layer and the multi-head attention module in sequence to obtain a first global feature; Passing the first global feature through the second separable convolutional layer and the first convolutional layer in sequence to obtain a microwave spectrum feature; The material thickness curve data is passed through the material thickness feature extraction network to extract the material thickness feature. The specific process is as follows: Inputting the material thickness curve data into the five convolution modules in sequence to obtain local features; Inputting the local feature into the flattening layer and flattening it into a one-dimensional vector to obtain a flattened local feature; The flattened local features are sequentially integrated and extracted through three first fully connected layers to obtain material thickness features; The microwave spectrum feature and the material thickness feature are input into the feature fusion network to predict the moisture content. The specific process is as follows: Inputting the microwave spectrum feature and the material thickness feature into the summing module, performing element-by-element summing operation at corresponding feature positions, and obtaining a first mixed feature; Inputting the first mixed feature into the second fully connected layer, activating it through linear transformation and nonlinear activation function, to obtain a second mixed feature; Inputting the second mixed feature into the third fully connected layer, outputting a first weight and a second weight, performing a softmax normalization operation on the first weight and the second weight, respectively performing element-by-element multiplication with the microwave spectrum feature and the material thickness feature to obtain a weighted microwave spectrum feature and a weighted material thickness feature, and summing the weighted microwave spectrum feature and the weighted material thickness feature to obtain a fusion feature; The fused features are sequentially passed through the second convolutional layer and the fourth fully connected layer to obtain a moisture content prediction value.

2. The fine aggregate moisture content detection method based on depth image and microwave spectrum according to claim 1 is characterized in that: The step of calculating the material thickness of the fine aggregate at the location according to the material depth image to obtain material thickness curve data specifically includes: An RGB-D depth camera is used to collect a depth image of an empty conveyor belt to obtain a belt depth image, wherein the depth value of each pixel point of the belt depth image represents the actual distance between the camera and the surface of the conveyor belt; The depth value of each pixel in the material depth image represents the actual distance between the camera and the surface of the fine aggregate. The difference between the depth value of the same pixel in the belt depth image and the depth value in the material depth image is calculated to obtain the material thickness data of the pixel. The material thickness data of all pixel points in the material depth image are calculated to obtain material thickness curve data.

3. The fine aggregate moisture content detection method based on depth image and microwave spectrum according to claim 1 is characterized in that: The training data used in the training process of the multimodal moisture content prediction model is obtained by the following steps: Draw a material thickness curve of the fine aggregate on a pixel point-material thickness plane according to the material thickness curve data of the fine aggregate; Dividing the pixel point-material thickness plane into a plurality of material thickness ranges according to a plurality of material thickness curves, wherein the area between each two material thickness curves is a material thickness range, defining a material thickness baseline for each of the material thickness ranges, and constructing and training a moisture content regression model to obtain a trained moisture content regression model; Calculating respectively the envelope areas of the material thickness curve of the fine aggregate and all the material thickness baselines, and selecting the material thickness baseline with the smallest envelope area and the trained moisture content regression model corresponding thereto; Inputting the microwave spectrum curve of the fine aggregate into the selected trained moisture content regression model, and outputting a moisture content label of the fine aggregate; The training data is constructed according to the material thickness curve data, the microwave spectrum curve and the corresponding moisture content label of the fine aggregate.

4. The method for detecting moisture content of fine aggregate based on depth image and microwave spectrum according to claim 3 is characterized in that: The step of constructing and training a moisture content regression model to obtain a trained moisture content regression model specifically includes: Constructing a moisture content regression model for each material thickness range, wherein the moisture content regression model includes one of a random forest regression model and a support vector regression model, and inputting the microwave spectrum curve of the fine aggregate into the moisture content regression model to obtain a moisture content label of the fine aggregate; Comparing the moisture content label of the fine aggregate with the actual moisture content of the fine aggregate to obtain a comparison result; The parameters of the moisture content regression model are adjusted according to the comparison result to obtain a trained moisture content regression model.

5. The method for detecting moisture content of fine aggregate based on depth image and microwave spectrum according to claim 1, characterized in that: The method further includes: determining a center line extending along the conveying direction of the conveyor belt, and dividing the conveyor belt into a left area and a right area based on the center line; According to the material thickness curve data, the average material thickness of the left area and the average material thickness of the right area are calculated respectively, and the average material thickness of the left area and the average material thickness of the right area are subtracted to obtain the absolute value of the material thickness of the left area and the right area. ; The absolute value of the material thickness The material deviation is judged with the set threshold. If it exceeds the set threshold, the distribution of fine aggregate in the left and right areas is adjusted until the absolute value of the material thickness reaches Less than the set threshold.

6. A fine aggregate moisture content detection device based on depth image and microwave spectrum, characterized in that: include: The data acquisition module is configured to use an RGB-D depth camera to collect a material depth image of fine aggregate on a conveyor belt, use a multi-frequency microwave transmitting probe and a microwave receiving probe to collect microwave signals of fine aggregate at the same location on the conveyor belt, calculate the material thickness of the fine aggregate at the location according to the material depth image, obtain material thickness curve data, and generate a microwave spectrum curve according to the microwave signal; The model building and training module is configured to build and train a multimodal moisture content prediction model to obtain a trained multimodal moisture content prediction model, wherein the multimodal moisture content prediction model includes a microwave spectrum feature extraction network, a material thickness feature extraction network, and a feature fusion network, wherein the microwave spectrum feature extraction network includes a multi-core one-dimensional separable convolution module, a first separable convolution layer, a multi-head attention module, a second separable convolution layer, and a first convolution layer connected in sequence, wherein the multi-core one-dimensional separable convolution module includes a plurality of one-dimensional separable convolution layers with different convolution kernel sizes, and the convolution kernel sizes of the first separable convolution layer and the second separable convolution layer are both , the convolution kernel size of the first convolutional layer is The material thickness feature extraction network includes five convolution modules, a flattening layer and three first fully connected layers connected in sequence, each of which includes a convolution kernel size of The first fully connected layer adopts the ReLU activation function; the feature fusion network includes a summation module, a second fully connected layer, a third fully connected layer, a second convolutional layer and a fourth fully connected layer connected in sequence, wherein the convolution kernel size of the second convolutional layer is , the output dimension of the second fully connected layer and the third fully connected layer is 2 dimensions; The moisture content prediction module is configured to input the microwave spectrum curve and material thickness curve data into the trained multi-modal moisture content prediction model, and the microwave spectrum curve passes through the microwave spectrum feature extraction network to extract microwave spectrum features. The specific process is as follows: Inputting the microwave spectrum curve into the multi-core one-dimensional separable convolution module to extract multi-scale features, respectively performing convolution operations on the microwave spectrum curve through a plurality of one-dimensional separable convolution layers with different convolution kernel sizes to obtain a first multi-scale feature, and concatenating the first multi-scale features in the depth dimension to obtain a second multi-scale feature; The second multi-scale feature passes through the first separable convolutional layer and the multi-head attention module in sequence to obtain a first global feature; Passing the first global feature through the second separable convolutional layer and the first convolutional layer in sequence to obtain a microwave spectrum feature; The material thickness curve data is passed through the material thickness feature extraction network to extract the material thickness feature. The specific process is as follows: Inputting the material thickness curve data into the five convolution modules in sequence to obtain local features; Inputting the local feature into the flattening layer and flattening it into a one-dimensional vector to obtain a flattened local feature; The flattened local features are sequentially integrated and extracted through three first fully connected layers to obtain material thickness features; The microwave spectrum feature and the material thickness feature are input into the feature fusion network to predict the moisture content. The specific process is as follows: Inputting the microwave spectrum feature and the material thickness feature into the summing module, performing element-by-element summing operation at corresponding feature positions, and obtaining a first mixed feature; Inputting the first mixed feature into the second fully connected layer, activating it through linear transformation and nonlinear activation function, to obtain a second mixed feature; Inputting the second mixed feature into the third fully connected layer, outputting a first weight and a second weight, performing a softmax normalization operation on the first weight and the second weight, respectively performing element-by-element multiplication with the microwave spectrum feature and the material thickness feature to obtain a weighted microwave spectrum feature and a weighted material thickness feature, and summing the weighted microwave spectrum feature and the weighted material thickness feature to obtain a fusion feature; The fused features are sequentially passed through the second convolutional layer and the fourth fully connected layer to obtain a moisture content prediction value.

7. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images