An intelligent detection system for conveyor belt running load

Abstract

The present application relates to the field of intelligent detection technology, and specifically discloses an intelligent detection system for conveyor belt running load, which uses a camera and a laser scanner to collect images and three-dimensional point cloud models of materials on the conveyor belt in real time, and extracts features from the three-dimensional point cloud model and image of the material through artificial intelligence technology based on deep learning, so as to mine the material's property characteristics and three-dimensional geometric features, and realizes multi-dimensional perception and intelligent estimation of the conveyor belt load by cross-modal joint encoding of the two. In this way, the running load of the conveyor belt can be accurately and quickly detected without direct contact with the material, providing strong support for the optimization of the production process and fault prevention.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection technology, and more specifically, to an intelligent detection system for conveyor belt running load. Background Art

[0002] Conveyor belts are a type of material handling equipment widely used in industrial production. They transport materials from one place to another through continuous or intermittent movement. With the continuous development of industrial automation and intelligence, conveyor belts have been widely used in many industries such as mining, ports, chemicals, and logistics. However, conveyor belts often need to carry a large amount of materials for long-distance transportation during operation. Long-term operation and large workloads may cause various faults, such as loss of belt ribs, broken belt cores, and damaged adhesive layers, which will not only affect production efficiency, but also pose a threat to workers' safety. Therefore, accurate monitoring of the operating load of the conveyor belt and timely detection of potential safety hazards are of great significance for preventing failures, improving production efficiency, and ensuring worker safety.

[0003] Traditional conveyor belt load detection methods mainly rely on contact sensor measurements, such as tension sensors and weight sensors. However, the installation position and method of this contact sensor have a significant impact on its measurement accuracy. More importantly, since the sensor needs to be in direct contact with the conveyor belt or material for a long time, it is susceptible to wear and damage, resulting in reduced measurement accuracy. Especially when the conveyor belt runs at a high speed, the material is heavy or contains sharp edges, the durability of the sensor is seriously affected, requiring the sensor to be regularly maintained and replaced, increasing maintenance costs and labor costs.

[0004] Therefore, an optimized intelligent detection system for the conveyor belt running load is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent detection system for the running load of a conveyor belt, which uses a camera and a laser scanner to collect images and three-dimensional point cloud models of materials on the conveyor belt in real time, and extracts features from the three-dimensional point cloud model and image of the material through artificial intelligence technology based on deep learning, so as to mine the property characteristics and three-dimensional geometric characteristics of the material, and realizes multi-dimensional perception and intelligent estimation of the conveyor belt load by cross-modal joint encoding of the two. In this way, the running load of the conveyor belt can be accurately and quickly detected without direct contact with the material, providing strong support for the optimization of the production process and fault prevention.

[0006] Accordingly, according to one aspect of the present application, there is provided an intelligent detection system for conveyor belt running load, comprising:

[0007] The conveying material information acquisition module is used to acquire the material image acquired by the camera and the material three-dimensional point cloud model acquired by the laser scanner;

[0008] A material property feature extraction module, used to extract material property features in the material image to obtain a material property semantic feature map;

[0009] A material property feature enhancement module, used for enhancing the attention of the material property semantic feature map to obtain an enhanced material property semantic feature vector;

[0010] A material geometric feature extraction module, used to extract the material geometric features in the three-dimensional point cloud model of the material to obtain a three-dimensional geometric feature map of the material;

[0011] The operating load estimation module is used to generate an estimated decoding value of the conveyor belt's operating load based on the cross-modal joint representation features of the enhanced material property semantic feature vector and the material's three-dimensional geometric feature map.

[0012] In the above-mentioned intelligent detection system for the conveyor belt running load, the material property feature extraction module is used to: input the material image into the material property feature extractor based on the void convolutional neural network model to obtain the material property semantic feature map.

[0013] In the above-mentioned intelligent detection system for the conveyor belt running load, the material property feature enhancement module includes: an attention enhancement unit, which is used to input the material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain an enhanced material property semantic feature map; a feature enhancement compression unit, which is used to perform global maximum pooling along the channel dimension on the enhanced material property semantic feature map to obtain the enhanced material property semantic feature vector.

[0014] In the above-mentioned intelligent detection system for the running load of the conveyor belt, the attention enhancement unit includes: a global mean pooling subunit, which is used to calculate the global mean of each feature matrix along the channel dimension of the material property semantic feature map to obtain a material property semantic feature compression information representation vector; a channel information association interaction subunit, which is used to perform one-dimensional convolution encoding on the material property semantic feature compression representation vector to obtain a material property semantic feature compression information association representation feature vector; a multi-scale fusion subunit, which is used to cascade the material property semantic feature compression information representation vector and the material property semantic feature compression information association representation feature vector to obtain a material property semantic feature compression information multi-scale representation vector; a multi-scale association interaction subunit A feature extraction subunit is used to input the multi-scale representation vector of the compressed information of the semantic feature of the material property into a compressed information feature extraction module based on a multi-layer perceptron model to obtain a multi-scale associated feature vector of the compressed information of the semantic feature of the material property, wherein the multi-layer perceptron model comprises two fully connected layers and a SiLU activation function; a normalization subunit is used to use a Sigmoid function to perform a normalization operation on the multi-scale associated feature vector of the compressed information of the semantic feature of the material property to obtain a material property semantic feature weight vector; a feature selection subunit is used to perform feature amplification and suppression operations on the material property semantic feature map based on the material property semantic feature weight vector to obtain the enhanced material property semantic feature map.

[0015] In the above-mentioned intelligent detection system for the conveyor belt running load, the feature selection subunit is used to: calculate the corresponding multiplication of each eigenvalue of the material property semantic feature weight vector and each feature matrix along the channel dimension of the material property semantic feature map to obtain the enhanced material property semantic feature map.

[0016] In the above-mentioned intelligent detection system for the conveyor belt running load, the material geometric feature extraction module is used to: input the material three-dimensional point cloud model into the material geometric feature extractor based on the three-dimensional convolutional neural network model to obtain the material three-dimensional geometric feature map.

[0017] In the above-mentioned intelligent detection system for the conveyor belt running load, the running load estimation module includes: a joint encoding unit, which is used to input the enhanced material property semantic feature vector and the material three-dimensional geometric feature map into a material property-material geometry joint encoder based on a meta-network model to obtain a material property-geometry joint representation feature map; a decoding unit, which is used to input the material property-geometry joint representation feature map into a decoder-based running load estimator to obtain the running load estimation decoding value.

[0018] In the above-mentioned intelligent detection system for the running load of the conveyor belt, the joint encoding unit is used to: pass the enhanced material property semantic feature vector through a point convolution layer to obtain a first convolution feature vector; pass the first convolution feature vector through a corrected linear unit based on the ReLU function to obtain a first corrected convolution feature vector; pass the first corrected convolution feature vector through a point convolution layer to obtain a second convolution feature vector; pass the second convolution feature vector through a corrected linear unit based on the Sigmoid function to obtain a second corrected convolution feature vector; and fuse the second corrected convolution feature vector with the material three-dimensional geometric feature map to obtain the material property-geometry joint representation feature map.

[0019] The above-mentioned intelligent detection system for the running load of the conveyor belt also includes a training module for training the material property feature extractor based on the void convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model, and the decoder-based running load estimator.

[0020] In the above-mentioned intelligent detection system for the running load of the conveyor belt, the training module includes: a training data acquisition unit, which is used to acquire training data, and the training data includes a training material image collected by a camera, a three-dimensional point cloud model of the training material collected by a laser scanner, and the true value of the running load; a training material property feature extraction unit, which is used to input the training material image into the material property feature extractor based on the void convolutional neural network model to obtain a training material property semantic feature map; a training material property feature attention enhancement unit, which is used to input the training material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain a training enhanced material property semantic feature map; a training feature enhancement compression unit, which is used to perform global maximum pooling along the channel dimension on the training enhanced material property semantic feature map to obtain a training enhanced material property semantic feature vector; a training material geometric feature extraction unit, which is used to input the training material three-dimensional point cloud model into the material geometric feature extractor based on the three-dimensional convolutional neural network model to obtain a training A three-dimensional geometric feature map of a material; a training data joint encoding unit, used to input the training enhanced material property semantic feature vector and the training material three-dimensional geometric feature map into the material property-material geometry joint encoder based on the meta-network model to obtain a training material property-geometry joint representation feature map; a decoding loss unit, used to input the training material property-geometry joint representation feature map into the decoder-based operating load estimator to obtain a decoding loss function value; a predetermined loss function value calculation unit, used to calculate the predetermined loss function value of the training material property-geometry joint representation feature map; a model training unit, used to train the material property feature extractor based on the void convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model and the decoder-based operating load estimator using the weighted sum of the decoding loss function value and the predetermined loss function value as the loss function value.

[0021] Compared with the prior art, the intelligent detection system for the conveyor belt running load provided by the present application uses a camera and a laser scanner to collect images and three-dimensional point cloud models of materials on the conveyor belt in real time, and extracts features from the three-dimensional point cloud model and image of the material through artificial intelligence technology based on deep learning, so as to mine the material's property characteristics and three-dimensional geometric features, and realizes multi-dimensional perception and intelligent estimation of the conveyor belt load by cross-modal joint encoding of the two. In this way, the running load of the conveyor belt can be accurately and quickly detected without direct contact with the material, providing strong support for the optimization of the production process and fault prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 It is a block diagram of an intelligent detection system for conveyor belt running load according to an embodiment of the present application.

[0024] Figure 2 Schematic diagram of the architecture of an intelligent detection system for conveyor belt running load according to an embodiment of the present application.

[0025] Figure 3 It is a block diagram of the material property characteristic enhancement module in the intelligent detection system of the conveyor belt running load according to the embodiment of the present application.

[0026] Figure 4 It is a block diagram of the attention enhancement unit in the intelligent detection system of the conveyor belt running load according to an embodiment of the present application.

[0027] Figure 5 It is a block diagram of an operating load estimation module in an intelligent detection system for conveyor belt operating load according to an embodiment of the present application.

[0028] Figure 6 It is a block diagram of a training module in the intelligent detection system of the conveyor belt running load according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0030] As mentioned in the background technology above, traditional conveyor belt load detection mainly uses contact sensors. Since contact sensors need to be in contact with the conveyor belt and materials for a long time, the sensors are prone to wear and damage, which affects their durability, and they need frequent maintenance and replacement, further increasing maintenance costs and labor costs. In response to the above technical problems, the technical concept of this application is to use cameras and laser scanners to collect images and three-dimensional point cloud models of materials on the conveyor belt in real time, and to extract features from the three-dimensional point cloud model and image of the material through artificial intelligence technology based on deep learning, so as to mine the material's property characteristics and three-dimensional geometric features, and to achieve multi-dimensional perception and intelligent estimation of the conveyor belt load by cross-modal joint encoding of the two. In this way, the operating load of the conveyor belt can be accurately and quickly detected without direct contact with the material, providing strong support for the optimization of the production process and fault prevention.

[0031] Figure 1 It is a block diagram of an intelligent detection system for conveyor belt running load according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of an intelligent detection system for conveyor belt running load according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the intelligent detection system 100 of the conveyor belt running load of the embodiment of the present application, it includes: a conveying material information acquisition module 110, which is used to obtain a material image collected by a camera and a material three-dimensional point cloud model collected by a laser scanner; a material property feature extraction module 120, which is used to extract the material property features in the material image to obtain a material property semantic feature map; a material property feature enhancement module 130, which is used to enhance the attention of the material property semantic feature map to obtain an enhanced material property semantic feature vector; a material geometric feature extraction module 140, which is used to extract the material geometric features in the material three-dimensional point cloud model to obtain a material three-dimensional geometric feature map; an running load estimation module 150, which is used to generate a decoded value of the conveyor belt running load estimation based on the cross-modal joint representation features of the enhanced material property semantic feature vector and the material three-dimensional geometric feature map.

[0032] In the above-mentioned intelligent detection system 100 for the conveyor belt running load, the conveying material information acquisition module 110 is used to obtain the material image collected by the camera and the material three-dimensional point cloud model collected by the laser scanner. It should be understood that the camera can capture the image information of the material on the conveyor belt in real time and provide rich visual data, and the rapid development of computer vision technology provides a powerful tool for the analysis and processing of material image information, making the contactless detection of the conveyor belt running load possible. However, although the image information can reflect the appearance characteristics of the material and obtain the property information of the material, it is difficult to directly quantify its mass and volume, resulting in limitations in the estimation of the conveyor belt running load. For this reason, in the technical solution of the present application, a laser scanner is introduced to collect the three-dimensional spatial data of the material and generate a three-dimensional point cloud model of the material to capture the geometric morphology information of the material, so as to more accurately quantify the mass and volume of the material. In this way, through the collaborative work of the camera and the laser scanner, the multi-dimensional perception of the material is realized, so as to more accurately estimate the running load of the conveyor belt.

[0033] In the intelligent detection system 100 for the conveyor belt running load, the material property feature extraction module 120 is used to extract the material property features in the material image to obtain the material property semantic feature map. In a specific example of the present application, the processing method for extracting the material property features in the material image is to input the material image into a material property feature extractor based on a dilated convolutional neural network model to obtain the material property semantic feature map. A person of ordinary skill in the art should know that the dilated convolutional neural network (DCNN) can increase the receptive field (Receptive Field) without increasing the number of parameters and the amount of calculation by introducing the dilated convolution operation, thereby capturing richer spatial context features. In the technical solution of the present application, the dilated convolutional neural network model is used to perform feature learning on the material image, and the unique spatial receptive field advantage of the spatial convolutional neural network can be used to extract the material property features in the image, such as the color, structure, shape, etc. of the material, thereby reflecting certain physical and chemical properties of the material, and providing an important basis for the load estimation of the conveyor belt.

[0034] In the above-mentioned intelligent detection system 100 for conveyor belt running load, the material property feature enhancement module 130 is used to enhance the attention of the material property semantic feature graph to obtain an enhanced material property semantic feature vector. Figure 3 FIG. 1 is a block diagram of a material property feature enhancement module in an intelligent detection system for conveyor belt operation load according to an embodiment of the present application. Figure 3As shown, the material property feature enhancement module 130 includes: an attention enhancement unit 131, which is used to input the material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain an enhanced material property semantic feature map; a feature enhancement compression unit 132, which is used to perform global maximum pooling along the channel dimension on the enhanced material property semantic feature map to obtain the enhanced material property semantic feature vector.

[0035] Specifically, the attention enhancement unit 131 is used to input the material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain an enhanced material property semantic feature map. It should be understood that, considering that in the process of material property semantic feature extraction, as the network depth deepens, some small details of useful information may be lost, thereby affecting the accuracy of subsequent operating load estimation. Therefore, in the technical solution of the present application, a feature attention enhancement module under the compression-suppression architecture is further introduced to perform feature enhancement processing on the material property semantic feature map. Specifically, the feature attention enhancement module compresses the spatial dimension of the material property semantic feature map to obtain its global channel information, and performs a series of convolution, nonlinear transformation and other operations on its global channel information to mine the multi-scale correlation relationship between information, and generate attention weights, and then based on the generated attention weights, the original material property semantic feature map is weighted channel by channel to achieve the amplification and suppression of each feature channel of the material property semantic feature map, thereby effectively highlighting the useful features of small details in the feature map, and suppressing the useless features such as noise and redundancy therein, and improving the expression ability of the material property semantic feature map.

[0036] Figure 4 FIG. 1 is a block diagram of an attention enhancement unit in an intelligent detection system for conveyor belt running load according to an embodiment of the present application. Figure 4As shown, the attention strengthening unit 131 includes: a global mean pooling subunit 1311, which is used to calculate the global mean of each feature matrix along the channel dimension of the material property semantic feature map to obtain a material property semantic feature compression information representation vector; a channel information association interaction subunit 1312, which is used to perform one-dimensional convolution encoding on the material property semantic feature compression representation vector to obtain a material property semantic feature compression information association representation feature vector; a multi-scale fusion subunit 1313, which is used to cascade the material property semantic feature compression information representation vector and the material property semantic feature compression information association representation feature vector to obtain a material property semantic feature compression information multi-scale representation vector; a multi-scale association feature extraction subunit 1314, which is used to perform one-dimensional convolution encoding on the material property semantic feature compression representation vector and the material property semantic feature compression information association representation feature vector to obtain a material property semantic feature compression information multi-scale representation vector; a multi-scale association feature extraction subunit 1315, which is used to perform one-dimensional convolution encoding on the material property semantic feature compression representation vector and the material property semantic feature compression information association representation feature vector to obtain a material property semantic feature compression information multi-scale representation vector; a multi-scale association feature extraction subunit 1316, which is used to perform one-dimensional convolution encoding on the material property semantic feature compression representation vector and the material property semantic feature compression information association representation feature vector Unit 1314 is used to input the multi-scale representation vector of the compressed information of the semantic feature of material properties into the compressed information feature extraction module based on the multi-layer perceptron model to obtain the multi-scale associated feature vector of the compressed information of the semantic feature of material properties, wherein the multi-layer perceptron model includes two fully connected layers and a SiLU activation function; a normalization subunit 1315 is used to use a Sigmoid function to normalize the multi-scale associated feature vector of the compressed information of the semantic feature of material properties to obtain a material property semantic feature weight vector; a feature selection subunit 1316 is used to perform feature amplification and suppression operations on the material property semantic feature map based on the material property semantic feature weight vector to obtain the enhanced material property semantic feature map.

[0037] More specifically, the feature selection subunit 1316 is used to calculate the corresponding multiplication of each feature value of the material property semantic feature weight vector and each feature matrix along the channel dimension of the material property semantic feature map to obtain the enhanced material property semantic feature map.

[0038] That is, the attention enhancement unit 131 is used to: perform attention selection on the material property semantic feature map according to the following feature enhancement formula to obtain the enhanced material property semantic feature map, wherein the feature enhancement formula is: ;in, A semantic feature graph representing the material properties, The first The coordinates in the channel are The eigenvalues ​​of a point, Indicates the height of the semantic feature graph of the material property, Indicates the width of the semantic feature graph of the material property, is the first vector representing the semantic feature compression information of material properties. eigenvalues, The semantic feature compression information representation vector of the material property is represented. represents a one-dimensional convolution operation, Indicates cascade operation, Represents the multi-scale representation vector of the semantic features of material properties and compressed information, represents a multi-layer perceptron, represents the sigmoid function, represents channel-wise dot product, A semantic feature graph representing the enhanced material properties.

[0039] Specifically, the feature enhancement compression unit 132 is used to perform global maximum pooling along the channel dimension on the enhanced material property semantic feature map to obtain the enhanced material property semantic feature vector. That is, in order to further improve the feature representativeness and discrimination of the enhanced material property semantic feature map and simplify its information representation, in the technical solution of the present application, the enhanced material property semantic feature map is subjected to global maximum pooling along the channel dimension to extract the maximum value on each channel as the feature representation of the channel, thereby effectively integrating the information of each channel, highlighting important features, reducing redundant information, and obtaining a more efficient and compact enhanced material property semantic feature vector.

[0040] In the above-mentioned intelligent detection system 100 for conveyor belt running load, the material geometric feature extraction module 140 is used to extract the material geometric features in the material three-dimensional point cloud model to obtain the material three-dimensional geometric feature map. In a specific example of the present application, the processing method for extracting the material geometric features in the material three-dimensional point cloud model is to input the material three-dimensional point cloud model into a material geometric feature extractor based on a three-dimensional convolutional neural network model to obtain the material three-dimensional geometric feature map. That is, in order to fully capture the three-dimensional spatial information and geometric features of the material, a three-dimensional convolutional neural network model is used to perform feature mining on the material three-dimensional point cloud model. It should be known to those skilled in the art that a three-dimensional convolutional neural network (3D Convolutional Neural Network, 3D CNN) has unique advantages in processing three-dimensional data. It can capture the spatial relationship of the material in different directions by using a three-dimensional convolution kernel to perform convolution operations in three directions of the material three-dimensional point cloud model at the same time, and extract the geometric features of the material such as volume, shape, and surface roughness, thereby reflecting the spatial occupancy of the material and providing accurate and comprehensive geometric information basis for conveyor belt load estimation.

[0041] In the above-mentioned intelligent detection system 100 for conveyor belt running load, the running load estimation module 150 is used to generate a decoded value of the conveyor belt running load estimation based on the cross-modal joint representation features of the enhanced material property semantic feature vector and the material three-dimensional geometric feature map. Figure 5FIG. 1 is a block diagram of an operation load estimation module in an intelligent detection system for the operation load of a conveyor belt according to an embodiment of the present application. Figure 5 As shown, the operating load estimation module 150 includes: a joint encoding unit 151, used for inputting the enhanced material property semantic feature vector and the material three-dimensional geometric feature map into a material property-material geometry joint encoder based on a meta-network model to obtain a material property-geometry joint representation feature map; a decoding unit 152, used for inputting the material property-geometry joint representation feature map into an operating load estimator based on a decoder to obtain the operating load estimation decoded value.

[0042] Specifically, the joint encoding unit 151 is used to input the enhanced material property semantic feature vector and the material three-dimensional geometric feature map into the material property-material geometry joint encoder based on the meta-network model to obtain the material property-geometry joint representation feature map. It should be understood that the enhanced material property semantic feature vector mainly contains the material's color, texture, material and other property information, while the material three-dimensional geometric feature map reflects the material's three-dimensional shape, size, stacking method and other geometric feature information. By jointly analyzing the two, the weight and volume of the material can be effectively estimated, thereby accurately evaluating the running load of the conveyor belt. Therefore, in order to achieve an effective fusion of the two, in the technical solution of the present application, a material property-material geometry joint encoder based on the meta-network model is introduced to process the enhanced material property semantic feature vector and the material three-dimensional geometric feature map. Specifically, the material property-material geometry joint encoder can dynamically adjust network parameters and structures through the self-learning mechanism of the Meta-Network model, and adaptively fusion encode the enhanced material property semantic feature vector and the material three-dimensional geometric feature map. Through a series of complex nonlinear transformations and feature mappings, it can achieve complementarity and enhancement between the two while maintaining their respective feature advantages, so as to generate a material property-geometry joint representation feature map with stronger representation ability and higher accuracy, and form a comprehensive description of material characteristics.

[0043] In a specific example of the present application, the joint encoding unit 151 is used to: pass the enhanced material property semantic feature vector through a point convolution layer to obtain a first convolution feature vector; pass the first convolution feature vector through a corrected linear unit based on a ReLU function to obtain a first corrected convolution feature vector; pass the first corrected convolution feature vector through a point convolution layer to obtain a second convolution feature vector; pass the second convolution feature vector through a corrected linear unit based on a Sigmoid function to obtain a second corrected convolution feature vector; and fuse the second corrected convolution feature vector with the material three-dimensional geometric feature map to obtain the material property-geometry joint representation feature map.

[0044] Specifically, the decoding unit 152 is used to input the material property-geometry joint representation feature map into the decoder-based operation load estimator to obtain the operation load estimation decoding value. That is, the material property-geometry joint representation feature map is decoded using the powerful decoding regression capability of the decoder model, and the high-dimensional features therein are mapped to a low-dimensional numerical space to generate a decoding value directly related to the conveyor belt operation load. Among them, the decoder uses multi-layer convolution and deconvolution operations inside, and by learning the context information in the material property-geometry joint representation feature map, the weights of the influence of material properties and geometric features on the conveyor belt operation load are gradually parsed, thereby achieving accurate estimation of the conveyor belt operation load.

[0045] It should be understood that before using the above neural network model, it needs to be trained. That is to say, in the intelligent detection system of the conveyor belt running load of the present application, it also includes a training module for training the material property feature extractor based on the hole convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model, and the running load estimator based on the decoder.

[0046] Figure 6 FIG. 1 is a block diagram of a training module in an intelligent detection system for conveyor belt running load according to an embodiment of the present application. Figure 6As shown, the training module 200 includes: a training data acquisition unit 210, which is used to acquire training data, and the training data includes a training material image collected by a camera, a training material three-dimensional point cloud model collected by a laser scanner, and a true value of the operating load; a training material property feature extraction unit 220, which is used to input the training material image into the material property feature extractor based on the void convolutional neural network model to obtain a training material property semantic feature map; a training material property feature attention enhancement unit 230, which is used to input the training material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain a training enhanced material property semantic feature map; a training feature enhancement compression unit 240, which is used to perform global maximum pooling along the channel dimension on the training enhanced material property semantic feature map to obtain a training enhanced material property semantic feature vector; a training material geometric feature extraction unit 250, which is used to input the training material three-dimensional point cloud model into the material geometric feature extractor based on the three-dimensional convolutional neural network model to obtain a training material three-dimensional geometric feature map. a training data joint encoding unit 260, used to input the training enhanced material property semantic feature vector and the training material three-dimensional geometric feature map into the material property-material geometry joint encoder based on the meta-network model to obtain the training material property-geometry joint representation feature map; a decoding loss unit 270, used to input the training material property-geometry joint representation feature map into the decoder-based operating load estimator to obtain a decoding loss function value; a predetermined loss function value calculation unit 280, used to calculate the predetermined loss function value of the training material property-geometry joint representation feature map; a model training unit 290, used to train the material property feature extractor based on the void convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model and the decoder-based operating load estimator using the weighted sum of the decoding loss function value and the predetermined loss function value as the loss function value.

[0047] Here, the training enhanced material property semantic feature vector is used to represent the image semantic features of the training material image obtained by dilated convolution coding, feature attention enhancement under the compression-suppression architecture, and global maximum pooling along the channel dimension for representing the material properties. When the training enhanced material property semantic feature vector and the training material three-dimensional geometric feature map are input into the material property-material geometry joint encoder based on the meta-network model, the obtained training material property-geometry joint representation feature map has not only the local geometric features of the material three-dimensional point cloud model but also the channel mixing of the image semantic features of the material properties along the channel dimension. Although this will make the training material property-geometry joint representation feature map have a multi-modal mixed local feature richness representation in its channel dimension, it will also make the training material property-geometry joint representation feature map as a feature distribution as a whole have decoding regression recognition difficulties based on the complex feature manifold structure, thereby affecting the decoding training efficiency.

[0048] Therefore, in the technical solution of the present application, a predetermined loss function other than the classification loss function is further introduced in the model training process, that is, the model is trained by gradient back propagation based on the loss function value, specifically including the following steps: expanding the training material property-geometry joint representation feature map into a training material property-geometry joint representation feature vector; calculating a first material property-geometry joint weight matrix and a second material property-geometry joint weight matrix based on the training material property-geometry joint representation feature vector, wherein the first material property-geometry joint weight matrix and the second material property-geometry joint weight matrix are The eigenvalues ​​of the positions are the first Eigenvalues ​​and The mean and half of the absolute value of the difference of the eigenvalues; perform query matrix multiplication of the training material property-geometry joint representation eigenvector with the first and second material property-geometry joint weight matrices respectively to obtain the first material property-geometry joint intermediate vector and the second material property-geometry joint intermediate vector; calculate the vector inner product of the first material property-geometry joint intermediate vector and the second material property-geometry joint intermediate vector to obtain the first material property-geometry joint loss term; perform matrix multiplication of the first material property-geometry joint weight matrix and the second material property-geometry joint weight matrix, and calculate the multiplication result matrix norm to obtain a second material property-geometry joint loss term; subtract the product of a predetermined weight hyperparameter and the second material property-geometry joint loss term from the first material property-geometry joint loss term to obtain the predetermined loss function value; based on the weighted sum of the predetermined loss function value and the decoding loss function value, optimize the model parameters through gradient back propagation.

[0049] That is, the predetermined loss function value of the training material property-geometry joint representation feature vector is calculated using the following material property-geometry joint loss function, wherein the material property-geometry joint loss function is specifically expressed as: ;in, represents the training material property-geometry joint representation feature vector after the training material property-geometry joint representation feature graph is expanded, The first one represents the training material property-geometry joint representation feature vector eigenvalues, The first one represents the training material property-geometry joint representation feature vector eigenvalues, The first one represents the training material property-geometry joint representation feature vector The eigenvalues ​​and The mean of the eigenvalues, is the first material property-geometry joint weight matrix, The first one represents the training material property-geometry joint representation feature vector The eigenvalues ​​and One-half the absolute value of the difference of the eigenvalues, is the second material property-geometry joint weight matrix, represents the matrix multiplication operation, is the predetermined weight hyperparameter, Represents the matrix norm, represents the predetermined loss function value.

[0050] Here, the material property-geometry joint loss function uses the structured feature representation of the short-range-long-range cross-scale detail links of the training material property-geometry joint representation feature vector to perform the query composition of the detail inner product space within the training material property-geometry joint representation feature vector, so as to approximate the low-rank independent observable composition of the linked detail composition provided by the structured detail interaction of the training material property-geometry joint representation feature vector. In this way, by training with the material property-geometry joint loss function, the distributed detail groups of the training material property-geometry joint representation feature vector can be used to perform detail group decomposition based on the complexity of the details, so as to promote the decoding regression decomposition and recognition of the complex feature structure of the training material property-geometry joint representation feature vector, and improve the decoding training efficiency.

[0051] In summary, the intelligent detection system of the conveyor belt running load according to the embodiment of the present application is explained, which uses a camera and a laser scanner to collect images and three-dimensional point cloud models of materials on the conveyor belt in real time, and extracts features from the three-dimensional point cloud model and image of the material through artificial intelligence technology based on deep learning, so as to mine the material's property characteristics and three-dimensional geometric features, and realizes multi-dimensional perception and intelligent estimation of the conveyor belt load by cross-modal joint encoding of the two. In this way, the running load of the conveyor belt can be accurately and quickly detected without direct contact with the material, providing strong support for the optimization of the production process and fault prevention.

[0052] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0053] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0054] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0055] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent detection system for conveyor belt running load, characterized in that: include: The conveying material information acquisition module is used to acquire the material image acquired by the camera and the material three-dimensional point cloud model acquired by the laser scanner; A material property feature extraction module, used to extract material property features in the material image to obtain a material property semantic feature map; A material property feature enhancement module, used for enhancing the attention of the material property semantic feature map to obtain an enhanced material property semantic feature vector; A material geometric feature extraction module, used to extract the material geometric features in the three-dimensional point cloud model of the material to obtain a three-dimensional geometric feature map of the material; An operation load estimation module, used for generating an estimated decoded value of the operation load of the conveyor belt based on the cross-modal joint representation features of the enhanced material property semantic feature vector and the material three-dimensional geometric feature map; Wherein, the material property characteristics enhancement module includes: An attention enhancement unit, used for inputting the material property semantic feature map into a feature attention enhancement module under a compression-suppression architecture to obtain an enhanced material property semantic feature map; A feature enhancement compression unit, used for performing global maximum pooling along the channel dimension on the enhanced material property semantic feature map to obtain the enhanced material property semantic feature vector; Wherein, the attention strengthening unit comprises: A global mean pooling subunit, used to calculate the global mean of each feature matrix along the channel dimension of the material property semantic feature map to obtain a material property semantic feature compression information representation vector; A channel information association interaction subunit is used to perform one-dimensional convolution encoding on the material property semantic feature compression information representation vector to obtain a feature vector representing the association between the material property semantic feature compression information; A multi-scale fusion sub-unit, used for cascading the material property semantic feature compression information representation vector and the material property semantic feature compression information association representation feature vector to obtain a material property semantic feature compression information multi-scale representation vector; A multi-scale correlation feature extraction subunit is used to input the multi-scale representation vector of the compressed information of the semantic feature of the material property into a compressed information feature extraction module based on a multi-layer perceptron model to obtain a multi-scale correlation feature vector of the compressed information of the semantic feature of the material property, wherein the multi-layer perceptron model includes two fully connected layers and a SiLU activation function; A normalization subunit, used for performing a normalization operation on the multi-scale associated feature vector of the material property semantic feature compression information using a Sigmoid function to obtain a material property semantic feature weight vector; The feature selection subunit is used to perform feature amplification and suppression operations on the material property semantic feature map based on the material property semantic feature weight vector to obtain the enhanced material property semantic feature map.

2. The intelligent detection system for conveyor belt running load according to claim 1 is characterized in that: The material property feature extraction module is used to: The material image is input into a material property feature extractor based on a hole convolutional neural network model to obtain the material property semantic feature map.

3. The intelligent detection system for conveyor belt running load according to claim 2 is characterized in that: The feature selection subunit is used to: The enhanced material property semantic feature map is obtained by calculating and multiplying the corresponding feature values ​​of the material property semantic feature weight vector and the feature matrices along the channel dimension of the material property semantic feature map.

4. The intelligent detection system for conveyor belt running load according to claim 3 is characterized in that: The material geometric feature extraction module is used to: The three-dimensional point cloud model of the material is input into a material geometric feature extractor based on a three-dimensional convolutional neural network model to obtain the three-dimensional geometric feature map of the material.

5. The intelligent detection system for conveyor belt running load according to claim 4 is characterized in that: The operation load estimation module comprises: A joint encoding unit, used for inputting the enhanced material property semantic feature vector and the material three-dimensional geometric feature map into a material property-material geometry joint encoder based on a meta-network model to obtain a material property-geometry joint representation feature map; A decoding unit is used to input the material property-geometry joint representation feature diagram into a decoder-based operating load estimator to obtain the operating load estimation decoded value.

6. The intelligent detection system for conveyor belt running load according to claim 5 is characterized in that: The joint coding unit is used for: Passing the enhanced material property semantic feature vector through a point convolution layer to obtain a first convolution feature vector; Passing the first convolution feature vector through a rectified linear unit based on a ReLU function to obtain a first rectified convolution feature vector; Passing the first modified convolutional feature vector through a point convolution layer to obtain a second convolutional feature vector; Passing the second convolution feature vector through a rectified linear unit based on a Sigmoid function to obtain a second rectified convolution feature vector; The second modified convolution feature vector is fused with the material three-dimensional geometric feature map to obtain the material property-geometry joint representation feature map.

7. The intelligent detection system for conveyor belt running load according to claim 6 is characterized in that: It also includes a training module for training the material property feature extractor based on the void convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model, and the decoder-based operating load estimator.

8. The intelligent detection system for conveyor belt running load according to claim 7 is characterized in that: The training module comprises: A training data acquisition unit, used to acquire training data, wherein the training data includes a training material image acquired by a camera, a three-dimensional point cloud model of the training material acquired by a laser scanner, and a true value of an operating load; A training material property feature extraction unit, used for inputting the training material image into the material property feature extractor based on the hole convolutional neural network model to obtain a training material property semantic feature map; A training material property feature attention enhancement unit, used for inputting the training material property semantic feature map into the feature attention enhancement module under the compression-suppression architecture to obtain a training enhanced material property semantic feature map; A training feature enhancement compression unit, used for performing global maximum pooling along the channel dimension on the training enhanced material property semantic feature map to obtain a training enhanced material property semantic feature vector; A training material geometric feature extraction unit, used for inputting the three-dimensional point cloud model of the training material into the material geometric feature extractor based on the three-dimensional convolutional neural network model to obtain a three-dimensional geometric feature map of the training material; A training data joint encoding unit, used for inputting the training enhanced material property semantic feature vector and the training material three-dimensional geometric feature map into the material property-material geometry joint encoder based on the meta-network model to obtain a training material property-geometry joint representation feature map; A decoding loss unit, used for inputting the training material property-geometry joint representation feature map into the decoder-based operation load estimator to obtain a decoding loss function value; A predetermined loss function value calculation unit, used to calculate a predetermined loss function value of the training material property-geometry joint representation feature map; A model training unit is used to train the material property feature extractor based on the void convolutional neural network model, the feature attention enhancement module under the compression-suppression architecture, the material geometry feature extractor based on the three-dimensional convolutional neural network model, the material property-material geometry joint encoder based on the meta-network model, and the decoder-based operating load estimator by using the weighted sum of the decoding loss function value and the predetermined loss function value as the loss function value.

Images