Belt tension control system and method based on multi-sensor data fusion

Abstract

Disclosed are a belt tensioning control system and method based on multi-sensor data fusion, which extracts multi-scale neighborhood correlation features of weight data of transported raw coal collected by a belt electronic scale at multiple predetermined time points within a predetermined time period in the time dimension through a multi-scale neighborhood feature extraction module; extracts dynamic features focusing on feature distribution information of raw coal on the belt plane in the transportation monitoring image through a first convolutional neural network model using spatial attention and a second convolutional neural network model using a three-dimensional convolution kernel; and then performs adaptive control of the belt tensioning force at the current time point based on the fusion feature information of the two. In this way, the belt tensioning force can be intelligently controlled in real time and accurately according to the actual situation, and the belt tensioning force can be reduced to meet the transportation demand while avoiding excessive belt tensioning force and reducing the service life.

Description

Technical Field

[0001] This application relates to the field of intelligent control technology, and more specifically, to a belt tensioning control system and method based on multi-sensor data fusion. Background Technology

[0002] Excavators are among the most typical, complex, and widely used engineering machines, playing a vital role in construction projects in industry and civil engineering, transportation, water conservancy and hydropower, mining, and military engineering.

[0003] Currently, during the operation of excavators, raw coal is generally transported via the excavating arm belt. However, the amount and size of raw coal excavated each time are variable, which causes the flow rate and weight of raw coal on the belt to change in real time. Consequently, the belt tension also changes with the flow rate of raw coal on the belt. Therefore, in order to avoid the belt from being too tight and thus reducing its lifespan, adaptive control of the belt tension is required.

[0004] Therefore, an optimized belt tension control system is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a belt tension control system and method based on multi-sensor data fusion. It extracts multi-scale neighborhood feature features in the time dimension from the weight data of transported raw coal collected by a belt scale at multiple predetermined time points within a predetermined time period using a multi-scale neighborhood feature extraction module. It then extracts dynamic features focusing on the feature distribution information of raw coal on the belt plane from the transportation monitoring image using a first convolutional neural network model with spatial attention and a second convolutional neural network model using a three-dimensional convolutional kernel. Finally, it performs adaptive control of the belt tension at the current time point based on the fused feature information of these two models. This allows for real-time and accurate intelligent control of the belt tension according to actual conditions, ensuring that the belt tension meets transportation requirements while avoiding excessive belt tension that could shorten the belt's lifespan.

[0006] According to one aspect of this application, a belt tensioning control system based on multi-sensor data fusion is provided, comprising:

[0007] The multi-sensor monitoring unit is used to acquire the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as the transport monitoring images at the multiple predetermined time points.

[0008] The weight data encoding unit is used to arrange the weight data of the transported raw coal collected by the belt electronic scale at the multiple predetermined time points into a weight input vector according to the time dimension, and then obtain the weight feature vector through the multi-scale neighborhood feature extraction module.

[0009] The monitoring image encoding unit is used to obtain multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention to process the transportation monitoring images at each predetermined time point;

[0010] The monitoring dynamic coding unit is used to arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map.

[0011] The dimensionality reduction unit is used to perform global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector.

[0012] A multi-sensor feature fusion unit is used to fuse the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector; and

[0013] The belt tension control result generation unit is used to pass the classification feature vector through a classifier to obtain a classification result, which is used to indicate whether the belt tension should be increased or decreased at the current time point.

[0014] According to another aspect of this application, a belt tension control method based on multi-sensor data fusion is provided, comprising:

[0015] The weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as the transportation monitoring images at the multiple predetermined time points, are obtained.

[0016] The weight data of the transported raw coal collected by the belt scale at the multiple predetermined time points are arranged into a weight input vector according to the time dimension, and then the weight feature vector is obtained by the multi-scale neighborhood feature extraction module.

[0017] The transportation monitoring images at each predetermined time point are used to obtain multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention.

[0018] After arranging the multiple transportation monitoring feature matrices into a three-dimensional input tensor, a second convolutional neural network model with three-dimensional convolutional kernels is used to obtain a transportation monitoring feature map.

[0019] Global mean pooling is performed on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector;

[0020] The weight feature vector and the transportation monitoring feature vector are fused to obtain a classification feature vector; and

[0021] The classification feature vector is passed through a classifier to obtain a classification result, which is used to indicate whether the belt tension should be increased or decreased at the current time point.

[0022] Compared with existing technologies, the belt tension control system and method based on multi-sensor data fusion provided in this application extracts multi-scale neighborhood correlation features in the time dimension of the weight data of transported raw coal collected by belt scales at multiple predetermined time points within a predetermined time period through a multi-scale neighborhood feature extraction module; it extracts dynamic features of the feature distribution information of raw coal on the belt plane from the transportation monitoring image by using a first convolutional neural network model with spatial attention and a second convolutional neural network model using a three-dimensional convolutional kernel; and then performs adaptive control of belt tension at the current time point based on the fused feature information of these two models. In this way, the belt tension can be intelligently controlled in real time and accurately according to the actual situation, thereby ensuring that the belt tension meets the transportation requirements while avoiding excessive belt tension that would reduce belt life. Attached Figure Description

[0023] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 This is an application scenario diagram of a belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0025] Figure 2 This is a block diagram of a belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0026] Figure 3 This is a block diagram of the weight data encoding unit in a belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0027] Figure 4 This is a block diagram of the monitoring dynamic coding unit in the belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0028] Figure 5 This is a block diagram of the dimension reduction unit in the belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0029] Figure 6 This is a block diagram of the local feature enhancement subunit in a belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application.

[0030] Figure 7 This is a block diagram of the belt tension control result generation unit in the belt tension control system based on multi-sensor data fusion according to an embodiment of this application.

[0031] Figure 8 This is a flowchart of a belt tension control method based on multi-sensor data fusion according to an embodiment of this application.

[0032] Figure 9 This is a schematic diagram of the system architecture of a belt tension control method based on multi-sensor data fusion according to an embodiment of this application. Detailed Implementation

[0033] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0034] Scene Overview

[0035] As mentioned above, excavators are among the most typical, complex, and widely used engineering machines, playing a vital role in construction of industrial and civil buildings, transportation, water conservancy and hydropower projects, mining, and military engineering.

[0036] Currently, during excavator operation, raw coal is typically transported via a belt conveyor on the excavating arm. However, the amount and size of raw coal excavated each time are variable, causing the flow rate and weight of raw coal on the belt to change in real time. Consequently, the belt tension also varies with the flow rate of raw coal. Therefore, to prevent the belt from becoming too tense and reducing its lifespan, adaptive belt tension control is required. Thus, an optimized belt tension control system is desired.

[0037] Currently, deep learning and neural networks have been widely applied in fields such as computer vision, natural language processing, and speech signal processing. Furthermore, deep learning and neural networks have demonstrated near-human or even superior performance in areas such as image classification, object detection, semantic segmentation, and text translation.

[0038] In recent years, the development of deep learning and neural networks has provided new ideas and solutions for intelligent control of belt tensioning.

[0039] Accordingly, in the technical solution of this application, it is expected that a belt scale will be installed on the excavator boom belt to measure the excavation flow rate in real time, and further intelligent control algorithms will be added to achieve the setting and adaptive adjustment of belt tension. However, considering that raw coal accumulates in different forms at different positions on the belt, and the belt scale can only measure the weight data of the transported raw coal, but cannot obtain the distribution characteristics of the raw coal on the belt plane, it is expected that belt tension control will be carried out by combining both aspects and using data fusion based on multiple sensors, so that the belt tension can meet the transportation requirements on the one hand, and on the other hand, the belt life will not be reduced due to excessive tension.

[0040] Specifically, in the technical solution of this application, a deep learning-based artificial intelligence control algorithm is used to extract the multi-scale neighborhood correlation features of the weight data of the transported raw coal in the time dimension, and to extract the dynamic features of the feature distribution information of the raw coal focused on the belt plane in the raw coal transportation monitoring image. Then, based on the fused feature information of these two, adaptive control of the belt tension at the current time point is performed. In this way, the belt tension can be intelligently controlled in real time and accurately according to the actual situation, thereby ensuring that the belt tension meets the transportation requirements while avoiding excessive belt tension that would reduce the belt's lifespan.

[0041] More specifically, in the technical solution of this application, firstly, the weight data of the transported raw coal collected by a belt conveyor at multiple predetermined time points within a predetermined time period is obtained. Then, considering that the weight data of the transported raw coal has different pattern characteristics at different time spans within the predetermined time period, in order to accurately extract the dynamic change feature information of the weight data of the transported raw coal, in the technical solution of this application, the weight data of the transported raw coal collected by the belt conveyor at the multiple predetermined time points is arranged into a weight input vector according to the time dimension, and then feature mining is performed through a multi-scale neighborhood feature extraction module to extract the multi-scale neighborhood association features of the weight data of the transported raw coal at different time spans, thereby obtaining a weight feature vector.

[0042] Furthermore, considering that raw coal accumulates in different forms at different locations on the conveyor belt, and that the conveyor belt scale can only measure the weight of the transported raw coal and cannot obtain the distribution characteristics of the raw coal on the conveyor belt plane, the technical solution of this application further acquires transport monitoring images at multiple predetermined time points using a camera, and performs feature mining on the transport monitoring images at each predetermined time point using a convolutional neural network model that excels in extracting latent features of images, thereby extracting the latent feature distribution information of the raw coal on the conveyor belt. However, considering that when the excavator's boom conveyor belt transports raw coal, there will be a large amount of raw coal interference feature information at other locations in the transport monitoring images, and given that the attention mechanism can select the focus position, in order to accurately extract the raw coal distribution feature information on the conveyor belt for adaptive adjustment of the belt tension, a first convolutional neural network model with spatial attention is used to perform feature mining on the transport monitoring images at each predetermined time point to extract the latent feature distribution information of the raw coal focused on the conveyor belt plane in each raw coal transport monitoring image, thereby obtaining multiple transport monitoring feature matrices.

[0043] Next, for the multiple transportation monitoring feature matrices that have implicit feature distribution information of raw coal on the conveyor plane at multiple predetermined time points, the multiple transportation monitoring feature matrices are further arranged into a three-dimensional input tensor and then feature extraction is performed using a second convolutional neural network model with a three-dimensional convolutional kernel to extract the dynamic features of the feature distribution information of raw coal on the conveyor plane, thereby obtaining a transportation monitoring feature map.

[0044] Then, global mean pooling is performed on each feature matrix along the channel dimension of the transportation monitoring feature map to reduce the dimensionality of the implicit features of the raw coal distribution on the conveyor belt at each time point in the time channel dimension, thereby obtaining the transportation monitoring feature vector. In this way, the dynamic feature information of the implicit feature distribution of raw coal on the conveyor belt can be preserved while performing data dimensionality reduction, thereby improving the accuracy of subsequent classification.

[0045] Specifically, in the technical solution of this application, when performing global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector, it is desirable to enhance the correlation between the various feature matrices so that the transportation monitoring feature vector can better express the overall feature distribution of the transportation monitoring feature map while expressing the feature distribution along the channel dimension of the transportation monitoring feature map.

[0046] Therefore, if each feature matrix is ​​considered as a local feature distribution, what needs to be improved is the explicit correlation between each local feature distribution and the global feature distribution. Specifically, this can be achieved by setting predetermined weights as hyperparameters for each feature matrix. However, since the weights as hyperparameters need to be obtained during model training, this increases the training burden on the model. Therefore, the predetermined weights can be obtained by calculating the multi-distribution binary classification continuity factor of multiple feature matrices, which is expressed as:

[0047]

[0048] softmaxv(M) = [p(M), 1 - p(M)]

[0049] Among them, M i M is the i-th characteristic matrix. r It is a reference matrix based on the multiple feature matrices, for example, it can be set as the mean matrix of the multiple feature matrices, and p(M) represents the probability value obtained by the feature matrix through the classifier.

[0050] In other words, to avoid the difficulty in converging towards the target classification domain due to excessive fragmentation of the decision boundaries corresponding to the local feature distributions of each feature matrix in a feature distribution-based classification task, the classification continuity factor of the local feature distribution of each feature matrix relative to the global feature distribution is predicted by calculating the class probability offset distribution modulus information of the binary classification of the local feature distribution of each feature matrix relative to the global average feature distribution. Thus, by using this as a weight to weight the multiple feature matrices, the optimization of hyperparameters during training can be transformed from backpropagation into a classification problem based on the binary classification of multiple distributions, thereby improving the representation effect of the transportation monitoring feature vector on the overall feature distribution of the transportation monitoring feature map.

[0051] Furthermore, by fusing the feature information from the weight feature vector and the transportation monitoring feature vector, and using this as a classification feature vector for classification processing by a classifier, a classification result indicating whether the belt tension should be increased or decreased at the current time point can be obtained. This allows for real-time and accurate adaptive control of the belt tension based on the actual situation of the raw coal on the belt, ensuring that the belt tension meets transportation requirements while avoiding excessive tension that could shorten the belt's lifespan.

[0052] Based on this, this application provides a belt tensioning control system based on multi-sensor data fusion, comprising: a multi-sensor monitoring unit, used to acquire weight data of transported raw coal collected by a belt scale at multiple predetermined time points within a predetermined time period, as well as transport monitoring images at the multiple predetermined time points; a weight data encoding unit, used to arrange the weight data of transported raw coal collected by the belt scale at the multiple predetermined time points into a weight input vector according to the time dimension, and then obtain a weight feature vector through a multi-scale neighborhood feature extraction module; and a monitoring image encoding unit, used to encode multiple transport monitoring images at each predetermined time point by using a first convolutional neural network model with spatial attention. The system comprises: a transport monitoring feature matrix; a monitoring dynamic encoding unit, used to arrange the multiple transport monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transport monitoring feature map; a dimensionality reduction unit, used to perform global mean pooling on each feature matrix along the channel dimension of the transport monitoring feature map to obtain a transport monitoring feature vector; a multi-sensor feature fusion unit, used to fuse the weight feature vector and the transport monitoring feature vector to obtain a classification feature vector; and a belt tension control result generation unit, used to pass the classification feature vector through a classifier to obtain a classification result, the classification result being used to indicate whether the belt tension should be increased or decreased at the current time point.

[0053] Figure 1 This is an application scenario diagram of a belt tension control system based on multi-sensor data fusion according to an embodiment of this application. For example... Figure 1 As shown, in this application scenario, firstly, the belt-mounted electronic scale (e.g., [missing information]) acquires data from multiple predetermined time points within a predetermined time period. Figure 1 The weight data of the transported raw coal collected by T (as shown in the figure) Figure 1 C1 as shown in the diagram); and, by a camera (e.g., Figure 1 The transport monitoring images (e.g., M) collected at the multiple predetermined time points as shown in the figure. Figure 1 (as shown in C2); then, the acquired weight data and transportation monitoring images are input into a server deployed with a belt tension control algorithm based on multi-sensor data fusion (e.g., Figure 1 As shown in S), the server is able to process the weight data and the transportation monitoring image using a belt tension control algorithm based on multi-sensor data fusion to generate a classification result indicating whether the belt tension should be increased or decreased at the current time point.

[0054] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0055] Exemplary System

[0056] Figure 2 This is a block diagram of a belt tension control system based on multi-sensor data fusion according to an embodiment of this application. Figure 2 As shown, the belt tensioning control system 100 based on multi-sensor data fusion according to an embodiment of this application includes: a multi-sensor monitoring unit 110, used to acquire weight data of transported raw coal collected by a belt scale at multiple predetermined time points within a predetermined time period, as well as transport monitoring images at the multiple predetermined time points; a weight data encoding unit 120, used to arrange the weight data of transported raw coal collected by the belt scale at the multiple predetermined time points into a weight input vector according to the time dimension, and then obtain a weight feature vector through a multi-scale neighborhood feature extraction module; and a monitoring image encoding unit 130, used to encode multiple transport images by using a first convolutional neural network model with spatial attention for each predetermined time point. The system includes: a monitoring feature matrix; a monitoring dynamic encoding unit 140, used to arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map; a dimensionality reduction unit 150, used to perform global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain a transportation monitoring feature vector; a multi-sensor feature fusion unit 160, used to fuse the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector; and a belt tension control result generation unit 170, used to pass the classification feature vector through a classifier to obtain a classification result, the classification result being used to indicate whether the belt tension should be increased or decreased at the current time point.

[0057] Specifically, in this embodiment, the multi-sensor monitoring unit 110 is used to acquire weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as transportation monitoring images at the multiple predetermined time points. As mentioned above, excavators are among the most typical, complex, and widely used engineering machines, playing a vital role in construction in industrial and civil buildings, transportation, water conservancy and hydropower projects, mining, and military engineering.

[0058] Currently, during excavator operation, raw coal is typically transported via a belt conveyor on the excavating arm. However, the amount and size of raw coal excavated each time are variable, causing the flow rate and weight of raw coal on the belt to change in real time. Consequently, the belt tension also varies with the flow rate of raw coal. Therefore, to prevent the belt from becoming too tense and reducing its lifespan, adaptive belt tension control is required. Thus, an optimized belt tension control system is desired.

[0059] Currently, deep learning and neural networks have been widely applied in fields such as computer vision, natural language processing, and speech signal processing. Furthermore, deep learning and neural networks have demonstrated near-human or even superior performance in areas such as image classification, object detection, semantic segmentation, and text translation.

[0060] In recent years, the development of deep learning and neural networks has provided new ideas and solutions for intelligent control of belt tensioning.

[0061] Accordingly, in the technical solution of this application, it is expected that a belt scale will be installed on the excavator boom belt to measure the excavation flow rate in real time, and further intelligent control algorithms will be added to achieve the setting and adaptive adjustment of belt tension. However, considering that raw coal accumulates in different forms at different positions on the belt, and the belt scale can only measure the weight data of the transported raw coal, but cannot obtain the distribution characteristics of the raw coal on the belt plane, it is expected that belt tension control will be carried out by combining both aspects and using data fusion based on multiple sensors, so that the belt tension can meet the transportation requirements on the one hand, and on the other hand, the belt life will not be reduced due to excessive tension.

[0062] Specifically, in the technical solution of this application, a deep learning-based artificial intelligence control algorithm is used to extract the multi-scale neighborhood correlation features of the weight data of the transported raw coal in the time dimension, and to extract the dynamic features of the feature distribution information of the raw coal focused on the belt plane in the raw coal transportation monitoring image. Then, based on the fused feature information of these two, adaptive control of the belt tension at the current time point is performed. In this way, the belt tension can be intelligently controlled in real time and accurately according to the actual situation, thereby ensuring that the belt tension meets the transportation requirements while avoiding excessive belt tension that would reduce the belt's lifespan.

[0063] More specifically, in the technical solution of this application, firstly, the weight data of the transported raw coal collected by the belt conveyor scale at multiple predetermined time points within a predetermined time period, as well as the transport monitoring images at the multiple predetermined time points, are acquired. In this way, the weight data of the transported raw coal measured by the belt conveyor scale and the distribution characteristics of the raw coal on the belt plane can be combined. In belt tension control, both are integrated using multi-sensor data fusion to achieve tension control, ensuring that the belt tension meets transportation requirements while preventing a reduction in belt life due to excessive tension.

[0064] Specifically, in this embodiment, the weight data encoding unit 120 is used to arrange the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points into a weight input vector according to the time dimension, and then pass it through a multi-scale neighborhood feature extraction module to obtain a weight feature vector. Then, considering that the weight data of the transported raw coal has different pattern characteristics at different time spans within the predetermined time period, in order to accurately extract the dynamic change feature information of the weight data of the transported raw coal, in the technical solution of this application, the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points is arranged into a weight input vector according to the time dimension, and then feature mining is performed through a multi-scale neighborhood feature extraction module to extract the multi-scale neighborhood association features of the weight data of the transported raw coal at different time spans, thereby obtaining a weight feature vector.

[0065] More specifically, in the embodiments of this application, Figure 3 This is a block diagram of the weight data encoding unit in a belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application, as follows: Figure 3 As shown, the weight data encoding unit includes: a first-scale weight feature extraction subunit 210, used to input the weight input vector into the first convolutional layer of the multi-scale neighborhood feature extraction module to obtain a first-scale weight feature vector, wherein the first convolutional layer has a first one-dimensional convolutional kernel of a first length; a second-scale weight feature extraction subunit 220, used to input the weight input vector into the second convolutional layer of the multi-scale neighborhood feature extraction module to obtain a second-scale weight feature vector, wherein the second convolutional layer has a second one-dimensional convolutional kernel of a second length, the first length being different from the second length; and a multi-scale concatenation subunit 230, used to concatenate the first-scale weight feature vector and the second-scale weight feature vector to obtain the weight feature vector.

[0066] Further, the first convolutional layer of the multi-scale neighborhood feature extraction module is used to perform one-dimensional convolutional encoding on the weight input vector using the following formula to obtain the first-scale weight feature vector; wherein, the formula is:

[0067]

[0068] Where a is the width of the first convolution kernel in the x direction, F(a) is the parameter vector of the first convolution kernel, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the weight input vector.

[0069] Furthermore, the second convolutional layer of the multi-scale neighborhood feature extraction module performs one-dimensional convolutional encoding on the weight input vector using the following formula to obtain the second-scale weight feature vector; wherein, the formula is:

[0070]

[0071] Where b is the width of the second convolution kernel in the x direction, F(b) is the parameter vector of the second convolution kernel, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the weight input vector.

[0072] In this way, the dynamic change characteristics of the weight data of the transported raw coal can be accurately extracted, so that the generated control results are more precise.

[0073] Specifically, in this embodiment, the monitoring image encoding unit 130 is used to obtain multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention to process the transportation monitoring images at each predetermined time point. Furthermore, considering that raw coal accumulates in different forms at different locations on the conveyor belt, and that the conveyor belt scale can only measure the weight data of the transported raw coal and cannot obtain the distribution characteristics of the raw coal on the conveyor belt plane, the technical solution of this application further involves acquiring the transportation monitoring images at the multiple predetermined time points using a camera, and then performing feature mining on the transportation monitoring images at each predetermined time point using a convolutional neural network model with excellent performance in extracting latent features of images, thereby extracting the latent feature distribution information of the raw coal on the conveyor belt.

[0074] However, considering that when the excavator's boom belt transports raw coal, there will be a large amount of raw coal interference feature information in other locations in the transport monitoring image, and given that the attention mechanism can select the focus position, in order to accurately extract the raw coal distribution feature information on the belt for adaptive adjustment of belt tension, a first convolutional neural network model with spatial attention is used to perform feature mining on the transport monitoring images at each predetermined time point to extract the implicit feature distribution information of raw coal focused on the belt plane in each raw coal transport monitoring image, thereby obtaining multiple transport monitoring feature matrices.

[0075] As you can understand, attention mechanisms are a data processing method in machine learning, widely used in various types of machine learning tasks such as natural language processing, image recognition, and speech recognition. On one hand, attention mechanisms aim to enable the network to automatically learn the areas in an image or text sequence that require attention; on the other hand, attention mechanisms generate a mask through neural network operations, with weights assigned to the values ​​on the mask. Generally, spatial attention mechanisms calculate the average across different channels of the same pixel, then perform convolution and upsampling operations to obtain spatial features, with each pixel in each layer of the spatial features being assigned different weights.

[0076] Furthermore, during the forward propagation of the first convolutional neural network model using spatial attention, each layer performs the following operations on the input data: convolution processing to generate a convolutional feature map; pooling processing to generate a pooled feature map; non-linear activation to generate an activation feature map; calculating the mean along the channel dimension of each position of the activation feature map to generate a spatial feature matrix; calculating the softmax-like function value at each position in the spatial feature matrix to obtain a spatial score matrix; and calculating the positional dot product of the spatial feature matrix and the spatial score matrix to obtain a feature matrix; wherein the feature matrix output by the last layer of the first convolutional neural network model using spatial attention is the plurality of transportation monitoring feature matrices.

[0077] It should be understood that by using a first convolutional neural network model with spatial attention to perform feature mining on the transportation monitoring images at each predetermined time point, the implicit feature distribution information of raw coal focused on the belt plane in each raw coal transportation monitoring image can be extracted, so as to accurately extract the raw coal distribution feature information on the belt and thus make adaptive adjustment of belt tension.

[0078] Specifically, in this embodiment, the monitoring dynamic encoding unit 140 is used to arrange the plurality of transportation monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map. Next, for the plurality of transportation monitoring feature matrices that have implicit feature distribution information focusing on raw coal on the conveyor belt plane at multiple predetermined time points, the plurality of transportation monitoring feature matrices are further arranged into a three-dimensional input tensor and then feature extraction is performed using a second convolutional neural network model with three-dimensional convolutional kernels to extract the dynamic features of the feature distribution information of raw coal on the conveyor belt plane, thereby obtaining the transportation monitoring feature map.

[0079] Specifically, here, the convolutional kernel of the second convolutional neural network model is a three-dimensional convolutional kernel, which has W (width), H (height), and C (channel dimension). In the technical solution of this application, the channel dimension of the three-dimensional convolutional kernel corresponds to the time dimension of the three-dimensional input tensor. Therefore, during three-dimensional convolutional encoding, it is possible to extract the dynamic distribution characteristics of the raw coal on the conveyor belt plane at multiple predetermined time points of the three-dimensional model, which are focused on the changes in the time dimension. It should be understood that by adding the time dimension to the input of the neural network, the neural network can simultaneously extract temporal and spatial features for behavior recognition and video processing.

[0080] More specifically, in the embodiments of this application, Figure 4 The following is a block diagram of the monitoring dynamic coding unit in the belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application, as shown below. Figure 4 As shown, the monitoring dynamic coding unit includes: a three-dimensional input tensor generation subunit 310, used to arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor; and a convolutional coding subunit 320, used to perform three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing on the input data in the forward propagation of the layers using the second convolutional neural network model using the three-dimensional convolutional kernel, so that the output of the last layer of the second convolutional neural network model using the three-dimensional convolutional kernel is the transportation monitoring feature map, wherein the input of the first layer of the second convolutional neural network model using the three-dimensional convolutional kernel is the three-dimensional input tensor.

[0081] Specifically, in this embodiment, the dimensionality reduction unit 150 is used to perform global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain a transportation monitoring feature vector. Then, it further performs global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to reduce the dimensionality of the implicit features of the raw coal distribution on the conveyor belt at each time point in the time channel dimension, thereby obtaining the transportation monitoring feature vector. This allows for the preservation of the dynamic feature information of the implicit feature distribution of raw coal on the conveyor belt while performing data dimensionality reduction, thereby improving the accuracy of subsequent classification.

[0082] More specifically, Figure 5 The following is a block diagram of the dimension reduction unit in the belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application, as shown below. Figure 5As shown, the dimensionality reduction unit includes: a local feature enhancement subunit 410, used to optimize the feature distribution of each feature matrix along the channel dimension of the transportation monitoring feature map to obtain multiple optimized feature matrices; and a global pooling subunit 420, used to calculate the global mean of each optimized feature matrix in the multiple optimized feature matrices to obtain the transportation monitoring feature vector.

[0083] Specifically, in the technical solution of this application, when performing global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector, it is desirable to enhance the correlation between the various feature matrices so that the transportation monitoring feature vector can better express the overall feature distribution of the transportation monitoring feature map while expressing the feature distribution along the channel dimension of the transportation monitoring feature map.

[0084] Therefore, if each feature matrix is ​​considered as a local feature distribution, what needs to be improved is the explicit correlation between each local feature distribution and the global feature distribution. Specifically, this can be achieved by setting predetermined weights as hyperparameters for each feature matrix. However, since the weights as hyperparameters need to be obtained during model training, this increases the training burden on the model. Therefore, the predetermined weights can be obtained by calculating the multi-distribution binary classification continuity factors of multiple feature matrices.

[0085] Furthermore, in the embodiments of this application, Figure 6 The following is a block diagram of the local feature enhancement subunit in the belt tensioning control system based on multi-sensor data fusion according to an embodiment of this application, as shown below. Figure 6 As shown, the local feature enhancement subunit includes: a secondary subunit 510 for calculating continuity factors, used to calculate the multi-distribution binary classification continuity factors of each feature matrix along the channel dimension of the transportation monitoring feature map using the following formula; wherein, the formula is:

[0086]

[0087] softmaxv(M) = [p(M), 1 - p(M)]

[0088] Among them, M i M is the i-th feature matrix along the channel dimension of the transportation monitoring feature map. r It is a reference matrix based on multiple feature matrices along the channel dimension of the transportation monitoring feature map, where p(M) represents the probability value obtained by the classifier from the feature matrix, and w iThe multi-distribution binary classification continuity factor of the i-th feature matrix along the channel dimension of the transportation monitoring feature map is represented by , log represents the logarithmic function to the base 2, ||·||2 represents the L2 norm of the feature matrix, and softmaxv(·) represents the softmaxv activation function; the weighted optimization secondary subunit 520 is used to perform weighted optimization on each feature matrix along the channel dimension of the transportation monitoring feature map using the multi-distribution binary classification continuity factor of each feature matrix along the channel dimension of the transportation monitoring feature map as weighting coefficients to obtain the multiple optimized feature matrices.

[0089] In other words, to avoid the difficulty in converging towards the target classification domain due to excessive fragmentation of the decision boundaries corresponding to the local feature distributions of each feature matrix in a feature distribution-based classification task, the classification continuity factor of the local feature distribution of each feature matrix relative to the global feature distribution is predicted by calculating the class probability offset distribution modulus information of the binary classification of the local feature distribution of each feature matrix relative to the global average feature distribution. Thus, by using this as a weight to weight the multiple feature matrices, the optimization of hyperparameters during training can be transformed from backpropagation into a classification problem based on the binary classification of multiple distributions, thereby improving the representation effect of the transportation monitoring feature vector on the overall feature distribution of the transportation monitoring feature map.

[0090] Specifically, in this embodiment, the multi-sensor feature fusion unit 160 is used to fuse the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector. Further, by fusing the feature information from the weight feature vector and the transportation monitoring feature vector, and using this as the classification feature vector for classification processing by a classifier, a classification result indicating whether the belt tension should increase or decrease at the current time point can be obtained.

[0091] Further, the weight feature vector and the transportation monitoring feature vector are fused using the following formula to obtain the classification feature vector; wherein, the formula is:

[0092]

[0093] Wherein, V is the classification feature vector, V1 is the weight feature vector, and V2 is the transportation monitoring feature vector. This indicates positional addition, where λ and β are weighting parameters used to control the balance between the weight feature vector and the transportation monitoring feature vector.

[0094] Specifically, in this embodiment, the belt tension control result generation unit 170 is used to pass the classification feature vector through a classifier to obtain a classification result. The classification result is used to indicate whether the belt tension should be increased or decreased at the current time point. This allows for real-time and accurate adaptive control of the belt tension based on the actual situation of the raw coal on the belt, thereby ensuring that the belt tension meets transportation requirements while avoiding excessive belt tension that could shorten its lifespan.

[0095] In the embodiments of this application, Figure 7 This is a block diagram of the belt tension control result generation unit in the belt tension control system based on multi-sensor data fusion according to an embodiment of this application, as shown below. Figure 7 As shown, the belt tension control result generation unit includes: a fully connected encoding subunit 610, used to perform fully connected encoding on the classification feature vector using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and a classification subunit 620, used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0096] In a specific example of this application, the classifier processes the classification feature vector using the following formula to obtain the classification result: softmax{(W n B n ):…:(W1,B1)|X}, where W1 to W n The weight matrix is ​​B1 to B1. n X is the bias vector, and X is the classification feature vector.

[0097] In summary, the belt tension control system 100 based on multi-sensor data fusion according to the embodiments of this application is explained. It extracts multi-scale neighborhood correlation features in the time dimension from the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period using a multi-scale neighborhood feature extraction module. It then extracts dynamic features focusing on the feature distribution information of the raw coal on the belt plane from the transportation monitoring image using a first convolutional neural network model with spatial attention and a second convolutional neural network model using a three-dimensional convolutional kernel. Finally, it performs adaptive control of the belt tension at the current time point based on the fused feature information of these two models. In this way, the belt tension can be intelligently controlled in real time and accurately according to the actual situation, thereby ensuring that the belt tension meets transportation requirements while avoiding excessive belt tension that could reduce belt life.

[0098] As described above, the belt tensioning control system 100 based on multi-sensor data fusion according to the embodiments of this application can be implemented in various terminal devices, such as servers for belt tensioning control system algorithms based on multi-sensor data fusion. In one example, the belt tensioning control system 100 based on multi-sensor data fusion can be integrated into the terminal device as a software module and / or a hardware module. For example, the belt tensioning control system 100 based on multi-sensor data fusion can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the belt tensioning control system 100 based on multi-sensor data fusion can also be one of many hardware modules of the terminal device.

[0099] Alternatively, in another example, the belt tensioning control system 100 based on multi-sensor data fusion and the terminal device can also be separate devices, and the belt tensioning control system 100 based on multi-sensor data fusion can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0100] Exemplary methods

[0101] Figure 8 This is a flowchart of a belt tension control method based on multi-sensor data fusion according to an embodiment of this application. Figure 8 As shown, the belt tension control method based on multi-sensor data fusion according to an embodiment of this application includes: S110, acquiring weight data of transported raw coal collected by a belt scale at multiple predetermined time points within a predetermined time period, and transport monitoring images at the multiple predetermined time points; S120, arranging the weight data of transported raw coal collected by the belt scale at the multiple predetermined time points into a weight input vector according to the time dimension, and then obtaining a weight feature vector through a multi-scale neighborhood feature extraction module; S130, using a first convolutional neural network model with spatial attention to obtain... S140: Obtain multiple transportation monitoring feature matrices; S150: Arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map; S160: Perform global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain a transportation monitoring feature vector; S170: Fuse the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector; and S180: Pass the classification feature vector through a classifier to obtain a classification result, the classification result being used to indicate whether the belt tension at the current time point should increase or decrease.

[0102] Figure 9This is a schematic diagram of the system architecture of a belt tension control method based on multi-sensor data fusion according to an embodiment of this application. Figure 9 As shown, in the system architecture of the belt tension control method based on multi-sensor data fusion, firstly, the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as the transport monitoring images at the multiple predetermined time points, are acquired; then, the weight data of the transported raw coal collected by the belt scale at the multiple predetermined time points are arranged into a weight input vector according to the time dimension and then processed by a multi-scale neighborhood feature extraction module to obtain a weight feature vector; next, the transport monitoring images at each predetermined time point are processed by a first convolutional neural network model with spatial attention to obtain multiple transport monitoring feature matrices; then, the multiple transport monitoring feature matrices are arranged into a three-dimensional input tensor and processed by a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transport monitoring feature map; next, global mean pooling is performed on each feature matrix along the channel dimension of the transport monitoring feature map to obtain a transport monitoring feature vector; then, the weight feature vector and the transport monitoring feature vector are fused to obtain a classification feature vector; and finally, the classification feature vector is processed by a classifier to obtain a classification result, which is used to indicate whether the belt tension at the current time point should be increased or decreased.

[0103] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of arranging the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points into a weight input vector according to the time dimension and then obtaining a weight feature vector through a multi-scale neighborhood feature extraction module includes: inputting the weight input vector into the first convolutional layer of the multi-scale neighborhood feature extraction module to obtain a first-scale weight feature vector, wherein the first convolutional layer has a first one-dimensional convolutional kernel of a first length; inputting the weight input vector into the second convolutional layer of the multi-scale neighborhood feature extraction module to obtain a second-scale weight feature vector, wherein the second convolutional layer has a second one-dimensional convolutional kernel of a second length, and the first length is different from the second length; and concatenating the first-scale weight feature vector and the second-scale weight feature vector to obtain the weight feature vector.

[0104] In a specific example, in the belt tension control method based on multi-sensor data fusion described above, the step of inputting the weight input vector into the first convolutional layer of the multi-scale neighborhood feature extraction module to obtain a first-scale weight feature vector, wherein the first convolutional layer has a first one-dimensional convolutional kernel of a first length, further includes: using the first convolutional layer of the multi-scale neighborhood feature extraction module to perform one-dimensional convolutional encoding on the weight input vector using the following formula to obtain the first-scale weight feature vector; wherein the formula is:

[0105]

[0106] Where a is the width of the first convolution kernel in the x direction, F(a) is the parameter vector of the first convolution kernel, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the weight input vector;

[0107] The step of inputting the weight input vector into the second convolutional layer of the multi-scale neighborhood feature extraction module to obtain a second-scale weight feature vector, wherein the second convolutional layer has a second one-dimensional convolutional kernel of a second length, the first length being different from the second length, further includes: using the second convolutional layer of the multi-scale neighborhood feature extraction module to perform one-dimensional convolutional encoding on the weight input vector using the following formula to obtain the second-scale weight feature vector; wherein the formula is:

[0108]

[0109] Where b is the width of the second convolution kernel in the x direction, F(b) is the parameter vector of the second convolution kernel, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the weight input vector.

[0110] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of obtaining multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention to obtain the transportation monitoring images at each predetermined time point further includes: each layer of the first convolutional neural network model with spatial attention performs the following operations on the input data during the forward propagation of the layer: convolution processing on the input data to generate a convolutional feature map; pooling processing on the convolutional feature map to generate a pooled feature map; nonlinear activation on the pooled feature map to generate an activation feature map; calculating the mean along the channel dimension of each position of the activation feature map to generate a spatial feature matrix; calculating the softmax-like function value of each position in the spatial feature matrix to obtain a spatial score matrix; and calculating the positional dot product of the spatial feature matrix and the spatial score matrix to obtain a feature matrix; wherein, the feature matrix output by the last layer of the first convolutional neural network model with spatial attention is the multiple transportation monitoring feature matrices.

[0111] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of arranging the multiple transportation monitoring feature matrices into a three-dimensional input tensor and then using a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map includes: arranging the multiple transportation monitoring feature matrices into a three-dimensional input tensor; and performing three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing on the input data in the forward propagation of the layers using the second convolutional neural network model with three-dimensional convolutional kernels, so that the output of the last layer of the second convolutional neural network model with three-dimensional convolutional kernels is the transportation monitoring feature map, wherein the input of the first layer of the second convolutional neural network model with three-dimensional convolutional kernels is the three-dimensional input tensor.

[0112] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of performing global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector includes: performing feature distribution optimization on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain multiple optimized feature matrices; and calculating the global mean of each optimized feature matrix among the multiple optimized feature matrices to obtain the transportation monitoring feature vector.

[0113] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of optimizing the feature distribution of each feature matrix along the channel dimension of the transportation monitoring feature map to obtain multiple optimized feature matrices includes: calculating the multi-distribution binary classification continuity factor of each feature matrix along the channel dimension of the transportation monitoring feature map using the following formula; wherein, the formula is:

[0114]

[0115] softmaxv(M) = [p(M), 1 - p(M)]

[0116] Among them, M i M is the i-th feature matrix along the channel dimension of the transportation monitoring feature map. r It is a reference matrix based on multiple feature matrices along the channel dimension of the transportation monitoring feature map, where p(M) represents the probability value obtained by the classifier from the feature matrix, and w iLet represent the multi-distribution binary classification continuity factor of the i-th feature matrix along the channel dimension of the transportation monitoring feature map, log represents the logarithmic function to the base 2, ||·||2 represents the L2 norm of the feature matrix, and softmaxv(·) represents the softmaxv activation function. The multi-distribution binary classification continuity factor of each feature matrix along the channel dimension of the transportation monitoring feature map is used as a weighting coefficient to perform weighted optimization on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the multiple optimized feature matrices.

[0117] In a specific example, in the above-mentioned belt tension control method based on multi-sensor data fusion, the step of fusing the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector further includes: fusing the weight feature vector and the transportation monitoring feature vector to obtain the classification feature vector using the following formula; wherein, the formula is:

[0118]

[0119] Wherein, V is the classification feature vector, V1 is the weight feature vector, and V2 is the transportation monitoring feature vector. This indicates positional addition, where λ and β are weighting parameters used to control the balance between the weight feature vector and the transportation monitoring feature vector.

[0120] In a specific example, in the belt tension control method based on multi-sensor data fusion described above, the step of passing the classification feature vector through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the belt tension should increase or decrease at the current time point, includes: using multiple fully connected layers of the classifier to fully connect and encode the classification feature vector to obtain an encoded classification feature vector; and passing the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0121] Here, those skilled in the art will understand that the specific operations of each step in the above-described belt tension control method based on multi-sensor data fusion have been referenced above. Figures 1 to 7 The belt tension control system based on multi-sensor data fusion has been described in detail in the previous section, and therefore, its repeated description will be omitted.

[0122] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0123] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0124] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0125] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A belt tension control system based on multi-sensor data fusion, characterized in that, include: The multi-sensor monitoring unit is used to acquire the weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as the transport monitoring images at the multiple predetermined time points. The weight data encoding unit is used to arrange the weight data of the transported raw coal collected by the belt electronic scale at the multiple predetermined time points into a weight input vector according to the time dimension, and then obtain the weight feature vector through the multi-scale neighborhood feature extraction module. The monitoring image encoding unit is used to obtain multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention to process the transportation monitoring images at each predetermined time point; The monitoring dynamic coding unit is used to arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor and then use a second convolutional neural network model with three-dimensional convolutional kernels to obtain a transportation monitoring feature map. The dimensionality reduction unit is used to perform global mean pooling on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector. A multi-sensor feature fusion unit is used to fuse the weight feature vector and the transportation monitoring feature vector to obtain a classification feature vector; as well as A belt tension control result generation unit is used to pass the classification feature vector through a classifier to obtain a classification result, which is used to indicate whether the belt tension should be increased or decreased at the current time point. The dimensionality reduction unit includes: The local feature enhancement subunit includes: a secondary subunit for calculating continuity factors, used to calculate the multi-distribution binary classification continuity factors of each feature matrix along the channel dimension of the transportation monitoring feature map using the following formula; The formula is as follows: in, It is the first feature map along the channel dimension of the transportation monitoring feature map. Each feature matrix It is a reference matrix based on multiple feature matrices along the channel dimension of the aforementioned transportation monitoring feature map. This represents the probability values ​​obtained by the classifier from the feature matrix. The first feature map along the channel dimension represents the transportation monitoring feature map. The multi-distribution binary classification continuity factor of the feature matrix, Represents the logarithmic function with base 2. The 2-norm of the characteristic matrix, express Activation function; The weighted optimization secondary subunit is used to perform weighted optimization on each feature matrix along the channel dimension of the transportation monitoring feature map using the multi-distribution binary classification continuity factor of each feature matrix along the channel dimension as a weighting coefficient to obtain multiple optimized feature matrices. A global pooling subunit is used to calculate the global mean of each of the multiple optimized feature matrices to obtain the transportation monitoring feature vector.

2. The belt tensioning control system based on multi-sensor data fusion according to claim 1, characterized in that, The weight data encoding unit includes: A first-scale weight feature extraction subunit is used to input the weight input vector into the first convolutional layer of the multi-scale neighborhood feature extraction module to obtain a first-scale weight feature vector, wherein the first convolutional layer has a first one-dimensional convolutional kernel of a first length; A second-scale weight feature extraction subunit is used to input the weight input vector into the second convolutional layer of the multi-scale neighborhood feature extraction module to obtain a second-scale weight feature vector, wherein the second convolutional layer has a second one-dimensional convolutional kernel of a second length, and the first length is different from the second length; and A multi-scale cascaded subunit is used to cascade the first-scale weight feature vector and the second-scale weight feature vector to obtain the weight feature vector.

3. The belt tensioning control system based on multi-sensor data fusion according to claim 2, characterized in that, The first-scale weight feature extraction subunit is further configured to: use the first convolutional layer of the multi-scale neighborhood feature extraction module to perform one-dimensional convolutional encoding on the weight input vector to obtain the first-scale weight feature vector using the following formula; The formula is as follows: Where a is the width of the first convolution kernel in the x-direction, For the first convolution kernel parameter vector, The vector matrix is ​​the local vector matrix that operates with the convolution kernel function, where w is the size of the first convolution kernel and X represents the weight input vector; The second-scale weight feature extraction subunit is further configured to: use the second convolutional layer of the multi-scale neighborhood feature extraction module to perform one-dimensional convolutional encoding on the weight input vector to obtain the second-scale weight feature vector using the following formula; The formula is as follows: Where b is the width of the second convolution kernel in the x-direction, For the second convolution kernel parameter vector, Let m be the local vector matrix that operates with the convolution kernel function, m be the size of the second convolution kernel, and X be the weight input vector.

4. The belt tension control system based on multi-sensor data fusion according to claim 3, characterized in that, The monitoring image encoding unit is further configured to: process the input data in each layer of the first convolutional neural network model using spatial attention during the forward propagation of the layer, respectively: The input data is processed by convolution to generate convolutional feature maps; The convolutional feature map is subjected to pooling to generate a pooled feature map; The pooled feature map is nonlinearly activated to generate an activated feature map; Calculate the mean value along the channel dimension at each position of the activated feature map to generate a spatial feature matrix; Calculate the softmax function value at each position in the spatial feature matrix to obtain the spatial score matrix; as well as The feature matrix is ​​obtained by calculating the positional dot product of the spatial feature matrix and the spatial score matrix; The feature matrix output by the last layer of the first convolutional neural network model using spatial attention is the plurality of transportation monitoring feature matrices.

5. The belt tensioning control system based on multi-sensor data fusion according to claim 4, characterized in that, The monitoring dynamic encoding unit includes: A three-dimensional input tensor generation subunit is used to arrange the multiple transportation monitoring feature matrices into a three-dimensional input tensor. A convolutional coding subunit is used to perform three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing on the input data during the forward propagation of the layers using the second convolutional neural network model using three-dimensional convolutional kernels, respectively, so that the output of the last layer of the second convolutional neural network model using three-dimensional convolutional kernels is the transportation monitoring feature map, wherein the input of the first layer of the second convolutional neural network model using three-dimensional convolutional kernels is the three-dimensional input tensor.

6. The belt tension control system based on multi-sensor data fusion according to claim 5, characterized in that, The multi-sensor feature fusion unit is further configured to: fuse the weight feature vector and the transportation monitoring feature vector using the following formula to obtain the classification feature vector; The formula is as follows: in, For the classification feature vector, The weight feature vector, The transportation monitoring feature vector, This indicates addition by position. The weighting parameter is used to control the balance between the weight feature vector and the transportation monitoring feature vector.

7. The belt tensioning control system based on multi-sensor data fusion according to claim 6, characterized in that, The belt tension control result generation unit includes: A fully connected encoding subunit is configured to perform fully connected encoding on the classification feature vector using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and The classification subunit is used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

8. A belt tension control method based on multi-sensor data fusion, using the belt tension control system based on multi-sensor data fusion as described in any one of claims 1-7, characterized in that, include: The weight data of the transported raw coal collected by the belt scale at multiple predetermined time points within a predetermined time period, as well as the transportation monitoring images at the multiple predetermined time points, are obtained. The weight data of the transported raw coal collected by the belt scale at the multiple predetermined time points are arranged into a weight input vector according to the time dimension, and then the weight feature vector is obtained by the multi-scale neighborhood feature extraction module. The transportation monitoring images at each predetermined time point are used to obtain multiple transportation monitoring feature matrices by using a first convolutional neural network model with spatial attention. After arranging the multiple transportation monitoring feature matrices into a three-dimensional input tensor, a second convolutional neural network model with three-dimensional convolutional kernels is used to obtain a transportation monitoring feature map. Global mean pooling is performed on each feature matrix along the channel dimension of the transportation monitoring feature map to obtain the transportation monitoring feature vector; The weight feature vector and the transportation monitoring feature vector are fused to obtain a classification feature vector; as well as The classification feature vector is passed through a classifier to obtain a classification result, which is used to indicate whether the belt tension should be increased or decreased at the current time point.

Images