A sieve plate falling-off intelligent monitoring method based on multi-scale feature extraction

Abstract

The present application relates to a kind of based on multiscale feature extraction's screen plate fall-off intelligent monitoring method, comprising the following steps: with the exciter of target vibrating screen as demarcation, on both sides respectively set up the image capture equipment opposite screen plate;After image information is obtained from the image capture equipment and is segmented and perspective conversion processing, input pre-trained screen plate fall-off detection model, obtain the state monitoring result of the target vibrating screen, wherein the screen plate fall-off detection model includes multiscale feature extraction network, multiple residual blocks and convolution block attention module connected in turn.Compared with prior art, the present application realizes the effective monitoring of screen plate state, while monitoring accuracy is high, solves or partially solves the problems of large detection hysteresis, high misjudgment and other problems in prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent monitoring, in particular to a screen plate falling intelligent monitoring method based on multi-scale feature extraction. BACKGROUND

[0002] The vibrating screen is a common solid particle material screening equipment, which is usually used for material classification in industrial production. The basic principle of the vibrating screen is to use vibration force to throw the material upward and screen it through the screen. Compared with other screening equipment, the vibrating screen has the advantages of simple structure, high screening efficiency and low energy consumption, and is widely used in coal mines, chemical industry, metallurgy and other fields.

[0003] The screen plate of the vibrating screen is made of polyurethane material, which has good hydrophobicity and wear resistance. The screen plate and the gauge seat are connected through concave-convex keys. With the operation of the vibrating screen, the concave-convex keys between the screen plate and the gauge seat will be worn. When the wear reaches a certain degree, the screen plate will loosen or even fall off. If the fault cannot be found in time, the material above the screen plate will enter the screen under the chute and the pipeline, which will not only pollute the recovered medium, but also cause pipeline blockage and other production accidents.

[0004] At present, there is no online monitoring device to realize the timely diagnosis of this fault. Most of the coal preparation plants in China use manual inspection method for detection. This method not only consumes a lot of manpower, material resources and time, but also cannot guarantee the timeliness and accuracy of detection.

[0005] Wangjialing coal preparation plant uses a bolt pressure plate scheme to fix the screen plate and the gauge seat as a whole, so that the overall fixation method changes from soft connection to hard link, which can greatly reduce the probability of screen plate falling off. However, screen plate falling off fault is still inevitable, and real-time monitoring of screen plate fault cannot be realized.

[0006] Wuliheng coal mine uses foreign object detection box, detection probe plate, photoelectric switch, controller and alarm to form a screen plate falling monitoring device, but its mechanical structure is relatively complex, and the monitoring has a certain hysteresis.

[0007] The Chinese patent application No. CN201911186854.2 provides a kind of screen fault intelligent monitoring method based on depth image, comprising the following steps: step 1, according to the work station of screen and the internal parameter of camera itself, determine the matrix arrangement of depth camera installation;Step 2, dimension conversion is carried out to original data;Step 3, the invalid data in the data after dimension conversion is handled;Step 4, the data after invalid data processing is rendered with pseudo-color;Step 5, in depth image, according to the work station of each screen, divide out sub-image, analyze and judge working condition for each sub-image;Step 6, the signal intensity data of camera capture image is obtained, and compared with the signal intensity data when normal operation, to determine whether there is coal on the screen, further determine whether the raw coal hopper is blocked.

[0008] The above method does not need manual intervention, and is a non-invasive monitoring method that does not affect normal production, but the above method compares the depth value of the screen area in the image with the normal depth value to determine whether a fault occurs, which requires multiple adjustments according to the use environment to determine the normal depth value, and this scheme is also easily affected by environmental light.

[0009] In view of the existing screen shedding fault monitoring problems such as misjudgment and lag, it is necessary to design a screen shedding intelligent monitoring method. SUMMARY

[0010] The purpose of the present application is to overcome the defects of the prior art and provide a screen shedding intelligent monitoring method based on multi-scale feature extraction, to effectively monitor the state of the screen and improve the robustness of state monitoring.

[0011] The purpose of the present application can be achieved by the following technical solutions:

[0012] The present application provides a screen shedding intelligent monitoring method based on multi-scale feature extraction, comprising the following steps:

[0013] The image capture device is arranged opposite to the screen on both sides with the exciter of the target vibrating screen as the boundary;

[0014] After the image information is obtained from the image capture device and segmented and perspective transformation is processed, it is input into the pre-trained screen shedding detection model to determine whether the screen of the target vibrating screen is shed, wherein the screen shedding detection model comprises a multi-scale feature extraction network, a plurality of residual blocks and a convolution block attention module connected in sequence.

[0015] As a preferred technical solution, the acquisition of the pre-trained screen shedding detection model comprises the following steps:

[0016] Obtaining original images of different states at different times obtained by the image capture device and performing segmentation and perspective transformation processing, obtaining a sample set through data augmentation and performing labeling;

[0017] The sample set is divided and a screen plate falling detection model is trained, and when the accuracy and loss function meet preset conditions, the model weight parameters are fixed and saved, and the pre-trained screen plate falling detection model is obtained.

[0018] As a preferred technical solution, the perspective transformation processing is realized by the following formula:

[0019]

[0020] Among them, u , v is an original coordinate point, is an image coordinate point after perspective transformation processing, w is a parameter, and the corresponding transformed coordinate is , , in the transformation matrix, , , , represents linear transformation, , represents translation, , represents perspective transformation.

[0021] As a preferred technical solution, the data augmentation includes the following steps:

[0022] At least one of rotation, scaling, inversion, translation transformation is performed on the segmented sunburn picture, or one of image brightness, contrast, saturation, hue is adjusted, to obtain a new image, and data augmentation is realized.

[0023] As a preferred technical solution, the sample set is divided in particular as follows:

[0024] The sample set is divided into a training set, a validation set and a test set, the training set is used for training the model, the validation set is used for selecting hyperparameters, and the test set is used for determining the generalization ability of the model.

[0025] As a preferred technical solution, the multi-scale feature extraction module includes a plurality of parallel branches and a Concat layer connected to each branch, at least one convolution layer is arranged on each branch, and the convolution kernel size of the convolution layer on each branch is different from the convolution kernel size of the convolution layer of other branches.

[0026] As a preferred technical scheme, for any one of the plurality of residual blocks, each includes a plurality of convolution units, and each convolution unit includes a plurality of convolution layers.

[0027] As a preferred technical scheme, the convolution unit includes two connected convolution layers.

[0028] As a preferred technical scheme, in the screen plate falling detection model, after the convolution block attention module, a global average pooling layer and a fully connected layer are further included.

[0029] As a preferred technical scheme, in the screen plate falling detection model, the multi-scale feature extraction network and the plurality of residual blocks are further connected with a max-pooling layer.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] (1) The screen plate state is effectively monitored: the existing scheme compares the depth value of the screen plate area in the image with the normal depth value to determine whether a fault occurs, which is different from the prior art. The present application monitors the state of the screen plate of the vibrating screen by constructing a screen plate falling detection model, and at the same time, multi-scale feature extraction and attention modules are added to the model to effectively extract image signals, thereby realizing the monitoring of the normal and falling states of the screen plate.

[0032] (2) High monitoring accuracy: in order to solve or partially solve the problem that the existing environmental light source easily affects the monitoring result, the sample set is expanded by adjusting the image brightness, contrast, saturation and hue before training the screen plate falling detection model. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The flowchart of the screen plate falling intelligent monitoring method based on multi-scale feature extraction in the embodiment;

[0034] Figure 2 The structure diagram of the screen plate falling detection model;

[0035] Figure 3 The accuracy and loss function diagram of the training set and the verification set. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0037] Embodiment 1

[0038] As Figure 1 described, the embodiment provides a screen plate falling intelligent monitoring method based on multi-scale feature extraction to solve the problems of large detection lag, high misjudgment and the like in the prior art. Mainly includes the following steps:

[0039] Step S1, set up an industrial camera and collect image data;

[0040] Step S2, perform image segmentation and data augmentation processing to form a sample;

[0041] Step S3, construct a sample set and divide it;

[0042] Step S4, use the sample set to train MFE-CBMA-ResNet;

[0043] Step S5, save the model parameters and obtain the trained model.

[0044] In order to achieve the above purpose, the idea of the present application is:

[0045] Firstly, the image of the screen plate working operation is obtained by the industrial camera, which is transmitted to the communication module of the local control cabinet through the RJ45 interface of the industrial camera, and the image collected by the industrial camera is obtained by the industrial computer through the network cable in real time. Then the image is preprocessed by the industrial computer, and the online detection of the screen plate is completed by the real-time monitoring and fault diagnosis system. The key of the present application is the network model design of the screen plate image fault classification based on deep learning, including the preprocessing of the original image, the segmentation method of the image, the design of the multi-scale feature extraction module, the position of the attention module, and the setting of the network model parameters. The output result of the screen plate falling real-time monitoring and fault diagnosis system based on multi-scale feature extraction will be dynamically displayed on the designed host computer interface. The convolutional neural network based on multi-scale feature extraction includes three processes of training, verification and testing. Among them, the data comes from the real samples of Pingdingshan No. 8 Mine Coal Preparation Plant. The input image size of the convolutional neural network is 140 color three-channel image.

[0046] The industrial camera is installed above the vibrating screen system where the screen plate may fall off, and the camera lens is directed to the screen surface. Since the range covered by a single industrial camera lens is limited, the position of the industrial camera needs to be determined when installed, so that the camera lens covers a certain overlapping area, which ensures that each screen plate is in real-time monitoring. Therefore, two industrial cameras are used for monitoring for each vibrating screen, and the industrial camera shoots the screen plate above the vibrator, and the other industrial camera shoots the screen plate below the vibrator. Adjust the size and proportion of the picture taken by the industrial camera, so that the screen plate is located in the middle of the lens.

[0047] According to the above idea, the embodiment adopts the technical scheme as follows:

[0048] The intelligent screen plate falling monitoring method based on multi-scale feature extraction is used to obtain the running condition of the screen plate in the vibration screen system and the fault positioning point of the screen plate falling. The method comprises the following steps:

[0049] Step 1: According to the height and screen plate position of the vibration screen in the coal preparation plant workshop, the installation position and installation method of the industrial camera are determined.

[0050] Step 2: Realize the communication between the industrial camera and the industrial computer, and perform image expansion and data segmentation on the original image transmitted to the industrial computer.

[0051] Step 3: Label the preprocessed data to form a data set, and divide the data set into a test set, a validation set and a training set in proportion.

[0052] Step 4: Design a multi-scale feature extraction module; design a convolutional neural network.

[0053] Step 5: Train the convolutional neural network based on multi-scale feature extraction and attention mechanism, and adjust the parameters of the network model.

[0054] Step 6: Test the neural network model with adjusted parameters, and count the test results.

[0055] Step 7: Integrate the trained model into the intelligent screen plate falling detection system, and record the state information related to the screen plate falling into the log.

[0056] Step 1 specifically comprises the following steps:

[0057] Step 1.1: Cut off the power adapter of the industrial camera, keep the interface connected to the camera, and connect a power cord of appropriate length according to the distance from the industrial camera installation position to the control room. Since the output voltage of the self-contained power adapter is 12V, the input end of the power cord needs to be connected to a 12V DC power supply.

[0058] Step 1.2: Install two industrial cameras for each vibration screen. The industrial cameras are installed on the roof of the coal preparation workshop, and the shooting position is divided by the vibration exciter of the vibration screen. One industrial camera shoots the screen plate above the vibration screen, and the other industrial camera shoots the screen plate below the vibration screen, so that each screen plate of each vibration screen can be shot.

[0059] Step 1.3: Connect the industrial camera and the switch in the local control cabinet with a network cable; the industrial computer connects the switch through a network cable, so that it can receive the screen plate images transmitted by multiple industrial cameras at the same time.

[0060] Step 2 specifically includes the following steps:

[0061] Step 2.1: Assign an IP address under the same local area network to each industrial camera. Try to connect the industrial camera with the software on the industrial computer to determine whether the communication between the industrial camera and the industrial computer is successful.

[0062] Step 2.2: Save the data collected by the industrial camera in different folders according to the IP address of the camera. The industrial camera has a top-down view, so the screen surface is narrow at the top and wide at the bottom. Divide the original image according to the size of the screen plate, and use the perspective transformation algorithm to make each screen plate size 140 140 color three-channel image. The formula for perspective transformation is:

[0063]

[0064] wherein, u , v is the original coordinate point, written in the form of an augmented vector, w = 1. The corresponding transformed coordinates are , . In the transformation matrix, , , , represent linear transformation, , represent translation, , represent perspective transformation.

[0065] Step 2.3: Rotate, scale, flip, and translate the segmented screen plate images, and adjust the brightness, contrast, saturation, and hue of the data images to generate new data samples and expand the size of the training set.

[0066] Step 3 includes the following steps:

[0067] Step 3.1: Traverse all screen plate images and label them as 1, 2, and 3 according to normal, damaged, and shed, respectively.

[0068] Step 3.2: Divide the labeled data set into training set, validation set, and test set according to the ratio of 8:1:1. The training set is used to train the model; the validation set is used to observe the model training situation and select appropriate hyperparameters; the test set is used to evaluate the generalization ability of the model.

[0069] Step 4 includes the following steps:

[0070] Step 4.1: The multi-scale feature extraction module is composed of convolution kernels 1 1, 3 3, 5 5 and 7 The convolutional layer group of 7 is composed, and finally the feature maps of the four branches are channel spliced using the Concat layer. In order to improve the expression ability and convergence speed of the model, a BN layer and an activation function ReLU are added after the convolutional layer of each branch of the multi-scale feature extraction module. The number of convolution kernels of each convolutional layer is 16, and the output feature map with a channel number of 64 is obtained after the Concat layer.

[0071] Step 4.2: Design residual module Conv2x. Conv2_x is composed of three residual modules. Each residual module is composed of two convolutional layers. Each two convolutional layers are directly connected through a shortcut channel, and the number of convolution kernels of each convolutional layer is 64.

[0072] Step 4.3: Design residual module Conv3x. Conv3x is composed of four residual modules. Each residual module is composed of two convolutional layers.

[0073] The number of convolution kernels of each convolutional layer is 128.

[0074] Step 4.4: Design residual module Conv4_x. Conv4x is composed of five residual modules. Each residual module is composed of two convolutional layers.

[0075] The number of convolution kernels of each convolutional layer is 256.

[0076] Step 4.5: Design residual module Conv5_x. Conv5x is composed of three residual modules. Each residual module is composed of two convolutional layers.

[0077] The number of convolution kernels of each convolutional layer is 512.

[0078] Step 4.6: Design convolutional neural network. The multi-scale feature extraction module, the maximum pooling layer, Conv2x, Conv3x, Conv4_x, Conv5_x, the spatial attention mechanism module, the global average pooling layer, and the fully connected layer are sequentially connected to form a multi-scale feature extraction-based convolutional neural network structure.

[0079] Step 5 includes the following steps:

[0080] Step 5.1: Put the test set data into the multi-scale feature extraction-based convolutional neural network for testing, and set the initial weight parameters, the Batch batch size of each input, the number of training times Epoch, the learning rate, and other hyperparameters.

[0081] Step 5.2: Put the validation set into the trained neural network every time Epoch is trained, and select appropriate hyperparameters.

[0082] Step 5.3: Calculate the accuracy and loss function of the training set and the validation set, as shown in Figure 3 , where (a) is a Loss curve diagram, and (b) is an Accuracy curve diagram.

[0083] Step 6 includes the following steps:

[0084] Step 6.1: Save the weight parameters of the trained network model in step 5 as a PKL file.

[0085] Step 6.2: Put the test set into the trained network model, and calculate the accuracy of the test set.

[0086] Step 7 includes the following steps:

[0087] Step 7.1: Integrate the trained model into the screen plate shedding real-time monitoring and fault diagnosis system.

[0088] Step 7.2: After the image collected by the industrial camera in real time is preprocessed by step 2, the working condition of the screen plate is diagnosed by the screen plate shedding real-time monitoring and fault diagnosis system.

[0089] Step 7.3: When the system diagnoses the screen plate as a shedding state, the information of the shed screen plate is saved in the log, and a warning is made in the screen plate monitoring area of the industrial computer interface.

[0090] The present application monitors the state of the screen plate of the vibrating screen by constructing a screen plate shedding detection model, and at the same time, the model adds a multi-scale feature extraction and attention module to effectively extract the image signal, thereby realizing the monitoring of the normal, shedding and damage states of the screen plate.

[0091] Example 2

[0092] Compared with example 1, the present embodiment replaces the 5 5 convolution kernel in the multi-scale feature extraction module with a cascade of two 3 3 small convolution kernels; replaces the 7 7 convolution kernel with a cascade of 3 3, 1 5 and 5 1 convolution kernels. Details are as Figure 2 indicated.

[0093] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for intelligent monitoring of screen plate shedding based on multi-scale feature extraction, characterized in that, The method comprises the following steps: The image capture device is arranged opposite to the screen plate on both sides of the vibrator of the target vibrating screen as a boundary; After the image information is obtained from the image capture device and is subjected to segmentation and perspective transformation processing, the pre-trained screen plate falling detection model is inputted to determine whether the screen plate of the target vibrating screen falls off, wherein the screen plate falling detection model comprises a multi-scale feature extraction network, a plurality of residual blocks and a convolution block attention module connected in sequence, The multi-scale feature extraction module comprises a plurality of parallel branches and a Concat layer connected with each branch, at least one convolution layer is arranged on each branch, and the convolution kernel size of the convolution layer on each branch is different from that of the convolution layer of other branches, Each of the plurality of residual blocks comprises a plurality of convolution units, and each convolution unit comprises a plurality of convolution layers, The convolution unit comprises two connected convolution layers, The screen plate falling detection model further comprises a global average pooling layer and a fully connected layer after the convolution block attention module, The multi-scale feature extraction network and the plurality of residual blocks are further connected with a max-pooling layer.

2. The method according to claim 1, wherein, The pre-trained screen plate falling detection model comprises the following steps: The original images of different states and different times obtained by the image capture device are segmented and subjected to perspective transformation processing, a sample set is obtained through data augmentation, and the sample set is labeled; The sample set is divided, and the screen plate falling detection model is trained, the model weight parameters are fixed and saved when the accuracy and the loss function meet the preset conditions, and the pre-trained screen plate falling detection model is obtained.

3. The method according to claim 1 or 2, characterized in that, The perspective transformation processing is realized by the following formula: wherein, u , v is the original coordinate point, is the image coordinate point after perspective transformation processing, w is a parameter, and the corresponding transformed coordinate is , , in the transformation matrix, , , , represents a linear transformation, , represents a translation, , represents a perspective transformation.

4. The method of claim 2, wherein the method is characterized by, The data augmentation comprises the following steps: At least one of rotation, scaling, inversion, translation transformation is performed on the segmented sunburn picture, or one of image brightness, contrast, saturation and hue is adjusted to obtain a new image, thereby realizing data augmentation.

5. The method of claim 2, wherein the method is characterized by, The sample set is divided as follows: The sample set is divided into a training set, a validation set and a test set, the training set is used for training the model, the validation set is used for selecting hyperparameters, and the test set is used for determining the generalization ability of the model.

Images