Image recognition-based automatic speed adjustment method and system for apron feeder

Abstract

The application discloses a kind of based on image recognition's plate feeder speed automatic regulating method and system thereof.It is first obtained current frame coal type image by the first convolutional neural network model including deep and shallow feature fusion module to obtain current frame coal type image feature map, and the second convolutional neural network model including deep and shallow feature fusion module is used to obtain last frame coal type image feature map by the last frame coal type image obtained, then, the difference feature map between the corrected current frame coal type image feature map and the corrected last frame coal type image feature map obtained after respectively carrying out feature distribution correction to the current frame coal type image feature map and the last frame coal type image feature map is calculated, finally, the difference feature map is classified by classifier to obtain the classification result for indicating that the plate feeder speed of current time point should increase or should decrease.In this way, the speed regulation of plate feeder can be more adapted to actual coal type monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control, and more particularly, to a plate feeder speed automatic adjustment method and system based on image recognition. BACKGROUND

[0002] The plate feeder is also called a plate feeder, which is a kind of mechanical device for continuously conveying materials. It can continuously feed and transport materials to crushers, hopper cars or other working machines along the horizontal or inclined direction, and is suitable for loose, blocky and large feeding processes in large and medium-sized coal mining production lines, especially for conveying large, high-temperature and sharp materials. It can work reliably in high-temperature and high-humidity harsh environments, and can work reliably in open air and humid harsh environments.

[0003] When the plate feeder is used to transport the mined coal materials to the crusher, the coal materials are of different sizes, and the coal surfaces have edges and corners. When the materials roll onto the conveying chain of the plate feeder, they will be stacked to different degrees. In order to break the coal materials transported to the crusher into uniform sizes for the use of coal, the speed of the plate feeder needs to be controlled. However, since the speed of the existing plate feeder is set manually in advance, the feeder cannot control the speed according to the actual situation of the materials on the conveying chain, which affects the breaking effect and reduces the quality of coal mining.

[0004] Therefore, an optimized plate feeder speed automatic adjustment scheme is expected. SUMMARY

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a plate feeder speed automatic adjustment method and system based on image recognition. First, the current frame coal type image obtained is input into a first convolutional neural network model containing a deep and shallow feature fusion module to obtain a current frame coal type image feature map, and the previous frame coal type image obtained is input into a second convolutional neural network model containing a deep and shallow feature fusion module to obtain a previous frame coal type image feature map. Then, the difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map obtained by respectively performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map is calculated. Finally, the difference feature map is input into a classifier to obtain a classification result representing whether the speed of the plate feeder at the current time point should be increased or decreased. In this way, the speed adjustment of the plate feeder can be more adapted to the actual coal type monitoring.

[0006] According to one aspect of the present application, a plate feeder speed automatic adjustment method based on image recognition is provided, which comprises:

[0007] obtaining a current frame coal type image and a previous frame coal type image;

[0008] passing the current frame coal type image through a first convolutional neural network model comprising a deep and shallow feature fusion module to obtain a current frame coal type image feature map;

[0009] passing the previous frame coal type image through a second convolutional neural network model comprising a deep and shallow feature fusion module to obtain a previous frame coal type image feature map;

[0010] respectively performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map to obtain a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map;

[0011] calculating a differential feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; and

[0012] passing the differential feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the speed of the apron feeder at the current time point should be increased or decreased.

[0013] In the image recognition-based apron feeder speed automatic adjustment method described above, the passing of the current frame coal type image through the first convolutional neural network model comprising the deep and shallow feature fusion module to obtain the current frame coal type image feature map comprises using each layer of the first convolutional neural network model comprising the deep and shallow feature fusion module to respectively perform the following on input data in forward transmission of the layer:

[0014] performing convolution processing on the input data to obtain a convolution feature map;

[0015] performing pooling processing on the convolution feature map to obtain a pooled feature map; and

[0016] performing activation processing on the pooled feature map to obtain an activated feature map.

[0017] In the image recognition-based apron feeder speed automatic adjustment method described above, the passing of the current frame coal type image through the first convolutional neural network model comprising the deep and shallow feature fusion module to obtain the current frame coal type image feature map comprises:

[0018] extracting a shallow layer feature map from an i-th layer of the first convolutional neural network model;

[0019] extracting a deep layer feature map from a j-th layer of the first convolutional neural network model; and

[0020] using the deep and shallow feature fusion module to fuse the shallow layer feature map and the deep layer feature map to obtain the current frame coal type image feature map.

[0021] In the image recognition-based automatic speed adjustment method of the apron feeder, the i-th layer is the first to sixth layers of the first convolutional neural network model, and a ratio between the j-th layer and the i-th layer is greater than or equal to 5.

[0022] In the image recognition-based automatic speed adjustment method of the apron feeder, the feature distribution correction of the current frame of coal type image feature map and the last frame of coal type image feature map respectively to obtain the corrected current frame of coal type image feature map and the corrected last frame of coal type image feature map comprises:

[0023] The feature distribution correction of the current frame of coal type image feature map is performed according to the following formula to obtain the corrected current frame of coal type image feature map.

[0024] The formula is:

[0025]

[0026] F1 represents the current frame of coal type image feature map, represents the feature value of each position of the current frame of coal type image feature map, and is the mean of all feature values of the current frame of coal type image feature map, and log represents a logarithmic function with 2 as the base.

[0027] In the image recognition-based automatic speed adjustment method of the apron feeder, the feature distribution correction of the current frame of coal type image feature map and the last frame of coal type image feature map respectively to obtain the corrected current frame of coal type image feature map and the corrected last frame of coal type image feature map further comprises:

[0028] The feature distribution correction of the last frame of coal type image feature map is performed according to the following formula to obtain the corrected last frame of coal type image feature map.

[0029] The formula is:

[0030]

[0031] F2 represents the last frame of coal type image feature map, represents the feature value of each position of the last frame of coal type image feature map, and is the mean of all feature values of the last frame of coal type image feature map, and log represents a logarithmic function with 2 as the base.

[0032] In the image recognition-based automatic speed adjustment method of the plate feeder, the calculation of the difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map comprises:

[0033] The difference between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map is calculated to obtain the difference feature map.

[0034] In the image recognition-based automatic speed adjustment method of the plate feeder, the difference feature map is input into a classifier to obtain a classification result, and the classification result is used to indicate whether the speed of the plate feeder at the current time point should be increased or decreased.

[0035] The difference feature map is processed by the classifier using the following formula to obtain the classification result, wherein the formula is: O = softmax {(W n ,B n ):…:(W1,B1)|Project(F d )}, wherein Project(F d ) represents projecting the difference feature map into a vector, W1 to W n are weight matrices of each layer of fully connected layers, and B1 to B n are bias matrices of each layer of fully connected layers.

[0036] According to another aspect of the present application, an image recognition-based automatic speed adjustment system of a plate feeder is provided, which comprises:

[0037] A data acquisition module is configured to acquire a current frame coal type image and a previous frame coal type image.

[0038] A first convolutional encoding module is configured to input the current frame coal type image into a first convolutional neural network model comprising a deep-shallow feature fusion module to obtain a current frame coal type image feature map.

[0039] A second convolutional encoding module is configured to input the previous frame coal type image into a second convolutional neural network model comprising a deep-shallow feature fusion module to obtain a previous frame coal type image feature map.

[0040] A feature distribution correction module is configured to perform feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map.

[0041] A difference feature map calculation module is configured to calculate a difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map.

[0042] a classification result generation module configured to pass the difference feature map through a classifier to obtain a classification result, the classification result being used to indicate that the speed of the apron feeder at the current time point should be increased or should be decreased.

[0043] In the image recognition-based apron feeder speed automatic adjustment system described above, the first convolutional encoding module is further configured to use each layer of the first convolutional neural network model comprising the deep-shallow feature fusion module to respectively perform the following on the input data in the forward transmission of the layer:

[0044] convolutional processing on the input data to obtain a convolutional feature map;

[0045] pooling processing on the convolutional feature map to obtain a pooled feature map; and

[0046] activation processing on the pooled feature map to obtain an activated feature map.

[0047] Compared with the prior art, the image recognition-based apron feeder speed automatic adjustment method and system provided by the present application first pass the obtained current frame coal type image through a first convolutional neural network model comprising a deep-shallow feature fusion module to obtain a current frame coal type image feature map, and pass the obtained previous frame coal type image through a second convolutional neural network model comprising a deep-shallow feature fusion module to obtain a previous frame coal type image feature map, then calculate a difference feature map between a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map obtained by respectively performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map, and finally pass the difference feature map through a classifier to obtain a classification result used to indicate that the speed of the apron feeder at the current time point should be increased or should be decreased. In this way, the speed adjustment of the apron feeder can be more adapted to actual coal type monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and serve to explain the principles of the present application, but are not intended to limit the present application. In the drawings, like reference numerals refer to like elements or steps throughout.

[0049] Figure 1 A scene schematic diagram of the image recognition-based apron feeder speed automatic adjustment method according to the embodiments of the present application.

[0050] Figure 2 A flowchart of the image recognition-based apron feeder speed automatic adjustment method according to the embodiments of the present application.

[0051] Figure 3 The schematic diagram of the architecture of the image recognition based automatic speed adjustment method of the plate feeder according to the embodiment of the present application.

[0052] Figure 4 The block diagram of the image recognition based automatic speed adjustment system of the plate feeder according to the embodiment of the present application. DETAILED DESCRIPTION

[0053] Hereinafter, the example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0054] Scenario overview

[0055] As described above, when using the plate feeder to transfer the mined coal material to the crusher, since the coal material has large and small sizes, and the coal surface has edges and corners, the material will be accumulated to different degrees when rolling down to the conveying chain of the plate feeder. In order to make the coal material conveyed to the crusher be broken into uniform size for the use of the coal, the speed of the plate feeder needs to be controlled. However, since the speed of the existing plate feeder is set in advance by manual, this will result in that the feeder cannot control the speed according to the actual situation of the material on the conveying chain, thereby affecting the breaking effect and reducing the mining quality of the coal. Therefore, an optimized automatic speed adjustment scheme of the plate feeder is expected.

[0056] At present, deep learning and neural networks have been widely applied in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond human in the fields of image classification, object detection, semantic segmentation, text translation and other fields.

[0057] In recent years, the development of deep learning and neural networks has provided new solutions and schemes for the self-adaptive adjustment of the speed of the plate feeder.

[0058] Accordingly, considering that in actual use of the plate feeder to deliver the mined coal material to the crusher, the crushing effect of the crusher is comprehensively determined based on the size and quantity of the coal type and the feeding speed of the plate feeder, if the crushing effect of the crusher is to be stabilized within a certain quality range, the delivery speed of the plate feeder needs to be controlled in real time based on the size and data of the coal type. Based on this, in the technical solution of the present application, it is expected to apply a high-definition camera to develop an image recognition system, to convert the coal type image information of the receiving hopper of the crusher into floating-point data of the coal storage amount, and after processing by a control program, to realize automatic adjustment of the speed of the plate feeder. Specifically, in the technical solution of the present application, an artificial intelligence monitoring technology based on machine vision is adopted to extract the feature difference between the deep and shallow fusion features of the current frame and the previous frame of the coal type image, so as to control the speed of the feeder in real time. That is, the artificial intelligence technology and image recognition technology are combined and applied to the actual coal mining to construct a self-adaptive adjustment scheme of the plate feeder, to regulate the speed of the feeder based on the relative change characteristics of the coal type image features in time sequence rather than based on the absolute amount. In this way, the speed adjustment of the plate feeder can be more adapted to the actual coal type monitoring, to improve the accuracy of the speed control of the plate feeder, and thus to ensure the crushing effect and improve the mining quality of the coal.

[0059] Specifically, in the technical solution of the present application, first, the current frame coal type image and the previous frame coal type image are acquired by the camera of the high-definition camera. Then, a convolutional neural network model with excellent performance in image feature extraction is used as a feature extractor to extract the local implicit features of the coal type image. In particular, in the technical solution of the present application, it is considered that when the features of the coal type image are extracted, the shape, contour, texture and other shallow features of the coal type will have an impact on crushing, and when the convolutional neural network is encoded, the shallow features will become blurred or even be overwhelmed by noise as the depth increases. Therefore, in the technical solution of the present application, the coal type image is processed by a convolutional neural network model containing a deep and shallow feature fusion module. Specifically, the current frame coal type image is processed by a first convolutional neural network model containing a deep and shallow feature fusion module to obtain a current frame coal type image feature map, and the previous frame coal type image is processed by a second convolutional neural network model containing a deep and shallow feature fusion module to obtain a previous frame coal type image feature map. It can be understood that compared with a standard convolutional neural network model, the convolutional neural network model containing a deep and shallow feature fusion module according to the present application can preserve the shallow features and deep features in the coal type image, so as to make the feature information more abundant and different depth features can be preserved, to improve the accuracy of coal type recognition.

[0060] Further, considering that the absolute quantity of the coal type feature information at the current time is used to control the speed of the plate feeder based on the coal type based implicit feature information, the accuracy of the control is not high. Therefore, in the technical solution of the present application, the difference feature map between the current frame coal type image feature map and the last frame coal type image feature map is calculated to represent the change feature of the coal type implicit feature in the time dimension, and the change feature is used to adjust the speed of the plate feeder at the current time point in a relative quantity, so as to improve the accuracy of the speed regulation of the plate feeder. That is, the difference feature map is further used as a classification feature map for classification processing in the classifier to obtain a classification result indicating that the speed of the plate feeder at the current time point should be increased or decreased.

[0061] In particular, in the technical solution of the present application, when the difference feature map between the current frame coal type image feature map and the last frame coal type image feature map is calculated as a classification feature map, it is expected that the feature distribution of the current frame coal type image feature map and the last frame coal type image feature map has as small spatial position error as possible in the high-dimensional feature space, so that the difference feature map obtained by point-by-point difference calculation can accurately express the image difference feature.

[0062] Here, in the technical solution of the present application, considering that the current frame coal type image and the last frame coal type image are homologous images passing through the same model branch, the current frame coal type image feature map and the last frame coal type image feature map have certain correspondence, and therefore, the current frame coal type image feature map and the last frame coal type image feature map are respectively corrected based on the relative class angle probability information, which is represented as:

[0063]

[0064]

[0065]

[0066] wherein and are the feature values of the current frame coal type image F1 and the last frame coal type image F2, and and are the mean values of all feature values of the current frame coal type image F1 and the last frame coal type image F2.

[0067] Here, the relative class angle probability information represents correcting the geometric dilution of precision of the spatial position error of the feature distribution of the current frame coal type image F1 and the last frame coal type image F2 in the high-dimensional feature space by the relative class angle probability information representation between the current frame coal type image F1 and the last frame coal type image F2, so that in the case that the current frame coal type image F1 and the last frame coal type image F2 have a certain correspondence, the implicit contextual correspondence correction of the features is performed by point-by-point regression by position based on the distribution constraint of the feature value distribution of each position of the current frame coal type image F1 and the last frame coal type image F2 compared to each other as a whole, so as to improve the expression effect of the difference feature map obtained by point-by-point difference calculation of the current frame coal type image F1 and the last frame coal type image F2. In this way, the speed regulation of the apron feeder can be more adapted to the actual coal type monitoring situation, so as to improve the accuracy of the apron feeder speed control, thereby ensuring the crushing effect and improving the coal mining quality.

[0068] Based on this, the application provides an apron feeder speed automatic regulation method based on image recognition, which comprises the following steps: acquiring a current frame coal type image and a last frame coal type image; inputting the current frame coal type image into a first convolutional neural network model comprising a deep and shallow feature fusion module to obtain a current frame coal type image feature map; inputting the last frame coal type image into a second convolutional neural network model comprising a deep and shallow feature fusion module to obtain a last frame coal type image feature map; performing feature distribution correction on the current frame coal type image feature map and the last frame coal type image feature map respectively to obtain a corrected current frame coal type image feature map and a corrected last frame coal type image feature map; calculating a difference feature map between the corrected current frame coal type image feature map and the corrected last frame coal type image feature map; and inputting the difference feature map into a classifier to obtain a classification result, wherein the classification result is used to represent whether the apron feeder speed at the current time point should be increased or decreased.

[0069] Figure 1 The application scenario diagram of the apron feeder speed automatic regulation method based on image recognition according to the embodiment of the application is shown in FIG. 1. Figure 1 As shown in FIG. 1, in the application scenario, a current frame coal type image (for example, D1 shown in FIG. 2) and a last frame coal type image (for example, D2 shown in FIG. 2) are acquired by a camera (for example, C shown in FIG. 2) of a high-definition camera. Figure 1 Figure 1 Figure 1 Figure 1 ​​​In the S shown, the server is able to generate a classification result indicating whether the speed of the plate feeder should be increased or decreased at the current time point based on the image recognition-based automatic speed adjustment algorithm for the plate feeder.

[0070] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0071] Exemplary methods

[0072] Figure 2 This is a flowchart of an image recognition-based automatic speed adjustment method for a plate feeder according to an embodiment of this application. Figure 2 As shown, the automatic speed adjustment method for a plate feeder based on image recognition according to an embodiment of this application includes the following steps: S110, acquiring the current frame coal type image and the previous frame coal type image; S120, passing the current frame coal type image through a first convolutional neural network model including a deep and shallow feature fusion module to obtain a feature map of the current frame coal type image; S130, passing the previous frame coal type image through a second convolutional neural network model including a deep and shallow feature fusion module to obtain a feature map of the previous frame coal type image; S140, performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map; S150, calculating the difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; and S160, passing the difference feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the speed of the plate feeder at the current time point should increase or decrease.

[0073] Figure 3 This is a schematic diagram of the architecture of an image recognition-based automatic speed adjustment method for a plate feeder according to an embodiment of this application. Figure 3 As shown, in this network architecture, firstly, the current frame coal type image and the previous frame coal type image are acquired; then, the current frame coal type image is processed through a first convolutional neural network model containing a deep and shallow feature fusion module to obtain a feature map of the current frame coal type image; next, the previous frame coal type image is processed through a second convolutional neural network model containing a deep and shallow feature fusion module to obtain a feature map of the previous frame coal type image; then, feature distribution correction is performed on the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain corrected current frame coal type image feature map and corrected previous frame coal type image feature map; next, the difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map is calculated; finally, the difference feature map is processed through a classifier to obtain a classification result, which is used to indicate whether the speed of the plate feeder at the current time point should increase or decrease.

[0074] More specifically, in step S110, the current frame coal type image and the previous frame coal type image are obtained. Considering that when the plate feeder is used to transport the mined coal material to the crusher, the crushing effect of the crusher is determined by the size and quantity of the coal type and the feeding speed of the plate feeder, so if you want to stabilize the crushing effect of the crusher within a certain quality range, you need to control the feeding speed of the plate feeder in real time based on the size and quantity of the coal type. Based on this, in the technical solution of the present application, an artificial intelligence monitoring technology based on machine vision is used to extract the feature difference between the deep and shallow fusion features of the current frame and the previous frame of the coal type image, so as to control the speed of the feeder in real time.

[0075] In particular, in the technical solution of the present application, it is considered that when the coal type image is feature-extracted, the shape, contour, texture and other shallow features of the coal type will affect the crushing, and when the convolutional neural network is encoded, the shallow features will become blurred or even be overwhelmed by noise as the depth increases. Therefore, in the technical solution of the present application, a convolutional neural network model containing a deep-shallow feature fusion module is used to process the coal type image. It can be understood that compared with a standard convolutional neural network model, the convolutional neural network model containing a deep-shallow feature fusion module according to the present application can retain the shallow features and deep features in the coal type image, so as to not only make the feature information more abundant, but also retain the features of different depths, thereby improving the accuracy of coal type recognition.

[0076] More specifically, in step S120, the current frame coal type image is processed by a first convolutional neural network model containing a deep-shallow feature fusion module to obtain a current frame coal type image feature map.

[0077] Correspondingly, in one specific example, in the image recognition-based automatic speed adjustment method of the plate feeder, the current frame coal type image is processed by a first convolutional neural network model containing a deep-shallow feature fusion module to obtain a current frame coal type image feature map, which includes using each layer of the first convolutional neural network model containing a deep-shallow feature fusion module to respectively perform the following operations on the input data in the forward transmission of the layer: performing convolution processing on the input data to obtain a convolution feature map; performing pooling processing on the convolution feature map to obtain a pooling feature map; and performing activation processing on the pooling feature map to obtain an activation feature map.

[0078] Accordingly, in one specific example, in the image recognition-based apron feeder speed automatic adjustment method, the step of passing the current frame coal type image through a first convolutional neural network model comprising a deep-shallow feature fusion module to obtain a current frame coal type image feature map comprises: extracting a shallow layer feature map from an i-th layer of the first convolutional neural network model; extracting a deep layer feature map from a j-th layer of the first convolutional neural network model; and fusing the shallow layer feature map and the deep layer feature map using the deep-shallow feature fusion module to obtain the current frame coal type image feature map.

[0079] Accordingly, in one specific example, in the image recognition-based apron feeder speed automatic adjustment method, the i-th layer is the first layer to the sixth layer of the first convolutional neural network model; and a ratio between the j-th layer and the i-th layer is greater than or equal to 5.

[0080] More specifically, in step S130, the previous frame coal type image is passed through a second convolutional neural network model comprising a deep-shallow feature fusion module to obtain a previous frame coal type image feature map.

[0081] More specifically, in step S140, the current frame coal type image feature map and the previous frame coal type image feature map are respectively subjected to feature distribution correction to obtain a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map.

[0082] In particular, in the technical solution of the present application, when the difference feature map between the current frame coal type image feature map and the previous frame coal type image feature map is calculated as a classification feature map, it is expected that the feature distribution of the current frame coal type image feature map and the previous frame coal type image feature map has as small a spatial position error as possible in a high-dimensional feature space, so that the difference feature map obtained by point-by-point difference calculation can accurately express the image difference features. Here, in the technical solution of the present application, considering that the current frame coal type image and the previous frame coal type image are homologous images passing through the same model branch, the current frame coal type image feature map and the previous frame coal type image feature map have a certain correspondence, and thus the current frame coal type image feature map and the previous frame coal type image feature map are respectively subjected to relative class angle probability information representation correction.

[0083] Accordingly, in one specific example, in the image recognition-based apron feeder speed automatic adjustment method, the step of respectively subjecting the current frame coal type image feature map and the previous frame coal type image feature map to feature distribution correction to obtain a corrected current frame coal type image feature map and a corrected previous frame coal type image feature map comprises: subjecting the current frame coal type image feature map to feature distribution correction according to the following formula to obtain the corrected current frame coal type image feature map; wherein the formula is:

[0084]

[0085] wherein F1 represents the current frame coal type image feature map, represents the feature value of each position of the current frame coal type image feature map, and is the mean of all feature values of the current frame coal type image feature map, and log represents the logarithmic function with 2 as the base.

[0086] Correspondingly, in one specific example, in the image recognition based speed automatic adjustment method of the apron feeder, the feature distribution correction of the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map further comprises: performing feature distribution correction on the previous frame coal type image feature map according to the following formula to obtain the corrected previous frame coal type image feature map; wherein the formula is:

[0087]

[0088] wherein F2 represents the previous frame coal type image feature map, represents the feature value of each position of the previous frame coal type image feature map, and is the mean of all feature values of the previous frame coal type image feature map, and log represents the logarithmic function with 2 as the base.

[0089] Here, the relative class angle probability information represents the geometric precision dilution of the spatial position error of the feature distribution of the current frame coal type image F1 and the previous frame coal type image F2 in the high-dimensional feature space through the relative class angle probability information representation between the current frame coal type image F1 and the previous frame coal type image F2, so that in the case that the current frame coal type image F1 and the previous frame coal type image F2 have a certain correspondence, the implicit context correspondence correction of the features is performed through point-by-point regression by position based on the distribution constraint of the feature value distribution of each position of the current frame coal type image F1 and the previous frame coal type image F2 compared with each other as a whole, so as to improve the expression effect of the difference feature map obtained by point-by-point difference calculation of the current frame coal type image F1 and the previous frame coal type image F2. In this way, the speed adjustment of the apron feeder can be more adapted to the actual coal type monitoring situation, so as to improve the precision of the apron feeder speed control, and further to ensure the crushing effect and improve the coal mining quality.

[0090] More specifically, in step S150, a difference feature map is calculated between the corrected current frame coal morphology image feature map and the corrected previous frame coal morphology image feature map. Furthermore, considering that controlling the speed of the plate feeder based on the implicit feature information of the coal morphology, controlling based on the absolute amount of coal morphology feature information at the current moment would result in low control accuracy, the technical solution of this application calculates the difference feature map between the current frame coal morphology image feature map and the previous frame coal morphology image feature map to represent the change characteristics of the implicit features of the coal morphology in the time dimension. This change characteristic is used to relatively adjust the speed of the feeder at the current time point, thereby improving the accuracy of the plate feeder speed control.

[0091] Accordingly, in a specific example, in the image recognition-based automatic speed adjustment method for plate feeders, the step of calculating the differential feature map between the corrected current frame coal pattern image feature map and the corrected previous frame coal pattern image feature map includes: calculating the positional difference between the corrected current frame coal pattern image feature map and the corrected previous frame coal pattern image feature map to obtain the differential feature map.

[0092] More specifically, in step S160, the differential feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the speed of the plate feeder at the current time point should be increased or decreased.

[0093] Accordingly, in a specific example, the step of passing the differential feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the speed of the plate feeder at the current time point should increase or decrease, includes: processing the differential feature map using the classifier with the following formula to obtain the classification result, wherein the formula is:

[0094] O = softmax{(W n B n ):…:(W1,B1)|Project(F d )}, where Project(F d )surface

[0095] The difference feature map is projected as a vector, W1 to W... n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

[0096] In summary, the image recognition-based automatic speed adjustment method for a plate feeder, based on embodiments of this application, firstly obtains a feature map of the current frame coal morphology image by passing the acquired current frame coal morphology image through a first convolutional neural network model containing a deep and shallow feature fusion module, and then obtains a feature map of the previous frame coal morphology image by passing the acquired previous frame coal morphology image through a second convolutional neural network model containing a deep and shallow feature fusion module. Next, it calculates the difference feature map between the corrected current frame coal morphology image feature map and the corrected previous frame coal morphology image feature map after feature distribution correction of both the current and previous frame coal morphology image feature maps. Finally, it passes the difference feature map through a classifier to obtain a classification result indicating whether the plate feeder speed should increase or decrease at the current time point. This allows the speed adjustment of the plate feeder to be more adapted to actual coal morphology monitoring.

[0097] Exemplary System

[0098] Figure 4 This is a block diagram of an image recognition-based automatic speed adjustment system 100 for a plate feeder according to an embodiment of this application. Figure 4 As shown, the image recognition-based automatic speed adjustment system 100 for a plate feeder according to an embodiment of this application includes: a data acquisition module 110 for acquiring the current frame coal pattern image and the previous frame coal pattern image; a first convolutional coding module 120 for passing the current frame coal pattern image through a first convolutional neural network model including a deep and shallow feature fusion module to obtain a feature map of the current frame coal pattern image; a second convolutional coding module 130 for passing the previous frame coal pattern image through a second convolutional neural network model including a deep and shallow feature fusion module to obtain a feature map of the previous frame coal pattern image; and a feature distribution correction module. 140 is used to perform feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; 150 is used to calculate the difference feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; and 160 is used to pass the difference feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the speed of the plate feeder at the current time point should be increased or decreased.

[0099] In an example, in the image recognition based apron feeder speed automatic adjustment system 100 described above, the first convolutional encoding module 120 is further configured to: use layers of the first convolutional neural network model comprising the deep and shallow feature fusion module to respectively perform the following on the input data in the forward transmission of the layers: perform convolutional processing on the input data to obtain a convolutional feature map; perform pooling processing on the convolutional feature map to obtain a pooled feature map; and perform activation processing on the pooled feature map to obtain an activated feature map.

[0100] In an example, in the image recognition based apron feeder speed automatic adjustment system 100 described above, the first convolutional encoding module 120 is further configured to: extract a shallow layer feature map from an i-th layer of the first convolutional neural network model; extract a deep layer feature map from a j-th layer of the first convolutional neural network model; and fuse the shallow layer feature map and the deep layer feature map using the deep and shallow feature fusion module to obtain the current frame coal type image feature map.

[0101] In an example, in the image recognition based apron feeder speed automatic adjustment system 100 described above, the i-th layer is a first layer to a sixth layer of the first convolutional neural network model; and a ratio between the j-th layer and the i-th layer is greater than or equal to 5.

[0102] In an example, in the image recognition based apron feeder speed automatic adjustment system 100 described above, the feature distribution correction module 140 is further configured to: perform feature distribution correction on the current frame coal type image feature map using the following formula to obtain the corrected current frame coal type image feature map; wherein the formula is:

[0103]

[0104] wherein F1 represents the current frame coal type image feature map, represents a feature value of each position of the current frame coal type image feature map, and is a mean value of all feature values of the current frame coal type image feature map, and log represents a logarithmic function with 2 as a base.

[0105] In an example, in the image recognition based apron feeder speed automatic adjustment system 100 described above, the feature distribution correction module 140 is further configured to: perform feature distribution correction on the previous frame coal type image feature map using the following formula to obtain the corrected previous frame coal type image feature map; wherein the formula is:

[0106]

[0107]

[0108] Where F2 represents the feature map of the previous frame coal type image. This represents the feature values ​​at each position in the feature map of the previous frame's coal type image, and It is the mean of all feature values ​​of the previous frame coal type image feature map, and log represents the logarithmic function with base 2.

[0109] In one example, in the above-mentioned image recognition-based automatic speed adjustment system 100 for plate feeders, the differential feature map calculation module 150 is used to: calculate the positional difference between the corrected current frame coal pattern image feature map and the corrected previous frame coal pattern image feature map to obtain the differential feature map.

[0110] In one example, in the image recognition-based automatic speed adjustment system 100 for a plate feeder described above, the classification result generation module 160 is further configured to: process the differential feature map using the classifier according to the following formula to obtain the classification result, wherein the formula is: O = softmax{(W n B n ):…:(W1,B1)|Project(F d )}, where Project(F d ) indicates that the differential feature map is projected into a vector, W1 to W n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

[0111] Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the above-described image recognition-based automatic speed adjustment system 100 for plate feeders have been referenced above. Figures 1 to 3 The method for automatic speed adjustment of plate feeders based on image recognition has been described in detail, and therefore, its repeated description will be omitted.

[0112] As described above, the image recognition-based automatic speed adjustment system 100 for a plate feeder according to embodiments of this application can be implemented in various wireless terminals, such as servers for image recognition-based automatic speed adjustment algorithms for plate feeders. In one example, the image recognition-based automatic speed adjustment system 100 for a plate feeder according to embodiments of this application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the image recognition-based automatic speed adjustment system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the image recognition-based automatic speed adjustment system 100 can also be one of many hardware modules of the wireless terminal.

[0113] Alternatively, in another example, the image recognition based apron speed automatic adjustment system 100 and the wireless terminal can also be separate devices, and the image recognition based apron speed automatic adjustment system 100 can be connected to the wireless terminal through a wired and / or wireless network and transmit interactive information in an agreed data format.

[0114] The above describes the basic principles of the present application in combination with specific examples, but it should be noted that the advantages, advantages, effects, etc. mentioned in the present application are only examples and not limitations, and these advantages, advantages, effects, etc. cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details disclosed are only for the purpose of example and understanding, and are not limited to the above specific details. The above specific details do not limit the present application to be necessarily implemented with the above specific details.

[0115] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have", etc. are open-ended words, mean "include but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0116] It should also be noted that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0117] The above description of the disclosed aspects is provided so that any person skilled in the art can make or use the present application. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0118] The above description has been given for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain modifications, alterations, changes, additions and sub-combinations thereof.

Claims

1. A method for automatically adjusting the speed of a plate feeder based on image recognition, characterized in that, include: Get the current frame coal pattern image and the previous frame coal pattern image; The current frame coal pattern image is processed through a first convolutional neural network model containing a deep and shallow feature fusion module to obtain the feature map of the current frame coal pattern image; The previous frame coal morph image is processed through a second convolutional neural network model containing a deep and shallow feature fusion module to obtain the feature map of the previous frame coal morph image; The feature distribution of the current frame coal type image feature map and the previous frame coal type image feature map are respectively corrected to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; Calculate the difference feature map between the corrected current frame coal pattern image feature map and the corrected previous frame coal pattern image feature map; as well as The difference feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the speed of the plate feeder at the current time point should be increased or decreased. The step of performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map includes: The feature distribution of the current frame coal pattern image feature map is corrected using the following formula to obtain the corrected current frame coal pattern image feature map; The formula is as follows: Where F1 represents the feature map of the current frame coal type image. This represents the feature values ​​at each position in the feature map of the current frame coal type image, and It is the mean of all feature values ​​of the current frame coal type image feature map, and log represents the logarithmic function with base 2; The feature distribution of the previous frame coal morph image feature map is corrected using the following formula to obtain the corrected previous frame coal morph image feature map; The formula is as follows: Where F2 represents the feature map of the previous frame coal type image. This represents the feature values ​​at each position in the feature map of the previous frame's coal type image, and It is the mean of all feature values ​​of the previous frame coal type image feature map, and log represents the logarithmic function with base 2.

2. The automatic speed adjustment method for a plate feeder based on image recognition according to claim 1, characterized in that, The step of obtaining a feature map of the current coal type image by passing the current coal type image through a first convolutional neural network model containing a deep and shallow feature fusion module includes: using each layer of the first convolutional neural network model containing the deep and shallow feature fusion module to process the input data during the forward propagation of the layer: The input data is subjected to convolution processing to obtain a convolutional feature map; The convolutional feature map is subjected to pooling processing to obtain a pooled feature map; and The pooled feature map is activated to obtain an activated feature map.

3. The automatic speed adjustment method for a plate feeder based on image recognition according to claim 2, characterized in that, The step of passing the current frame coal pattern image through a first convolutional neural network model containing a deep and shallow feature fusion module to obtain the feature map of the current frame coal pattern image includes: Extract shallow feature maps from the i-th layer of the first convolutional neural network model; Extract the deep feature map from the j-th layer of the first convolutional neural network model; and The shallow and deep feature fusion module is used to fuse the shallow feature map and the deep feature map to obtain the current frame coal pattern image feature map.

4. The automatic speed adjustment method for a plate feeder based on image recognition according to claim 3, characterized in that, The i-th layer refers to the first to sixth layers of the first convolutional neural network model; the ratio between the j-th layer and the i-th layer is greater than or equal to 5.

5. The automatic speed adjustment method for a plate feeder based on image recognition according to claim 4, characterized in that, The calculation of the difference feature map between the corrected current frame coal morph image feature map and the corrected previous frame coal morph image feature map includes: The positional difference between the corrected current frame coal pattern image feature map and the corrected previous frame coal pattern image feature map is calculated to obtain the difference feature map.

6. The automatic speed adjustment method for a plate feeder based on image recognition according to claim 5, characterized in that, The step of passing the differential feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the speed of the plate feeder at the current time point should increase or decrease, includes: The classifier is used to process the difference feature map using the following formula to obtain the classification result, wherein the formula is: O = softmax{(W n B n ):…:(W1,B1)|Project(F d )}, where Project(F d ) indicates that the differential feature map is projected into a vector, W1 to W n Here are the weight matrices for each fully connected layer, B1 to B... n This represents the bias matrix of each fully connected layer.

7. An automatic speed adjustment system for a plate feeder based on image recognition, characterized in that, include: The data acquisition module is used to acquire the current frame coal type image and the previous frame coal type image; The first convolutional coding module is used to pass the current frame coal pattern image through a first convolutional neural network model containing a deep and shallow feature fusion module to obtain the current frame coal pattern image feature map; The second convolutional coding module is used to pass the previous frame coal type image through a second convolutional neural network model containing a deep and shallow feature fusion module to obtain the feature map of the previous frame coal type image; The feature distribution correction module is used to perform feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map respectively to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; The differential feature map calculation module is used to calculate the differential feature map between the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map; as well as The classification result generation module is used to pass the differential feature map through a classifier to obtain a classification result, which is used to indicate whether the speed of the plate feeder at the current time point should be increased or decreased. The step of performing feature distribution correction on the current frame coal type image feature map and the previous frame coal type image feature map to obtain the corrected current frame coal type image feature map and the corrected previous frame coal type image feature map includes: The feature distribution of the current frame coal pattern image feature map is corrected using the following formula to obtain the corrected current frame coal pattern image feature map; The formula is as follows: Where F1 represents the feature map of the current frame coal type image. This represents the feature values ​​at each position in the feature map of the current frame coal type image, and It is the mean of all feature values ​​of the current frame coal type image feature map, and log represents the logarithmic function with base 2; The feature distribution of the previous frame coal morph image feature map is corrected using the following formula to obtain the corrected previous frame coal morph image feature map; The formula is as follows: Where F2 represents the feature map of the previous frame coal type image. This represents the feature values ​​at each position in the feature map of the previous frame's coal type image, and It is the mean of all feature values ​​of the previous frame coal type image feature map, and log represents the logarithmic function with base 2.

8. The automatic speed adjustment system for a plate feeder based on image recognition according to claim 7, characterized in that, The first convolutional coding module is further configured to: process the input data using each layer of the first convolutional neural network model containing the deep and shallow feature fusion module during the forward propagation of the layer. The input data is subjected to convolution processing to obtain a convolutional feature map; The convolutional feature map is subjected to pooling processing to obtain a pooled feature map; as well as The pooled feature map is activated to obtain an activated feature map.

Images