Bulk recognition system and method based on feature matching

Abstract

The application discloses a large block identification system and method based on feature matching. First, the obtained detection raw coal image is subjected to a dust removal generator based on an adversarial generation network to obtain a generated detection raw coal image. Then, the generated detection raw coal image and a reference raw coal image are subjected to a twin network model comprising a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. Then, feature matching is performed on the detection feature map and the reference feature map, and feature distribution correction is performed to obtain a corrected differential feature map. Finally, the corrected differential feature map is subjected to a decoder to obtain a decoding value for representing the feature matching difference degree, and based on the decoding value, it is determined whether a large block exists in the detection raw coal image. In this way, the large block of raw coal on the plate feeder can be accurately detected, and an alarm can be issued when the large block of raw coal material is detected.

Description

Technical Field

[0001] This application relates to the field of intelligent recognition technology, and more specifically, to a large-block recognition system and method based on feature matching. Background Technology

[0002] A plate feeder, also known as a plate feeder, is a machine that continuously conveys materials. It can continuously feed and transfer materials to crushers, hopper cars, or other working machines in a horizontal or inclined direction. It is suitable for the feeding process of loose, lumpy, and large-volume materials in large and medium-sized raw coal mining production lines. It is especially suitable for conveying large, hot, and sharp materials. It can work reliably in harsh environments with high temperature and humidity, and can also work reliably in open-air and humid environments.

[0003] Currently, during the raw coal mining process, when using plate feeders to transfer the mined raw coal materials to the crushing system, the raw coal materials vary in size and shape. Therefore, if the raw coal materials are too large, it may cause blockage and shutdown of the crushing system, leading to serious consequences.

[0004] Therefore, an optimized feature-matching-based large-piece identification system is desired, which can detect and identify large pieces of raw coal on a plate feeder, so as to generate an early warning message when large pieces of raw coal are detected, and prevent the crushing system from being blocked and shut down. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a large-lump identification system and method based on feature matching. First, the acquired raw coal image is processed by a dust removal generator based on a generative adversarial network to generate a new raw coal image. Next, the generated raw coal image and a reference raw coal image are processed through a Siamese network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. Then, feature matching and feature distribution correction are performed on the detection feature map and the reference feature map to obtain a corrected differential feature map. Finally, the corrected differential feature map is processed by a decoder to obtain a decoded value representing the feature matching difference. Based on the decoded value, it is determined whether large lumps exist in the raw coal image. In this way, large lumps of raw coal on a plate feeder can be accurately detected, and an alarm can be issued when large lumps of raw coal are detected.

[0006] According to one aspect of this application, a large-block recognition system based on feature matching is provided, comprising:

[0007] The image acquisition module is used to acquire images of the raw coal to be detected and images of reference raw coal, wherein large blocks are present in the reference raw coal image;

[0008] The dust removal module is used to generate a new raw coal image by passing the detected raw coal image through a dust removal generator based on a generative adversarial network.

[0009] The twin detection module is used to pass the generated raw coal image and the reference raw coal image through a twin network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map;

[0010] The difference module is used to perform feature matching on the detected feature map and the reference feature map to obtain a difference feature map;

[0011] The feature distribution correction module is used to perform feature distribution correction on the difference feature map to obtain a corrected difference feature map;

[0012] A decoding regression module is used to pass the corrected difference feature map through a decoder to obtain a decoded value representing the feature matching difference; and

[0013] The recognition result generation module is used to determine whether there are large pieces in the detected raw coal image based on the decoded value.

[0014] In the aforementioned feature-matching-based large-block recognition system, the Generative Adversarial Network (GAN) includes a discriminator and a generator. The dust removal module is further configured to input the detected raw coal image into the dust removal generator based on the GAN so that the generator of the GAN generates the detected raw coal image through deconvolutional encoding.

[0015] In the aforementioned feature-matching-based block recognition system, the first image encoder and the second image encoder have the same network structure, wherein the first image encoder and the second image encoder are convolutional neural network models using spatial attention mechanisms.

[0016] In the above-described feature-matching-based large-block recognition system, the difference module is further used to calculate the positional difference between the detected feature map and the reference feature map to perform feature matching on the detected feature map and the reference feature map to obtain the difference feature map.

[0017] In the above-mentioned feature-matching-based large-block recognition system, the feature distribution correction module is further used to: perform feature distribution correction on the differential feature map using the following formula to obtain the corrected differential feature map;

[0018] The formula is as follows:

[0019]

[0020] Among them M and M cThese are the feature matrices along the channel dimension of the difference feature map and the corrected difference feature map, respectively, and ReLU(·) represents the ReLU activation function. This indicates matrix multiplication, where the division between the numerator and denominator matrices is a positional division of the matrix eigenvalues. exp(·) represents matrix exponentiation, which means calculating the natural exponential function value raised to the power of the eigenvalues ​​at each position in the matrix.

[0021] In the above-mentioned feature-matching-based large-block recognition system, the decoding regression module is further configured to: use multiple fully connected layers of the decoder to perform decoding regression on the corrected differential feature map using the following formula to obtain the decoded value;

[0022] The formula is as follows:

[0023]

[0024] Where X is the corrected differential feature map, Y is the decoded value, and W is the weight matrix. This represents matrix multiplication.

[0025] In the aforementioned feature-matching-based large-block recognition system, the recognition result generation module is further used to determine that a large block exists in the detected raw coal image in response to the decoded value being greater than or equal to a predetermined threshold.

[0026] According to another aspect of this application, a large-block recognition method based on feature matching is provided, comprising:

[0027] Acquire a raw coal image and a reference raw coal image, wherein the reference raw coal image contains large blocks;

[0028] The detected raw coal image is passed through a dust removal generator based on a generative adversarial network to obtain a generated detected raw coal image;

[0029] The generated raw coal image and the reference raw coal image are passed through a twin network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map.

[0030] Feature matching is performed on the detected feature map and the reference feature map to obtain a differential feature map;

[0031] The differential feature map is subjected to feature distribution correction to obtain a corrected differential feature map;

[0032] The corrected differential feature map is passed through a decoder to obtain decoded values ​​representing the feature matching difference; and

[0033] Based on the decoded value, it is determined whether there are large pieces in the detected raw coal image.

[0034] In the above-mentioned large-block recognition method based on feature matching, the adversarial generative network includes a discriminator and a generator. The step of passing the detected raw coal image through the dust removal generator based on the adversarial generative network to obtain a generated detected raw coal image further includes: inputting the detected raw coal image into the dust removal generator based on the adversarial generative network so that the generator of the adversarial generative network generates the generated detected raw coal image by deconvolution encoding.

[0035] In the above-mentioned feature-matching-based large-block recognition method, the first image encoder and the second image encoder have the same network structure, wherein the first image encoder and the second image encoder are convolutional neural network models using spatial attention mechanism.

[0036] Compared with existing technologies, this application provides a large coal lump recognition system and method based on feature matching. First, the acquired raw coal image is processed by a dust removal generator based on a generative adversarial network to generate a new raw coal image. Next, the generated raw coal image and a reference raw coal image are processed through a Siamese network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. Then, feature matching and feature distribution correction are performed on the detection feature map and the reference feature map to obtain a corrected differential feature map. Finally, the corrected differential feature map is processed by a decoder to obtain a decoded value representing the feature matching difference. Based on the decoded value, the presence of large lumps in the raw coal image is determined. In this way, large lumps of raw coal on a plate feeder can be accurately detected, and an alarm can be issued when large lumps of raw coal are detected. Attached Figure Description

[0037] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0038] Figure 1 This is an application scenario diagram of a feature-matching-based large block recognition system according to an embodiment of this application.

[0039] Figure 2 This is a block diagram of a feature-matching-based large block recognition system according to an embodiment of this application.

[0040] Figure 3 This is a flowchart of a feature-matching-based large block recognition method according to an embodiment of this application.

[0041] Figure 4 This is a schematic diagram of the system architecture of a feature-matching-based large block recognition method according to an embodiment of this application. Detailed Implementation

[0042] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0043] Scene Overview

[0044] A plate feeder, also known as a plate feeder, is a machine that continuously conveys materials. It can continuously feed and transfer materials to crushers, hopper cars, or other working machines in a horizontal or inclined direction. It is suitable for the feeding process of loose, lumpy, and large-volume materials in large and medium-sized raw coal mining production lines. It is especially suitable for conveying large, hot, and sharp materials. It can work reliably in harsh environments with high temperature and humidity, and can also work reliably in open-air and humid environments.

[0045] Currently, during raw coal mining, when using plate feeders to transfer mined raw coal materials to the crushing system, the varying sizes of the raw coal materials can cause blockages and shutdowns in the crushing system, leading to serious consequences. Therefore, an optimized feature-matching-based large-piece identification system is desired. This system should be able to detect and identify large pieces of raw coal on the plate feeder, generating an early warning message when large pieces are detected to prevent blockages and shutdowns in the crushing system.

[0046] Currently, deep learning and neural networks have been widely applied in fields such as computer vision, natural language processing, and speech signal processing. Furthermore, deep learning and neural networks have demonstrated near-human or even surpassed human-level performance in areas such as image classification, object detection, semantic segmentation, and text translation.

[0047] In recent years, the development of deep learning and neural networks has provided new solutions and approaches for large-block recognition based on feature matching.

[0048] Accordingly, considering that the most effective and convenient method for detecting large pieces of raw coal is to rely on the recognition of raw coal images on the plate feeder, it is difficult to detect large pieces of raw coal based solely on the information in the raw coal images due to the different proportions of the images taken by different shooting methods. Therefore, in the technical solution of this application, machine vision artificial intelligence detection technology is used to preprocess the raw coal images on the plate feeder to extract the feature matching difference information between the raw coal detection image and the reference image of large raw coal pieces in a high-dimensional feature space. Then, based on the correlation between the feature matching difference degree and the coal block volume difference, feature matching is performed under actual conditions to automatically identify large raw coal pieces on the plate feeder. This enables accurate detection of large raw coal pieces on the plate feeder, issuing an alarm when large raw coal pieces are detected to remind operators to intervene and prevent the crushing system from becoming blocked and shut down.

[0049] Specifically, in the technical solution of this application, firstly, a detection image of raw coal and a reference image of raw coal are acquired through a camera. Here, the reference image of raw coal contains large pieces. Next, considering that the detection image of raw coal is affected by a large amount of dust and other small environmental particles present in the coal mining site, which can blur the implicit features of the detection image during feature extraction due to interference from external environmental factors, thus reducing the accuracy of detecting large pieces on the plate feeder, the technical solution of this application uses a dust removal generator based on a generative adversarial network (GAN) to perform dust removal processing before feature extraction. Specifically, the detection image of raw coal is input into the dust removal generator based on the GAN so that the generator uses deconvolutional encoding to generate the generated detection image of raw coal. Specifically, the adversarial generative network here includes a discriminator and a generator, wherein the generator is used to generate a deduplicated image, the discriminator is used to calculate the difference between the deduplicated image and the real image, and the network parameters of the generator are updated by the gradient descent directional propagation algorithm to obtain a generator with deduplicated function.

[0050] Furthermore, a convolutional neural network model, which excels in extracting local latent features from images, is used for feature mining of the generated detection image of raw coal and the reference image of raw coal. Specifically, considering that when detecting large pieces of raw coal to avoid clogging the crushing system, it is necessary to focus on the feature information such as the size of the raw coal material on the plate feeder and filter out interfering feature information unrelated to the detection of large pieces of raw coal material on the plate feeder, and given that the attention mechanism can select the focus position, the technical solution of this application further uses a spatial attention mechanism for feature extraction of the generated detection image of raw coal and the reference image of raw coal. That is, specifically, the generated detection image of raw coal and the reference image of raw coal are passed through a Siamese network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. It is worth mentioning that here, the first image encoder and the second image encoder have the same network structure, and the first image encoder and the second image encoder are convolutional neural network models using a spatial attention mechanism. Specifically, by using the twin network model containing the same network structure of the image encoder to extract the spatial location features of the generated raw coal image and the reference raw coal image respectively, it is possible to mine the implicit feature information of the raw coal material focused on the spatial location of the plate feeder in the image features of the two that are not significantly different at the image source domain, that is, the feature matching difference between the two in the high-dimensional feature space, so as to detect large pieces of raw coal on the dumping machine.

[0051] Then, the positional difference between the detected feature map and the reference feature map is calculated to perform feature matching on the detected feature map and the reference feature map to obtain the differential feature map. That is, by calculating the positional difference between the detected feature map and the reference feature map, the feature matching difference degree between the two can be obtained, and the correlation feature information between this difference degree and the difference in raw coal volume can be established. Next, the differential feature map is decoded and regressed through a decoder to obtain a decoded value representing the feature matching difference degree, and then based on the decoded value, it is determined whether there are large pieces in the detected raw coal image. Specifically, in response to the decoded value being greater than or equal to a predetermined threshold, it is determined that there are large pieces in the detected raw coal image. That is to say, in actual large piece detection, based on the comparison between the extracted feature matching difference degree and the predetermined threshold, it can be determined whether there are large pieces of raw coal on the plate feeder in the detected raw coal image.

[0052] Specifically, in the technical solution of this application, when performing feature matching on the detection feature map and the reference feature map to obtain the differential feature map, position-by-position difference calculation is performed on the detection feature map and the reference feature map to obtain the differential feature map. Therefore, since the feature value at each position of the differential feature map is only the value obtained by directly subtracting the feature values ​​at the corresponding positions of the detection feature map and the reference feature map, without considering the direction of difference in feature values ​​between the detection feature map and the reference feature map, a negative correlation will exist between corresponding positions in the differential feature map. This affects the convergence speed of the decoding operation of the differential feature map by the decoder and the accuracy of the decoded value.

[0053] Therefore, the feature matrices of the difference feature map can be corrected by using a fully orthographic projection nonlinear reweighting method, specifically expressed as follows:

[0054]

[0055] M and M c These are the feature matrices of the difference feature map before and after the correction. Here, the feature matrices of the difference feature map are transformed into diagonal matrices through a linear transformation, and the division between the numerator and denominator matrices is a positional division of the matrix eigenvalues.

[0056] Here, the orthographic projection nonlinear reweighting uses the ReLU function to ensure the projection is entirely positive to avoid aggregating negatively correlated information. Simultaneously, a nonlinear reweighting mechanism is introduced to aggregate the eigenvalue distribution of the difference feature map, so that the intrinsic structure of each feature matrix of the difference feature map can penalize long-distance connections and strengthen local coupling. This achieves a synergistic effect between the spatial feature transformations corresponding to the orthographic projection reweighting in the high-dimensional feature space of each feature matrix of the difference feature map, thus suppressing the negative correlation between corresponding positions in the difference feature map. This improves the convergence speed of the decoding operation of the difference feature map by the decoder and the accuracy of the decoded values. Consequently, large pieces of raw coal on the plate feeder can be accurately detected, triggering an alarm when large pieces of raw coal are detected, alerting operators to intervene and preventing blockage and shutdown of the crushing system.

[0057] Based on this, this application provides a large coal block recognition system based on feature matching, comprising: an image acquisition module for acquiring a detection raw coal image and a reference raw coal image, wherein the reference raw coal image contains large coal blocks; a dust removal module for passing the detection raw coal image through a dust removal generator based on a generative adversarial network to obtain a generated detection raw coal image; a twin detection module for passing the generated detection raw coal image and the reference raw coal image through a twin network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map; a difference module for performing feature matching on the detection feature map and the reference feature map to obtain a difference feature map; a feature distribution correction module for performing feature distribution correction on the difference feature map to obtain a corrected difference feature map; a decoding regression module for passing the corrected difference feature map through a decoder to obtain a decoding value representing the feature matching difference degree; and a recognition result generation module for determining whether large coal blocks exist in the detection raw coal image based on the decoding value.

[0058] Figure 1 This is an application scenario diagram of a feature-matching-based large-block recognition system according to an embodiment of this application. For example... Figure 1 As shown, in this application scenario, acquiring images of raw coal for detection (e.g., as...) Figure 1 The image shown is D1) and the reference raw coal image (e.g., as shown). Figure 1 As shown in D2), the detected raw coal image and the reference raw coal image are then input into a server deployed with a large-block recognition algorithm based on feature matching (e.g., Figure 1 As shown in S), the server is able to use the feature-matching-based large-block recognition algorithm to process the detected raw coal image and the reference raw coal image to generate a decoding value representing the feature-matching difference, and based on the decoding value, determine whether there are large blocks in the detected raw coal image.

[0059] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0060] Exemplary System

[0061] Figure 2 This is a block diagram of a feature-matching-based large-block recognition system according to an embodiment of this application. Figure 2As shown, a large coal block recognition system 100 based on feature matching according to an embodiment of this application includes: an image acquisition module 110 for acquiring a detection raw coal image and a reference raw coal image, wherein the reference raw coal image contains large coal blocks; a dust removal module 120 for passing the detection raw coal image through a dust removal generator based on a generative adversarial network to obtain a generated detection raw coal image; a twin detection module 130 for passing the generated detection raw coal image and the reference raw coal image through a twin network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map; a difference module 140 for performing feature matching on the detection feature map and the reference feature map to obtain a difference feature map; a feature distribution correction module 150 for performing feature distribution correction on the difference feature map to obtain a corrected difference feature map; a decoding regression module 160 for passing the corrected difference feature map through a decoder to obtain a decoding value representing the feature matching difference degree; and a recognition result generation module 170 for determining whether large coal blocks exist in the detection raw coal image based on the decoding value.

[0062] More specifically, in this embodiment, the image acquisition module 110 is used to acquire a detection image of raw coal and a reference image of raw coal, wherein the reference image of raw coal contains large pieces. Considering that the most effective and convenient method for detecting large pieces of raw coal is to rely on the recognition of images of raw coal on a plate feeder, it is difficult to detect large pieces based solely on the information in the raw coal image itself, as different shooting methods result in different proportions in the raw coal image. Therefore, machine vision artificial intelligence detection technology is used to preprocess the raw coal image on the plate feeder to extract the feature matching difference information between the detection image of raw coal and the reference image of large pieces of raw coal in a high-dimensional feature space. Then, based on the correlation between the feature matching difference degree and the volume difference of the coal pieces, feature matching is performed under actual conditions to automatically identify large pieces of raw coal on the plate feeder. This allows for accurate detection of large pieces of raw coal on the plate feeder, triggering an alarm when large pieces of raw coal are detected to remind operators to intervene and prevent the crushing system from becoming blocked and shut down.

[0063] More specifically, in this embodiment, the dust removal module 120 is used to generate a new raw coal image by passing the raw coal detection image through a dust removal generator based on a generative adversarial network. Considering that a large amount of dust and other small environmental particles exist in the raw coal mining site, affecting the clarity of the raw coal detection image, the implicit features of the raw coal detection image become blurred due to interference from external environmental factors during feature extraction, thereby reducing the accuracy of detecting large pieces on the plate feeder. Therefore, in the technical solution of this application, dust removal processing is performed first using a dust removal generator based on a generative adversarial network before feature extraction.

[0064] Specifically, the detected raw coal image is input into the dust-removing generator based on the Generative Adversarial Network (GAN) so that the generator of the GAN generates the detected raw coal image through deconvolutional encoding. In particular, the GAN here includes a discriminator and a generator, wherein the generator generates the dust-removed image, and the discriminator calculates the difference between the dust-removed image and the real image, and updates the network parameters of the generator using a gradient descent directional propagation algorithm to obtain a generator with dust removal functionality.

[0065] Accordingly, in a specific example, the adversarial generative network includes a discriminator and a generator, wherein the dust removal module 120 is further configured to input the detected raw coal image into the dust removal generator based on the adversarial generative network so that the generator of the adversarial generative network generates the generated detected raw coal image by deconvolution encoding.

[0066] More specifically, in this embodiment, the twin detection module 130 is used to pass the generated raw coal image and the reference raw coal image through a twin network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. A convolutional neural network model, which excels in extracting local latent features of images, is used for feature mining of the generated raw coal image and the reference raw coal image. In particular, considering that when detecting large pieces of raw coal to avoid clogging the crushing system, it is necessary to focus on the feature information such as the size of the raw coal material on the plate feeder and filter out interfering feature information unrelated to the detection of large pieces of raw coal material on the plate feeder, and given that the attention mechanism can select the focus position, the technical solution of this application further uses a spatial attention mechanism to extract features from the generated raw coal image and the reference raw coal image. That is, specifically, the generated raw coal image and the reference raw coal image are passed through a twin network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. It is worth mentioning that the first image encoder and the second image encoder have the same network structure, and both are convolutional neural network models using a spatial attention mechanism. Specifically, using the Siamese network model with the same network structure to extract spatial location features from the generated raw coal image and the reference raw coal image respectively can uncover the implicit feature information of the raw coal material focused on the spatial location of the plate feeder in both images. This feature information is not significantly different at the image source domain, i.e., the feature matching difference between the two in the high-dimensional feature space, thereby enabling the detection of large pieces of raw coal on the dumping machine.

[0067] Accordingly, in a specific example, the first image encoder and the second image encoder have the same network structure, wherein the first image encoder and the second image encoder are convolutional neural network models using a spatial attention mechanism.

[0068] More specifically, in this embodiment, the difference module 140 is used to perform feature matching on the detected feature map and the reference feature map to obtain a difference feature map. That is, by calculating the positional difference between the detected feature map and the reference feature map, the feature matching difference degree between the two can be obtained, and the correlation feature information between this difference degree and the raw coal volume difference can be established.

[0069] Accordingly, in a specific example, the difference module 140 is further configured to calculate the positional difference between the detected feature map and the reference feature map to perform feature matching on the detected feature map and the reference feature map to obtain the difference feature map.

[0070] More specifically, in this embodiment of the application, the feature distribution correction module 150 is used to perform feature distribution correction on the differential feature map to obtain a corrected differential feature map.

[0071] Specifically, in the technical solution of this application, when performing feature matching on the detected feature map and the reference feature map to obtain the differential feature map, position-by-position difference calculation is performed on the detected feature map and the reference feature map to obtain the differential feature map. Therefore, since the feature value at each position of the differential feature map is only the value obtained by directly subtracting the feature values ​​at the corresponding positions of the detected feature map and the reference feature map, without considering the direction of difference in feature values ​​between the detected feature map and the reference feature map, a negative correlation will exist between corresponding positions in the differential feature map. This affects the convergence speed of the differential feature map during decoding by the decoder and the accuracy of the decoded value. Therefore, a fully orthographic projection nonlinear reweighting method can be used to correct each feature matrix of the differential feature map.

[0072] Accordingly, in a specific example, the feature distribution correction module 150 is further configured to: perform feature distribution correction on the difference feature map using the following formula to obtain the corrected difference feature map;

[0073] The formula is as follows:

[0074]

[0075] Among them M and M c These are the feature matrices along the channel dimension of the difference feature map and the corrected difference feature map, respectively, and ReLU(·) represents the ReLU activation function. This indicates matrix multiplication, where the division between the numerator and denominator matrices is a positional division of the matrix eigenvalues. exp(·) represents matrix exponentiation, which means calculating the natural exponential function value raised to the power of the eigenvalues ​​at each position in the matrix.

[0076] Here, the orthographic projection nonlinear reweighting uses the ReLU function to ensure the projection is entirely positive to avoid aggregating negatively correlated information. Simultaneously, a nonlinear reweighting mechanism is introduced to aggregate the eigenvalue distribution of the difference feature map, so that the intrinsic structure of each feature matrix of the difference feature map can penalize long-distance connections and strengthen local coupling. This achieves a synergistic effect between the spatial feature transformations corresponding to the orthographic projection reweighting in the high-dimensional feature space of each feature matrix of the difference feature map, thus suppressing the negative correlation between corresponding positions in the difference feature map. This improves the convergence speed of the decoding operation of the difference feature map by the decoder and the accuracy of the decoded values. Consequently, large pieces of raw coal on the plate feeder can be accurately detected, triggering an alarm when large pieces of raw coal are detected, alerting operators to intervene and preventing blockage and shutdown of the crushing system.

[0077] More specifically, in the embodiments of this application, the decoding regression module 160 is used to pass the corrected differential feature map through a decoder to obtain a decoded value representing the feature matching difference.

[0078] Accordingly, in a specific example, the decoding regression module 160 is further configured to: perform decoding regression on the corrected differential feature map using multiple fully connected layers of the decoder to obtain the decoded value using the following formula; wherein, the formula is: Where X is the corrected differential feature map, Y is the decoded value, and W is the weight matrix. This represents matrix multiplication.

[0079] More specifically, in this embodiment of the application, the recognition result generation module 170 is used to determine whether there are large pieces in the detected raw coal image based on the decoded value.

[0080] Accordingly, in a specific example, the recognition result generation module 170 is further configured to determine that a large piece exists in the detected raw coal image in response to the decoded value being greater than or equal to a predetermined threshold.

[0081] In other words, in actual large coal detection, based on the comparison between the extracted feature matching difference degree and the predetermined threshold, it can be determined whether there are large coal pieces on the plate feeder in the detected raw coal image.

[0082] In summary, the feature-matching-based large coal identification system 100 based on embodiments of this application is explained. First, the acquired raw coal image is processed by a dust removal generator based on a generative adversarial network to generate a new raw coal image. Next, the generated raw coal image and a reference raw coal image are processed by a Siamese network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map. Then, feature matching and feature distribution correction are performed on the detection feature map and the reference feature map to obtain a corrected differential feature map. Finally, the corrected differential feature map is processed by a decoder to obtain a decoded value representing the feature matching difference. Based on the decoded value, it is determined whether large coal pieces exist in the detected raw coal image. In this way, large coal pieces on a plate feeder can be accurately detected, and an alarm can be issued when large coal pieces are detected.

[0083] As described above, the feature-matching-based large block recognition system 100 according to the embodiments of this application can be implemented in various terminal devices, such as servers with feature-matching-based large block recognition algorithms. In one example, the feature-matching-based large block recognition system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the feature-matching-based large block recognition system 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the feature-matching-based large block recognition system 100 can also be one of many hardware modules of the terminal device.

[0084] Alternatively, in another example, the feature-matching-based block recognition system 100 and the terminal device can also be separate devices, and the feature-matching-based block recognition system 100 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0085] Exemplary methods

[0086] Figure 3 This is a flowchart of a large-block recognition method based on feature matching according to an embodiment of this application. Figure 3As shown, the large coal block recognition method based on feature matching according to an embodiment of this application includes: S110, acquiring a detection raw coal image and a reference raw coal image, wherein the reference raw coal image contains large coal blocks; S120, passing the detection raw coal image through a dust removal generator based on a generative adversarial network to obtain a generated detection raw coal image; S130, passing the generated detection raw coal image and the reference raw coal image through a Siamese network model including a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map; S140, performing feature matching on the detection feature map and the reference feature map to obtain a differential feature map; S150, performing feature distribution correction on the differential feature map to obtain a corrected differential feature map; S160, passing the corrected differential feature map through a decoder to obtain a decoded value representing the feature matching difference degree; and S170, determining whether large coal blocks exist in the detection raw coal image based on the decoded value.

[0087] Figure 4 This is a schematic diagram of the system architecture of a feature-matching-based large-block recognition method according to an embodiment of this application. Figure 4 As shown, in the system architecture of the large coal block recognition method based on feature matching, firstly, a detection raw coal image and a reference raw coal image are acquired, wherein large coal blocks exist in the reference raw coal image; next, the detection raw coal image is passed through a dust removal generator based on a generative adversarial network to obtain a generated detection raw coal image; then, the generated detection raw coal image and the reference raw coal image are passed through a Siamese network model containing a first image encoder and a second image encoder to obtain a detection feature map and a reference feature map; next, feature matching is performed on the detection feature map and the reference feature map to obtain a differential feature map; then, feature distribution correction is performed on the differential feature map to obtain a corrected differential feature map; next, the corrected differential feature map is passed through a decoder to obtain a decoded value representing the feature matching difference degree; finally, based on the decoded value, it is determined whether large coal blocks exist in the detection raw coal image.

[0088] Considering that the most effective and convenient method for detecting large pieces of raw coal is to rely on the recognition of raw coal images on the plate feeder, it is difficult to detect large pieces of raw coal based solely on the information in the raw coal images due to the different shooting methods used. Therefore, machine vision artificial intelligence detection technology is used to preprocess the raw coal images on the plate feeder to extract the feature matching difference information between the raw coal detection image and the reference image of large raw coal pieces in a high-dimensional feature space. Then, based on the correlation between the feature matching difference degree and the coal block volume difference, feature matching is performed in actual situations to automatically identify large raw coal pieces on the plate feeder.

[0089] In a specific example, in the above-described feature-matching-based large-block recognition method, the Generative Adversarial Network (GAN) includes a discriminator and a generator. The step of passing the detected raw coal image through a GAN-based dust-removing generator to obtain a generated detected raw coal image further includes: inputting the detected raw coal image into the GAN-based dust-removing generator so that the generator can generate the generated detected raw coal image through deconvolutional encoding. The generator is used to generate the dust-removed image, and the discriminator is used to calculate the difference between the dust-removed image and the real image, and to update the network parameters of the generator using a gradient descent directional propagation algorithm to obtain a generator with dust-removing functionality.

[0090] In a specific example, in the above-mentioned large-piece recognition method based on feature matching, the first image encoder and the second image encoder have the same network structure, wherein the first image encoder and the second image encoder are convolutional neural network models using a spatial attention mechanism. The attention mechanism can select the focus position; therefore, in the technical solution of this application, a spatial attention mechanism is further used to extract features from the generated detection raw coal image and the reference raw coal image. Specifically, using the Siamese network model containing the same network structure of the image encoder to extract spatially focused features from the generated detection raw coal image and the reference raw coal image respectively can uncover the implicit feature information of the raw coal material focused on the spatial position of the plate feeder in both images, where the differences are not significant at the image source domain, i.e., the feature matching difference between the two in the high-dimensional feature space, thereby enabling the detection of large pieces of raw coal on the dumping machine.

[0091] In a specific example, in the above-described feature-matching-based large-block recognition method, the step of performing feature matching on the detected feature map and the reference feature map to obtain a differential feature map further includes: calculating the positional difference between the detected feature map and the reference feature map to perform feature matching on the detected feature map and the reference feature map to obtain the differential feature map.

[0092] Specifically, in the technical solution of this application, when performing feature matching on the detected feature map and the reference feature map to obtain the differential feature map, position-by-position difference calculation is performed on the detected feature map and the reference feature map to obtain the differential feature map. Therefore, since the feature value at each position of the differential feature map is only the value obtained by directly subtracting the feature values ​​at the corresponding positions of the detected feature map and the reference feature map, without considering the direction of difference in feature values ​​between the detected feature map and the reference feature map, a negative correlation will exist between corresponding positions in the differential feature map. This affects the convergence speed of the differential feature map during decoding by the decoder and the accuracy of the decoded value. Therefore, a fully orthographic projection nonlinear reweighting method can be used to correct each feature matrix of the differential feature map.

[0093] In a specific example, in the above-mentioned feature-matching-based large block recognition method, the step of performing feature distribution correction on the differential feature map to obtain the corrected differential feature map further includes: performing feature distribution correction on the differential feature map using the following formula to obtain the corrected differential feature map;

[0094] The formula is as follows:

[0095]

[0096] Among them M and M c These are the feature matrices along the channel dimension of the difference feature map and the corrected difference feature map, respectively, and ReLU(·) represents the ReLU activation function. The expression represents matrix multiplication, where division between the numerator and denominator matrices is a positional division of the matrix eigenvalues. `exp(·)` represents matrix exponentiation, which calculates the natural exponential function value raised to the power of the eigenvalues ​​at each position in the matrix. By calculating the positional difference between the detected feature map and the reference feature map, the feature matching difference degree between the two can be obtained, and the correlation between this difference degree and the raw coal volume difference can be established.

[0097] Here, the orthographic projection nonlinear reweighting uses the ReLU function to ensure the projection is entirely positive to avoid aggregating negatively correlated information. Simultaneously, a nonlinear reweighting mechanism is introduced to aggregate the eigenvalue distribution of the difference feature map, so that the intrinsic structure of each feature matrix of the difference feature map can penalize long-distance connections and strengthen local coupling. This achieves a synergistic effect between the spatial feature transformations corresponding to the orthographic projection reweighting in the high-dimensional feature space of each feature matrix of the difference feature map, thus suppressing the negative correlation between corresponding positions in the difference feature map. This improves the convergence speed of the decoding operation of the difference feature map by the decoder and the accuracy of the decoded values. Consequently, large pieces of raw coal on the plate feeder can be accurately detected, triggering an alarm when large pieces of raw coal are detected, alerting operators to intervene and preventing blockage and shutdown of the crushing system.

[0098] In a specific example, in the above-described feature-matching-based large-block recognition method, the step of passing the corrected differential feature map through a decoder to obtain a decoded value representing the feature matching difference further includes: using multiple fully connected layers of the decoder to perform decoding regression on the corrected differential feature map using the following formula to obtain the decoded value; wherein, the formula is: Where X is the corrected differential feature map, Y is the decoded value, and W is the weight matrix. This represents matrix multiplication.

[0099] In a specific example, in the above-described large-block recognition method based on feature matching, determining whether a large block exists in the detected raw coal image based on the decoded value further includes: determining that a large block exists in the detected raw coal image in response to the decoded value being greater than or equal to a predetermined threshold.

[0100] Here, those skilled in the art will understand that the specific operations of each step in the above-described feature-matching-based large-block recognition method have been referenced above. Figures 1 to 2 The description of the feature-matching-based large-block recognition system is detailed here, and therefore, its repeated description will be omitted.

[0101] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0102] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0103] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0104] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0105] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A feature matching based blob recognition system, characterized by, The method comprises: an image acquisition module configured to acquire a detection raw coal image and a reference raw coal image, wherein the reference raw coal image contains a large piece; a dust removal module configured to generate a generated detection raw coal image by inputting the detection raw coal image into a dust removal generator based on a generative adversarial network; a twin detection module configured to obtain a detection feature map and a reference feature map by inputting the generated detection raw coal image and the reference raw coal image into a twin network model comprising a first image encoder and a second image encoder; a difference module configured to perform feature matching on the detection feature map and the reference feature map to obtain a difference feature map; a feature distribution correction module configured to perform feature distribution correction on the difference feature map according to the following formula to obtain a corrected difference feature map; wherein the formula is: wherein M and M c respectively represent each feature matrix of the difference feature map and the corrected difference feature map along the channel dimension, ReLU(·) represents a ReLU activation function, represents matrix multiplication, and the division between the numerator matrix and the denominator matrix is the positional division of the eigenvalues of the matrix; exp(·) represents the exponential operation of the matrix, and the exponential operation of the matrix represents the calculation of the natural exponential function value with the eigenvalue of each position in the matrix as the power. a decoding regression module configured to obtain a decoding value representing a feature matching difference degree by inputting the corrected difference feature map into a decoder; and an identification result generation module configured to determine whether the detection raw coal image contains a large piece based on the decoding value.

2. The feature matching based blob recognition system of claim 1, wherein, The generative adversarial network comprises a discriminator and a generator, wherein the dust removal module is further configured to input the detection raw coal image into the dust removal generator based on the generative adversarial network so that the generated detection raw coal image is generated by the generator of the generative adversarial network through transposed convolution coding.

3. The feature matching based blob recognition system of claim 2, wherein, The first image encoder and the second image encoder have the same network structure, wherein the first image encoder and the second image encoder are convolutional neural network models using a spatial attention mechanism.

4. The feature matching based blob recognition system of claim 3, wherein, The difference module is further configured to calculate a positional difference between the detection feature map and the reference feature map to perform feature matching on the detection feature map and the reference feature map to obtain the difference feature map.

5. The feature matching based blob recognition system of claim 4, wherein, The decoding regression module is further configured to use multiple fully connected layers of the decoder to perform decoding regression on the corrected difference feature map according to the following formula to obtain the decoding value; wherein the formula is: where X is the corrected difference feature map, Y is the decoded value, and W is a weight matrix, denotes a matrix multiplication.

6. The feature matching based blob recognition system of claim 5, wherein, The identification result generation module is further configured to determine that the detection raw coal image contains a large piece in response to the decoding value being greater than or equal to a predetermined threshold.

7. A method of bulk recognition based on feature matching, characterized in that, The method comprises: acquiring a detection raw coal image and a reference raw coal image, wherein the reference raw coal image contains a large piece; generating a generated detection raw coal image by inputting the detection raw coal image into a dust removal generator based on a generative adversarial network; obtaining a detection feature map and a reference feature map by inputting the generated detection raw coal image and the reference raw coal image into a twin network model comprising a first image encoder and a second image encoder; performing feature matching on the detection feature map and the reference feature map to obtain a difference feature map; performing feature distribution correction on the difference feature map to obtain a corrected difference feature map, comprising performing feature distribution correction on the difference feature map according to the following formula to obtain a corrected difference feature map; wherein the formula is: wherein M and M c respectively represent each feature matrix of the difference feature map and the corrected difference feature map along the channel dimension, ReLU(·) represents a ReLU activation function, represents matrix multiplication, and the division between the numerator matrix and the denominator matrix is the positional division of the eigenvalues of the matrix; exp(·) represents the exponential operation of the matrix, and the exponential operation of the matrix represents the calculation of the natural exponential function value with the eigenvalue of each position in the matrix as the power. obtaining a decoding value representing a feature matching difference degree by inputting the corrected difference feature map into a decoder; and determining whether the detection raw coal image contains a large piece based on the decoding value.

8. The feature matching based blob recognition method of claim 7, wherein, The adversarial generation network comprises a discriminator and a generator, wherein the passing of the detected raw coal image through the dust removal generation device based on the adversarial generation network to obtain a generated detected raw coal image further comprises: inputting the detected raw coal image into the dust removal generation device based on the adversarial generation network to generate the generated detected raw coal image by the generator of the adversarial generation network through transposed convolution coding.

9. The feature matching based blob recognition method of claim 8, wherein, The first image encoder and the second image encoder have the same network structure, and the first image encoder and the second image encoder are convolutional neural network models using a spatial attention mechanism.

Images