A foreign object cleaning method and system for a coal conveying belt based on visual features

Abstract

The present invention provides a method and system for cleaning foreign objects on a coal conveying belt with visual features, which relates to the field of computer vision technology. The method includes: obtaining an original image of the coal conveying belt by installing a camera; analyzing and processing the original image to determine the Gram angle sum field image and the Gram angle difference field image; determining a comprehensive descriptor of foreign objects on the surface of the coal conveying belt according to the original image, the Gram angle sum field image and the Gram angle difference field image; inputting the comprehensive descriptor into a classifier to obtain a foreign object recognition result; determining whether to start a cleaning device for cleaning according to the foreign object recognition result; if the cleaning device is started for cleaning, after the cleaning device completes the cleaning, it feeds back the cleaning effect until there are no foreign objects on the belt surface. According to the present invention, the relationship between image pixels and adaptive weights can be considered through the Gram angle field and the Gram difference field, so as to improve the accuracy of the description of foreign objects on the coal conveying belt and optimize the recognition result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a method and system for cleaning foreign objects on a coal conveying belt using visual features. Background Art

[0002] In the related art, CN115180367A discloses a foreign object recognition device for a belt conveyor, which relates to the field of foreign object recognition technology for conveyor belts. It includes a detection component that detects foreign objects on the conveyor belt through a detection camera. A movable housing is provided on the detection component, and a picking component for processing foreign objects on the belt is provided inside the movable housing. Two sets of sliding plates are provided inside the picking component, and the foreign objects on the belt are picked up by the movement of the sliding plates. An extinguishing component for cooling and extinguishing coal is also provided inside the movable housing. The extinguishing component can detect the temperature of the coal on the belt, premix the extinguishing gas in advance when detecting high-temperature coal, and spray it on the coal on the belt. A driving component is provided inside the movable housing, and the driving component changes the picking speed of the picking component and the mixing speed of the extinguishing component. This solution can automatically detect and pick up foreign objects on the belt and can prevent fires caused by excessive temperature of the coal on the belt.

[0003] CN117184812A discloses a method for detecting tears and foreign objects on a conveyor belt. The method includes: preprocessing the image collected by an image acquisition device; identifying the preprocessed conveyor belt image through a pre-established conveyor belt tear and foreign object recognition detection model to complete conveyor belt tear detection and foreign object recognition; the conveyor belt tear and foreign object recognition detection model uses an image enhancement method based on Retinex and realizes multi-scale object detection according to the detection idea of an image feature pyramid. This solution can minimize the length of conveyor belt tears, greatly promote the optimization and improvement of the converter process, and has great economic benefits and production significance.

[0004] Therefore, in the related art, although image processing technology is used to identify foreign objects, the related art often only performs feature analysis on the original image. In a complex and changeable mine environment, affected by factors such as light and dust, the recognition accuracy is not high, and the difference in feature performance is not considered. That is, there are defects in the inaccuracy of image description and the static nature of feature weights.

[0005] The information disclosed in the background art part of this application is only intended to deepen the understanding of the general background art of this application, and should not be regarded as an admission or any form of implication that this information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The present invention provides a method and system for cleaning foreign objects on a coal conveying belt with visual features, which can solve the technical problem of the inaccuracy of image description and the static nature of feature weights existing in the related art.

[0007] According to the first aspect of the present invention, there is provided a method for cleaning foreign objects on a coal conveying belt with visual features, including:

[0008] Obtain the original image of the coal conveying belt by installing a camera;

[0009] Analyze and process the original image to determine the Gram angle sum field image and the Gram angle difference field image;

[0010] Determine the comprehensive descriptor of foreign objects on the surface of the coal conveying belt according to the original image, the Gram angle sum field image and the Gram angle difference field image;

[0011] Input the comprehensive descriptor into a classifier to obtain the foreign object recognition result;

[0012] Determine whether to start the cleaning device for cleaning according to the foreign object recognition result;

[0013] If the cleaning device is started for cleaning, after the cleaning device completes the cleaning, feedback the cleaning effect until there are no foreign objects on the belt surface.

[0014] Further, analyzing and processing the original image to determine the Gram angle sum field image and the Gram angle difference field image includes:

[0015] Expand the original image to obtain an expanded image;

[0016] Evenly divide each channel of the expanded image to obtain 9 divided images;

[0017] Determine the statistical descriptor according to the divided images;

[0018] Determine the Gram angle sum field image and the Gram angle difference field image according to the statistical descriptor.

[0019] Further, determining the statistical descriptor according to the divided images includes:

[0020] Calculate the maximum value, minimum value, mean value, mode, median, range, variance and standard deviation for each divided image;

[0021] According to the formula Determine the description vector of the v-th divided image where, is the maximum value of the v-th divided image, is the minimum value of the v-th divided image, is the mean value of the v-th divided image, is the mode of the v-th segmented image, is the median of the v-th segmented image, is the range of the v-th segmented image, is the variance of the v-th segmented image, is the standard deviation of the v-th segmented image, where v ≤ 9 and v is a positive integer;

[0022] The description vectors of the segmented images are combined in a hierarchical combination manner. The first-level combination forms 4 different first-level description vectors according to 4 different combination methods, and the second-level combination constitutes a statistical descriptor according to the results of the first-level combination;

[0023] According to the formula determine the statistical descriptor , where, and are 4 different first-level description vectors, and are the description vectors of the v-th segmented image respectively.

[0024] Furthermore, according to the statistical descriptor, determine the Gram angle and field image and the Gram angle difference field image, including:

[0025] According to the formula obtain the i-th component of the normalized vector , where n is the dimension of the statistical descriptor, is the i-th component of the statistical descriptor , and both i and n are positive integers;

[0026] According to the formula , , determine the Gram angle and field image and the Gram angle difference field image , where, is the cosine of the angle of , is the radius corresponding to , is the timestamp corresponding to , M is the normalization factor, are respectively 's cosine of the angle, i ≤ n, j ≤ n, and j is a positive integer.

[0027] Furthermore, according to the original image, the Gram angle and field image, and the Gram angle difference field image, determine the comprehensive descriptor of the foreign objects on the surface of the coal conveyor belt, including:

[0028] Obtain the original image features, Gram angle sum field features, and Gram angle difference field features based on the original image, the Gram angle sum and field image, and the Gram angle difference field image;

[0029] Determine the original image feature weight, Gram angle sum field feature weight, and Gram angle difference field feature weight according to the original image features, the Gram angle sum field features, and the Gram angle difference field features;

[0030] Determine the comprehensive descriptor of the foreign object on the surface of the coal conveying belt according to the original image features, the Gram angle sum field features, the Gram angle difference field features, the original image feature weight, the Gram sum field feature weight, and the Gram angle difference field feature weight.

[0031] Further, determining the original image feature weight, Gram angle sum field feature weight, and Gram angle difference field feature weight according to the original image features, the Gram angle sum field features, and the Gram angle difference field features includes: , , , , , , , , , ,

[0032] Determine the Gram angle sum field feature weight 、Original image feature weight And Gram difference field feature weight , where Is the decision matrix, Is the Gram angle sum field feature, Original image feature, Is the Gram angle difference field feature, Is the value of the x-th row and y-th column of the standardized decision matrix, Is the value of the x-th row and y-th column of the decision matrix, Is the value of the y-th column of the decision matrix, 、 、…、 Are the components of the vector E respectively, Is the information entropy of the y-th column of the standardized decision matrix, E is the entropy vector, Is the normalized information entropy weight of the y-th column of the standardized decision matrix, Is the weighted decision matrix, Is the value of the 1st, 2nd, …, m-th columns of the standardized decision matrix, Is the normalized information entropy weight of the 1st, 2nd, …, m-th columns of the standardized decision matrix, The value of the positive ideal solution for the y-th column of the weighted decision matrix, is the value of the negative ideal solution for the y-th column of the weighted decision matrix, 、 and are the values of the 1st, 2nd, and 3rd rows of the weighted decision matrix, is the value of the x-th row and y-th column of the weighted decision matrix, is the distance from the x-th row of the weighted decision matrix to the positive ideal solution, is the distance from the x-th row of the weighted decision matrix to the negative ideal solution, is the characteristic weight of the x-th row of the decision matrix, m is the number of columns of the decision matrix, x ≤ 3, y ≤ m, and x, y, and m are all positive integers, max is the maximum value function, and min is the minimum value function.

[0033] Furthermore, according to the original image features, the Gram angle and field features, the Gram angle difference field features, the original image feature weights, the Gram sum field feature weights, and the Gram angle difference field feature weights, a comprehensive descriptor of the foreign object on the coal conveyor belt surface is determined, including:

[0034] According to the formula the comprehensive descriptor F of the foreign object on the coal conveyor belt surface is determined.

[0035] According to the second aspect of the present invention, a foreign object cleaning system for a coal conveyor belt with visual features is provided, including:

[0036] An original image module for obtaining the original image of the coal conveyor belt by installing a camera;

[0037] A Gram angle and field image and Gram angle difference field image module for analyzing and processing the original image to determine the Gram angle and field image and the Gram angle difference field image;

[0038] A comprehensive descriptor module for determining a comprehensive descriptor of the foreign object on the coal conveyor belt surface according to the original image, the Gram angle and field image, and the Gram angle difference field image;

[0039] A foreign object recognition result module for inputting the comprehensive descriptor into a classifier to obtain a foreign object recognition result;

[0040] A judgment module for determining whether to start a cleaning device for cleaning according to the foreign object recognition result;

[0041] A feedback module for, if the cleaning device is started for cleaning, feeding back the cleaning effect after the cleaning device completes the cleaning until there is no foreign object on the belt surface.

[0042] Technical effects: According to the present invention, by considering the relationship between image pixels through the Gram angle field and the Gram difference field, the accuracy of describing foreign objects on the coal conveying belt is improved. The adaptive weight strategy enables the single feature weight to be dynamically updated with the recognized image, further optimizing the robustness and accuracy of the recognition result. When determining the Gram angle sum image and the Gram angle difference image, statistical features of the original image can be extracted through multiple dimensions, effectively capturing various attributes of foreign objects on the coal conveying belt, providing rich information for subsequent recognition. By mapping the statistical features to a two-dimensional image to generate the Gram angle sum and the Gram angle difference field, an efficient representation of the features of the original image is achieved, thus significantly improving the accuracy of foreign object recognition. When determining the comprehensive descriptor of foreign objects on the surface of the coal conveying belt, the features of the original image, the Gram angle sum image, and the Gram angle difference image can be fused, and the features can be dynamically weighted by combining the adaptive weighting method, enabling the comprehensive descriptor to better reflect the essential features of foreign objects, thereby enhancing the representativeness of the comprehensive descriptor, effectively coping with the changes of foreign objects in different scenarios, and improving the stability and reliability of the recognition effect.

[0043] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present invention. According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present invention will become clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other embodiments can be obtained based on these drawings without creative efforts.

[0045] Figure 1 Exemplarily shows a flowchart of a method for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention;

[0046] Figure 2 Exemplarily shows an analysis process flowchart of a system for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention;

[0047] Figure 3 Exemplarily shows a feature engineering flowchart of a system for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention;

[0048] Figure 4 Exemplarily shows a block diagram of a system for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] The following uses specific embodiments to elaborate in detail on the technical solutions of the present invention. These several specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0051] Figure 1 The flowchart of the method for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention is exemplarily shown. The method includes:

[0052] Step S101: Obtain the original image of the coal conveying belt by installing a camera.

[0053] Step S102: Analyze and process the original image to determine the Gram angle sum field image and the Gram angle difference field image.

[0054] Step S103: Determine the comprehensive descriptor of foreign objects on the surface of the coal conveying belt based on the original image, the Gram angle sum field image, and the Gram angle difference field image.

[0055] Step S104: Input the comprehensive descriptor into a classifier to obtain the foreign object recognition result.

[0056] Step S105: Determine whether to start the cleaning device for cleaning based on the foreign object recognition result.

[0057] Step S106: If the cleaning device is started for cleaning, after the cleaning device completes the cleaning, feedback the cleaning effect until there are no foreign objects on the belt surface.

[0058] For the method for cleaning foreign objects on a coal conveying belt with visual features according to an embodiment of the present invention, by considering the image pixel relationship through the Gram angle field and the Gram difference field, the accuracy of the description of foreign objects on the coal conveying belt is improved. The adaptive weight strategy enables the single feature weight to be dynamically updated with the recognition image, further optimizing the robustness and accuracy of the recognition result.

[0059] According to an embodiment of the present invention, in step S101, install a camera device so that the camera can clearly capture the running condition of the coal conveying belt, and the original image of the coal conveying belt can be obtained in real time.

[0060] According to an embodiment of the present invention, in step S102, the original image is analyzed and processed to determine the Gram angle and field image and the Gram angle difference field image.

[0061] According to an embodiment of the present invention, step S102 includes: expanding the original image to obtain an expanded image; evenly dividing each channel of the expanded image to obtain nine divided images; determining a statistical descriptor according to the divided images; and determining the Gram angle and field image and the Gram angle difference field image according to the statistical descriptor.

[0062] According to an embodiment of the present invention, Figure 2 Exemplarily, an analysis process flowchart of a coal conveying belt foreign object cleaning system with visual features according to an embodiment of the present invention is shown. The original image is expanded. Denote the original image as Img, with a size of , being the width of the original image, being the length of the original image, being the number of channels of the original image, and the expansion amounts and in the width w direction and the length h direction. Pixel value 0 is added to the filled area to obtain an expanded image, and the size of the expanded image is . Each channel of the expanded image is evenly divided to obtain nine divided images, the statistical descriptor is calculated, and the GAF technology is used to obtain the Gram angle and field image and the Gram angle difference field image.

[0063] According to an embodiment of the present invention, determining the statistical descriptor according to the divided images includes: calculating the maximum value, minimum value, mean value, mode, median, range, variance, and standard deviation for each divided image; and determining the description vector of the v-th divided image according to formula (1) ,

[0064] (1)where is the maximum value of the v-th divided image, is the minimum value of the v-th divided image, is the mean value of the v-th divided image, is the mode of the v-th divided image, is the median of the v-th divided image, is the range of the v-th divided image, is the variance of the v-th divided image, is the standard deviation of the v-th segmented image, where v ≤ 9 and v is a positive integer; the description vectors of the segmented images are combined in a hierarchical combination manner. The first-level combination forms 4 different first-level description vectors according to 4 different combination methods, and the second-level combination constitutes a statistical descriptor according to the results of the first-level combination; the statistical descriptor is determined according to formulas (2), (3), (4), (5) and (6). ,

[0065] (2),

[0066] (3),

[0067] (4),

[0068] (5),

[0069] (6),

[0070] wherein, and are 4 different first-level description vectors, and are the description vectors of the v-th segmented image respectively.

[0071] According to an embodiment of the present invention, for each segmented image, calculate statistical quantities, construct description vectors, and form a final statistical descriptor by hierarchically combining the description vectors. The 4 different combination methods of the first-level combination are given in formulas (2), (3), (4) and (5), and are defined by rearranging the order of to . The statistical descriptor contains the information of all first-level description vectors.

[0072] According to an embodiment of the present invention, according to the statistical descriptor, determine the Gram angle and field image and the Gram angle difference field image, including: obtaining the component of the i-th dimension of the normalized vector ,

[0073] (7), wherein n is the dimension of the statistical descriptor, is the component of the i-th dimension of the statistical descriptor , and both i and n are positive integers; determine the Gram angle and field image and the Gram angle difference field image according to formulas (8), (9) and (10),

[0074] (8), (9),

[0075] (10), Among them, is the cosine of the angle of, is the corresponding radius, is the corresponding timestamp, M is the normalization factor, are respectively the cosine of the angle of, i ≤ n, j ≤ n, and j is a positive integer.

[0076] According to an embodiment of the present invention, the statistical descriptor is normalized and scaled to between [-1, 1], and the normalized vector the component of the i-th dimension of can be obtained. On the polar coordinate, the normalized vector is encoded, and the Gram angle and field image and the Gram angle difference field image can be generated based on different formulas.

[0077] In this way, the statistical features of the original image can be extracted in multiple dimensions, effectively capturing various attributes of the foreign objects on the coal conveying belt, providing rich information for subsequent recognition. By mapping the statistical features to a two-dimensional image and generating the Gram angle and field and the Gram angle difference field, the efficient representation of the features of the original image is realized, thus significantly improving the accuracy of foreign object recognition.

[0078] According to an embodiment of the present invention, in step S103, according to the original image, the Gram angle and field image, and the Gram angle difference field image, the comprehensive descriptor of the foreign object on the surface of the coal conveying belt is determined.

[0079] According to an embodiment of the present invention, step S103 includes: obtaining the original image features, the Gram angle difference field features, and the Gram angle and field features according to the original image, the Gram angle and field image, and the Gram angle difference field image; determining the original image feature weight, the Gram angle and field feature weight, and the Gram angle difference field feature weight according to the original image features, the Gram angle and field features, and the Gram angle difference field features; determining the comprehensive descriptor of the foreign object on the surface of the coal conveying belt according to the original image features, the Gram angle and field features, the Gram angle difference field features, the original image feature weight, the Gram and field feature weight, and the Gram angle difference field feature weight.

[0080] According to an embodiment of the present invention, Figure 3Exemplarily shown is the feature engineering flowchart of the foreign object cleaning system for the coal conveying belt with visual features according to an embodiment of the present invention. From the original image, the Gram angle and field image, and the Gram angle difference field image, the original image features, the Gram angle and field features, and the Gram angle difference field features are respectively extracted. These features represent the information of the image under different dimensions and transformations, which helps subsequent analysis and processing. Based on the above three extracted features, namely, the original image features, the Gram angle and field features, and the Gram angle difference field features, the respective feature weights are further determined. The feature weights reflect the importance and contribution degree of each feature when constructing the comprehensive descriptor. The comprehensive descriptor is a composite representation that integrates multiple feature information and can more comprehensively and accurately reflect the features of foreign objects on the surface of the coal conveying belt. Through the comprehensive descriptor, the detection and recognition of foreign objects can be carried out more effectively.

[0081] According to an embodiment of the present invention, according to the original image features, the Gram angle and field features, and the Gram angle difference field features, determining the original image feature weight, the Gram angle and field feature weight, and the Gram angle difference field feature weight includes: determining the Gram angle and field feature weight according to formulas (11), (12), (13), (14), (15), (16), (17), (18), (19), (20), and (21) , the original image feature weight and the Gram difference field feature weight ,

[0082] (11),

[0083] (12),

[0084] (13),

[0085] (14),

[0086] (15),

[0087] (16),

[0088] (17),

[0089] (18),

[0090] (19),

[0091] (20),

[0092] (21),

[0093] Among them, is the decision matrix, is the Gram angular sum field feature, Original image feature, is the Gram angular difference field feature, is the value of the x-th row and y-th column of the standardized decision matrix, is the value of the x-th row and y-th column of the decision matrix, is the value of the y-th column of the decision matrix, , , …, are the components of the vector E respectively, is the information entropy of the y-th column of the standardized decision matrix, and E is the entropy vector, is the normalized information entropy weight of the y-th column of the standardized decision matrix, is the weighted decision matrix, are the values of the 1st, 2nd, …, m-th columns of the standardized decision matrix, are the normalized information entropy weights of the 1st, 2nd, …, m-th columns of the standardized decision matrix, The value of the positive ideal solution of the y-th column of the weighted decision matrix, is the value of the negative ideal solution of the y-th column of the weighted decision matrix, , and are the values of the 1st, 2nd, and 3rd rows of the weighted decision matrix, is the value of the x-th row and y-th column of the weighted decision matrix, is the distance from the x-th row of the weighted decision matrix to the positive ideal solution, is the distance from the x-th row of the weighted decision matrix to the negative ideal solution, is the feature weight of the x-th row of the decision matrix, m is the number of columns of the decision matrix, x ≤ 3, y ≤ m, and x, y, and m are all positive integers, max is the maximum value function, and min is the minimum value function.

[0094] According to an embodiment of the present invention, a decision matrix is constructed through the original image feature, the Gram angular sum field feature, and the Gram angular difference field feature, the decision matrix is standardized to obtain a standardized decision matrix, the information entropy of each column of the standardized decision matrix is calculated based on the standardized decision matrix, an entropy vector is formed based on the information entropy, the information entropy weight of each column of the standardized decision matrix is calculated, and the information entropy weight is normalized to obtain a normalized information entropy weight. Each standardized element is multiplied by its corresponding normalized information entropy weight to obtain a weighted decision matrix. The value of the positive ideal solution and the value of the negative ideal solution of each column of the weighted decision matrix are calculated. Based on the value of the positive ideal solution and the value of the negative ideal solution, the distance from each row of the weighted decision matrix to the positive ideal solution and the distance from each row of the weighted decision matrix to the negative ideal solution are calculated. Finally, the weight of each row feature of the decision matrix is calculated and normalized.

[0095] According to an embodiment of the present invention, based on the original image features, the Gram angle and field features, the Gram angle difference field features, the original image feature weight, the Gram sum and field feature weight, and the Gram angle difference field feature weight, a comprehensive descriptor of foreign objects on the surface of the coal conveyor belt is determined, including: determining the comprehensive descriptor F of foreign objects on the surface of the coal conveyor belt according to formula (22). (22);

[0096] According to an embodiment of the present invention, each feature is weighted by its corresponding feature weight, and each weighted feature is normalized by its sum to obtain a comprehensive descriptor.

[0097] In this way, by fusing the features of the original image, the Gram angle and field image, and the Gram angle difference field image, and dynamically weighting the features by combining the adaptive weighting method, the comprehensive descriptor can better reflect the essential features of foreign objects, thereby enhancing the representativeness of the comprehensive descriptor, effectively coping with the changes of foreign objects in different scenarios, and improving the stability and reliability of the recognition effect.

[0098] According to an embodiment of the present invention, in step S104, the comprehensive descriptor fuses the original image features, the Gram angle and field features, and the Gram angle difference field features, and each feature is given a corresponding feature weight according to its importance. Therefore, the comprehensive descriptor comprehensively and accurately depicts the image information on the surface of the coal conveyor belt, especially the potential foreign object features. Taking the comprehensive descriptor as the input, it is passed to a pre-trained classifier, which is a neural network model and has the ability to learn and identify patterns from the input data. The classifier will output a foreign object recognition result. The foreign object recognition result is usually a label or a set of probability values, clearly indicating whether there is a foreign object in the image and the specific type of the foreign object.

[0099] According to an embodiment of the present invention, in step S105, during the transportation of coal, large pieces of gangue, anchor bolts and other foreign objects are often mixed. If the foreign objects are large, hard objects such as large pieces of gangue and anchor bolts, they will cause serious interference and even damage to the operation of the belt. Therefore, it is necessary to immediately start the cleaning device to remove them to ensure the normal operation of the belt and the smooth transportation of coal.

[0100] According to an embodiment of the present invention, in step S106, after the cleaning device completes the first cleaning, steps S101 to S105 can be repeated to obtain the foreign object recognition result again, so as to conduct an effect evaluation, that is, to determine whether to start the cleaning device for cleaning until there are no foreign objects on the surface of the belt.

[0101] The foreign object cleaning method for a coal conveying belt based on visual features according to an embodiment of the present invention considers the relationship between image pixels through the Gram angular field and the Gram difference field, improving the accuracy of the description of foreign objects on the coal conveying belt. The adaptive weight strategy enables the single feature weight to be dynamically updated with the recognized image, further optimizing the robustness and accuracy of the recognition result. When determining the Gram angular field image and the Gram angular difference field image, statistical features of the original image can be extracted through multiple dimensions, effectively capturing various attributes of foreign objects on the coal conveying belt and providing rich information for subsequent recognition. By mapping the statistical features to a two-dimensional image to generate the Gram angular field and the Gram angular difference field, an efficient representation of the features of the original image is achieved, thus significantly improving the accuracy of foreign object recognition. When determining the comprehensive descriptor of foreign objects on the surface of the coal conveying belt, the features of the original image, the Gram angular field image, and the Gram angular difference field image can be fused, and the features can be dynamically weighted by combining the adaptive weighting method, enabling the comprehensive descriptor to better reflect the essential features of foreign objects, thereby enhancing the representativeness of the comprehensive descriptor, effectively coping with the changes of foreign objects in different scenarios, and improving the stability and reliability of the recognition effect.

[0102] Figure 4 An exemplary block diagram of a foreign object cleaning system for a coal conveying belt based on visual features according to an embodiment of the present invention is shown, and the system includes:

[0103] An original image module for obtaining the original image of the coal conveying belt by installing a camera;

[0104] A Gram angular field image and Gram angular difference field image module for analyzing and processing the original image to determine the Gram angular field image and the Gram angular difference field image;

[0105] A comprehensive descriptor module for determining a comprehensive descriptor of foreign objects on the surface of the coal conveying belt according to the original image, the Gram angular field image, and the Gram angular difference field image;

[0106] A foreign object recognition result module for inputting the comprehensive descriptor into a classifier to obtain a foreign object recognition result;

[0107] A judgment module for determining whether to start a cleaning device for cleaning according to the foreign object recognition result;

[0108] A feedback module for, if starting the cleaning device for cleaning, feeding back the cleaning effect after the cleaning device completes the cleaning until there are no foreign objects on the belt surface.

[0109] The present invention can be a method, a device, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present invention.

[0110] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The object of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and without departing from the said principles, any variations or modifications can be made to the embodiments of the present invention.

Claims

1. A method for cleaning foreign matter from a coal conveyor belt based on visual features, characterized in that: include: By installing a camera, the original image of the coal conveyor belt is obtained; Analyzing and processing the original image to determine a Gram angle sum field image and a Gram angle difference field image; Determining a comprehensive descriptor of foreign matter on the surface of the coal conveyor belt according to the original image, the Gram angle sum field image and the Gram angle difference field image; Inputting the comprehensive descriptor into a classifier to obtain a foreign body recognition result; Determining whether to start a cleaning device for cleaning according to the foreign object identification result; If the cleaning device is started for cleaning, the cleaning device will provide feedback on the cleaning effect after it completes cleaning until there is no foreign matter on the belt surface; Expanding the original image to obtain an expanded image; Evenly divide each channel of the expanded image to obtain 9 segmented images; Determining a statistical descriptor according to the segmented image; comprising: Calculate the maximum, minimum, mean, mode, median, range, variance and standard deviation for each segmented image; According to the formula S v =(S v1 ,S v2 ,S v3 ,S v4 ,S v5 ,S v6 ,S v7 ,S v8 ) Determine the description vector S of the vth segmented image v , where S v1 is the maximum value of the vth segmented image, S v2 is the minimum value of the vth segmented image, S v3 is the mean of the vth segmented image, S v4 is the mode of the vth segmented image, S v5 is the median of the vth segmented image, S v6 is the range of the vth segmented image, S v7 is the variance of the vth segmented image, S v8 is the standard deviation of the vth segmented image, v≤9, and v is a positive integer; The description vectors of the segmented image are combined in a hierarchical combination manner, wherein the first-level combination forms four different first-level description vectors according to four different combination methods, and the second-level combination forms a statistical descriptor according to the result of the first-level combination; Based on the statistical descriptors, a Gram angle sum field image and a Gram angle difference field image are determined.

2. The method for cleaning foreign matter from a coal conveyor belt based on visual characteristics according to claim 1 is characterized in that: The description vectors of the segmented image are combined in a hierarchical combination manner. The first-level combination forms four different first-level description vectors according to four different combination methods. The second-level combination forms a statistical descriptor according to the result of the first-level combination, including: According to the formula <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> 1 <h2 style=";text-align:left;direction:ltr"> (S1, S2, S3, S6, S5, S4, S7, S8, S9) <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> (S1,S4,S7,S8,S5,S2,S3,S6,S9) <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> (S7,S8,S4,S1,S5,S9,S6,S2,S3) <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> 4 <h2 style=";text-align:left;direction:ltr"> (S1,S4,S2,S3,S5,S7,S8,S6,S9) S=(S 1 ,S 2 ,S 3 ,S 4 ) Determine the statistical descriptor S, where S 1 , S 2 , S 3 and S 4 There are four different first-level description vectors, S1, S2, S3, S4, S5, S6, S7, S8 and S9 are the description vectors of the vth segmented image respectively.

3. The method for cleaning foreign matter from a coal conveyor belt based on visual characteristics according to claim 1 is characterized in that: Determining a Gram angle sum field image and a Gram angle difference field image according to the statistical descriptor comprises: According to the formula Get the normalized vector The i-th dimension of Where n is the dimension of the statistical descriptor, s i is the component of the i-th dimension of the statistical descriptor S, and i and n are both positive integers; According to the formula Determine the Gram angle and field image A GASF (i, j) and the Gram angle difference field image A GADF (i,j), where θ i for The cosine of the angle, r i t i The corresponding radius, t i For i The corresponding timestamp, M is the normalization factor, θ1, θ2, …, θ j They are The cosine of the angle , i≤n, j≤n, and j is a positive integer.

4. The method for cleaning foreign matter from a coal conveyor belt based on visual characteristics according to claim 1 is characterized in that: Determining a comprehensive descriptor of foreign matter on the surface of the coal conveyor belt according to the original image, the Gram angle sum field image and the Gram angle difference field image, including: Obtaining original image features, Gram angle difference field features and Gram angle sum field features according to the original image, the Gram angle sum field image and the Gram angle difference field image; Determining original image feature weights, Gram angle and field feature weights, and Gram angle difference field feature weights according to the original image feature, the Gram angle and field feature, and the Gram angle difference field feature; A comprehensive descriptor of foreign matter on the surface of the coal conveyor belt is determined based on the original image features, the Gram angle and field features, the Gram angle difference field features, the original image feature weights, the Gram angle and field feature weights and the Gram angle difference field feature weights.

5. The method for cleaning foreign matter from a coal conveyor belt based on visual characteristics according to claim 4 is characterized in that: Determining original image feature weights, Gram angle and field feature weights, and Gram angle difference field feature weights according to the original image feature, the Gram angle and field feature, and the Gram angle difference field feature weights, including: According to the formula E=(E1,E2,...,E m ) W_B_DM=[B_DM1×W1,B_DM2×W2,...,B_DM m ×W m ] <h2 style=";text-align:left;direction:ltr">PIS<h2 style=";text-align:left;direction:ltr"> y <h2 style=";text-align:left;direction:ltr"> =max(W_B_DM1,W_B_DM2,W_B_DM3) <h2 style=";text-align:left;direction:ltr">NIS<h2 style=";text-align:left;direction:ltr"> y <h2 style=";text-align:left;direction:ltr"> = min(W_B_DM1,W_B_DM2,W_B_DM3) Determine the Gram angle and field feature weight H1, the original image feature weight H2 and the Gram difference field feature weight H3, where DM is the decision matrix, F_GASF is the Gram angle and field feature, F_I is the original image feature, F_GADF is the Gram angle difference field feature, B_DM xy is the value of the xth row and yth column of the standardized decision matrix, DM xy is the value of the decision matrix in row x and column y, DM y is the value of the yth column of the decision matrix, E1, E2, …, E m are the components of vector E, E y is the information entropy of the yth column of the standardized decision matrix, E is the entropy vector, W y is the normalized information entropy weight of the yth column of the standardized decision matrix, W_B_DM is the weighted decision matrix, B_DM1, B_DM2, …, B_DM m are the values ​​of the 1st, 2nd, …, mth columns of the standardized decision matrix, W1, W2, …, W m is the normalized information entropy weight of the 1st, 2nd, ..., mth columns of the standardized decision matrix, PIS y is the value of the positive ideal solution in the yth column of the weighted decision matrix, NIS y is the value of the negative ideal solution in the yth column of the weighted decision matrix, W_B_DM1, W_B_DM2 and W_B_DM3 are the values ​​of the 1st, 2nd and 3rd rows of the weighted decision matrix, W_B_DM xy is the value of the xth row and yth column of the weighted decision matrix, is the distance to the positive ideal solution in the xth row of the weighted decision matrix, is the distance to the negative ideal solution of the xth row of the weighted decision matrix, H x is the feature weight of the x-th row of the decision matrix, m is the number of decision matrix columns, x≤3, y≤m, and x, y and m are all positive integers, max is the maximum value function, and min is the minimum value function.

6. The method for cleaning foreign matter from a coal conveyor belt based on visual characteristics according to claim 5 is characterized in that: Determining a comprehensive descriptor of foreign matter on the surface of the coal conveyor belt according to the original image feature, the Gram angle and field feature, the Gram angle difference field feature, the original image feature weight, the Gram angle and field feature weight, and the Gram angle difference field feature weight includes: According to the formula A comprehensive descriptor F for determining foreign matter on the surface of coal conveyor belts.

7. A visual feature coal conveyor belt foreign body cleaning system, used to perform the visual feature coal conveyor belt foreign body cleaning method as described in any one of claims 1 to 6, characterized in that: include: The original image module is used to obtain the original image of the coal conveyor belt by installing a camera; A Gram angle sum field image and a Gram angle difference field image module, used for analyzing and processing the original image to determine a Gram angle sum field image and a Gram angle difference field image; A comprehensive descriptor module, for determining a comprehensive descriptor of foreign matter on the surface of the coal conveyor belt according to the original image, the Gram angle sum field image and the Gram angle difference field image; A foreign body identification result module, used for inputting the comprehensive descriptor into a classifier to obtain a foreign body identification result; A judgment module, used to determine whether to start a cleaning device for cleaning according to the foreign object recognition result; The feedback module is used to feedback the cleaning effect after the cleaning device is started to clean until there is no foreign matter on the belt surface; Expanding the original image to obtain an expanded image; Evenly divide each channel of the expanded image to obtain 9 segmented images; Determining a statistical descriptor according to the segmented image; comprising: Calculate the maximum, minimum, mean, mode, median, range, variance and standard deviation for each segmented image; According to the formula S v =(S v1 ,S v2 ,S v3 ,Sx4,S v5 ,S v6 ,S v7 ,S v8 ) Determine the description vector S of the vth segmented image v , where S v1 is the maximum value of the vth segmented image, S v2 is the minimum value of the vth segmented image, S v3 is the mean of the vth segmented image, S v4 is the mode of the vth segmented image, S v5 is the median of the vth segmented image, S v6 is the range of the vth segmented image, S v7 is the variance of the vth segmented image, S v8 is the standard deviation of the vth segmented image, v≤9, and v is a positive integer; The description vectors of the segmented image are combined in a hierarchical combination manner, wherein the first-level combination forms four different first-level description vectors according to four different combination methods, and the second-level combination forms a statistical descriptor according to the result of the first-level combination; Based on the statistical descriptors, a Gram angle sum field image and a Gram angle difference field image are determined.

Images