Belt unmanned management method and system based on multi-mode fusion photoelectric technology

Abstract

The invention discloses a belt unmanned management method and system based on a multi-mode fusion photoelectric technology, and belongs to the technical field of belt deviation rectification. The method comprises the steps that a coal conveying belt image is shot; capturing the edge of the coal conveying belt through the coal conveying belt image; comparing the captured edge of the coal conveying belt with the arranged edge of the coal conveying belt to judge whether the coal conveying belt deviates or not; if deviation occurs, deviation correction is conducted through pushing devices arranged on the two sides of the coal conveying belt. By means of the method, unmanned management of the coal conveying belt is achieved, deviation of the coal conveying belt can be found at the first time, automatic deviation correction is conducted at the first time, efficiency is improved, and meanwhile the labor intensity of workers is lowered.

Description

Technical Field

[0001] The present invention relates to the technical field of belt deviation correction, and in particular to an unmanned belt management method and system based on multi-modal fusion photoelectric technology. Background Art

[0002] The coal conveyor belt is an important facility in a coal-fired power plant. Its safe operation is directly related to the stable power supply and economic benefits of the power plant. Belt deviation is a common fault. In the existing technology, workers are required to detect whether the belt is deviated, which is inefficient and increases the labor intensity of the workers. Summary of the invention

[0003] In order to solve the problems of the prior art, the present invention provides an unmanned belt management method and system based on multi-modal fusion photoelectric technology.

[0004] On the one hand, a belt unmanned management method based on multi-modal fusion photoelectric technology is provided, the method comprising:

[0005] Take images of the coal conveyor belt;

[0006] capturing the edge of the coal conveyor belt through the coal conveyor belt image;

[0007] Determine whether the coal conveyor belt is deviated by comparing the captured edge of the coal conveyor belt with the set edge of the coal conveyor belt;

[0008] If it deviates, it will be corrected by the pushing devices installed on both sides of the coal conveyor belt.

[0009] Further, capturing the edge of the coal conveyor belt through the belt image specifically includes:

[0010] Binarizing the coal conveyor belt image;

[0011] Find out the light and dark distribution information of the image through the binary coal conveyor belt image;

[0012] Exclude all points with color values ​​higher than 127;

[0013] Linearize all remaining light and dark distribution points;

[0014] Calculate the slope angles of all lines and select the lines that meet the range of slope angles of the coal conveyor belt;

[0015] The sampling length is set, and lines longer than the sampling length are excluded to obtain the edge of the coal conveyor belt captured in real time.

[0016] Furthermore, the range slope angle of the coal conveyor belt is ±3.5° of the base angle of the coal conveyor belt.

[0017] Furthermore, the method further comprises:

[0018] Perform temperature scanning imaging on the designated area, calculate the temperature of the image block of the designated size, fit the characteristic temperature, and determine whether the temperature of the designated area is abnormal.

[0019] Furthermore, the method further comprises:

[0020] The sampled images are analyzed to calculate the characteristic values ​​of various faults, and the characteristic values ​​are logically calculated. All characteristic values ​​are logically compared with preset values, and finally a decision is made whether to output an early warning based on the comparison results.

[0021] On the other hand, a belt unmanned management system based on multi-modal fusion photoelectric technology is provided, which is used to implement the belt unmanned management method based on multi-modal fusion photoelectric technology, and the system includes:

[0022] A photographing device, used for photographing the image of the coal conveyor belt;

[0023] A coal conveyor belt edge capturing module, used for capturing the edge of the coal conveyor belt through the coal conveyor belt image;

[0024] A first judgment module is used to judge whether the coal conveying belt is deviated according to the comparison between the captured edge of the coal conveying belt and the set edge of the coal conveying belt;

[0025] The pushing device is arranged on both sides of the coal conveyor belt and is used to correct the deviation of the coal conveyor belt when it deviates.

[0026] Furthermore, the coal conveyor belt edge capture module comprises:

[0027] An image binarization module, used for binarizing the coal conveyor belt image;

[0028] A finding module is used to find out the light and dark distribution information of the image through the binary coal conveyor belt image;

[0029] The first exclusion module is used to exclude all points with color values ​​higher than 127;

[0030] A linear distribution module is used to linearly distribute all remaining light and dark distribution points;

[0031] The first calculation module is used to calculate the slope angles of all lines and select the lines within the range of slope angles that meet the coal conveyor belt;

[0032] The second exclusion module is used to exclude lines longer than a set sampling length according to the sampling length, so as to obtain the edge of the coal conveyor belt captured in real time.

[0033] Furthermore, the system further comprises:

[0034] Infrared spectrum scanning equipment, used to perform temperature scanning imaging of a specified area,

[0035] The second calculation module is used to calculate the temperature of the image block of a specified size and fit the characteristic temperature. The second judgment module is used to judge whether the temperature of the specified area is abnormal.

[0036] Furthermore, the system further comprises:

[0037] Image spectrum acquisition equipment, used to collect on-site images;

[0038] The analysis module is used to analyze the sampled images and calculate the characteristic values ​​of various faults;

[0039] A comparison module, used for performing logical calculation on the characteristic values ​​and performing logical comparison on all characteristic values ​​with preset values;

[0040] The early warning module is used to output an early warning when the comparison result is abnormal or when the temperature in the specified area is abnormal.

[0041] Furthermore, the pushing device is an electric push rod or a cylinder.

[0042] The beneficial effect brought about by the technical solution provided by the embodiment of the present invention is as follows: in the present invention, the edge of the coal conveyor belt is captured by shooting an image of the coal conveyor belt, and whether it has deviated is determined by comparing the captured edge of the coal conveyor belt with the set edge of the coal conveyor belt; if it has deviated, the deviation is corrected by the pushing devices arranged on both sides of the coal conveyor belt, thereby realizing unmanned management of the coal conveyor belt, being able to detect the deviation of the coal conveyor belt at the first time, and automatically correcting the deviation at the first time, thereby improving efficiency while reducing the labor intensity of the staff.

[0043] Secondly, the present invention performs temperature scanning imaging on the working area of ​​the coal conveyor belt, thereby determining whether the temperature in the designated area is abnormal, thereby improving safety.

[0044] In addition, in the present invention, images are taken of the working area of ​​the coal conveyor belt to determine whether a fault occurs. When a fault occurs, an early warning can be issued immediately to further improve safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 This is a flow chart of unmanned deviation correction of a coal conveyor belt provided by the present invention;

[0047] Figure 2 It is a flow chart of temperature warning and fault warning provided by the present invention;

[0048] Figure 3 It is a schematic diagram of an unmanned belt management system based on multi-modal fusion photoelectric technology provided by the present invention;

[0049] Figure 4 It is a display state diagram of an unmanned belt management system based on multi-modal fusion photoelectric technology provided by the present invention;

[0050] Figure 5 This is a calculation principle diagram of a belt slope angle provided by the present invention;

[0051] Figure 6 This is a binary image of a coal conveyor belt provided by the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] Embodiment 1

[0055] A belt unmanned management method based on multi-modal fusion photoelectric technology includes the following steps:

[0056] Step (1): a high-resolution industrial camera or a surveillance camera is installed above the coal conveyor belt to capture an image of the coal conveyor belt.

[0057] Step (2): Capture the edge of the coal conveyor belt through the coal conveyor belt image.

[0058] Specifically, it includes: binarizing the captured coal conveyor belt image, finding out the light and dark distribution information of the image through the binarized coal conveyor belt image, excluding all points with color values ​​higher than 127, and linearizing the distribution of all remaining light and dark distribution points; calculating the slope angle of all lines, and then comparing and screening with the range slope angle of the coal conveyor belt. The range slope angle of the coal conveyor belt is ±3.5° of the coal conveyor belt base angle. The coal conveyor belt base angle is the actual on-site measurement value, and the lines within the range slope angle of the coal conveyor belt are screened out; setting the sampling length, the actual set sampling length is a fixed value, excluding lines greater than the sampling length, and obtaining the coal conveyor belt edge captured in real time.

[0059] It should be noted that the binarized image refers to a grayscale image (or a color image after processing) converted into an image with only two pixel values, usually represented as a black and white image, with a pixel value range of 0 to 255. A threshold is determined. When the threshold is 127, pixel values ​​less than or equal to 127 will be set to 0 (black), and pixel values ​​greater than 127 will be set to 255 (white). For the conveyor belt, it can only be black, so all points with color values ​​higher than 127 are excluded as white, that is, the white area.

[0060] Secondly, the linearized distribution includes a dilation operation: connecting discrete dark points through morphological dilation to form a continuous area, a refinement operation: refining the dilated area into a single-pixel wide skeleton to achieve "linearized distribution", and extracting line segments through Hough transform.

[0061] In addition, when calculating the slope angles of all lines, the line segments detected by the Hough transform are stored in the form of endpoint coordinates, the slope is calculated by the difference between the coordinates of the two points, and the inverse tangent function is used to calculate the angle between the line segment and the horizontal direction. Then, the lines within the range of the slope angle of the coal conveyor belt are screened out and the Pythagorean theorem is used to calculate the length of the line segment in pixels. Lines longer than the sampling length are excluded to obtain the edge of the coal conveyor belt captured in real time.

[0062] It should also be noted that the above linear distribution is to find all the straight lines in the black area in a black and white pixel, that is, to find the black pixel points that can be connected into a line, and then calculate their angle and length; if it meets the requirements, then use its coordinates to calculate the deviation; see Figure 6 , Figure 6 is a binary image, the black and white boundary is the edge of the belt, and the final linearized area is Figure 6 The dashed boxes in C and D indicate distinct black and white areas.

[0063] It should also be noted that the above-mentioned coal conveyor belt reference angle refers to the relative slope angle of the belt sampled after the industrial camera is fixed, specifically the angle between the belt edge and the horizontal line, see Figure 5, after the camera is fixed, the angle between the edge of the belt and the horizontal plane (0° straight line) will also be fixed, Figure 5 The middle belt slope angle is the base angle of the coal conveyor belt.

[0064] The range slope angle of the coal conveyor belt is ±3.5° of the base angle of the coal conveyor belt. If the base angle of the coal conveyor belt is a, the range slope angle of the coal conveyor belt is less than a+3.5 and greater than a-3.5°. Lines within the range slope angle of the coal conveyor belt are selected to prevent visual edge tracking from reaching other non-belt areas.

[0065] It should also be noted that the sampling length is manually input. The belt is a running device with complex working conditions. In order to prevent the edge from catching up with other objects (such as a fence, which is also a straight line and is almost parallel to the belt, and the angles between parallel lines and the same horizontal line are completely equal, so continuing to use angle screening will not be suitable for the site, so length screening is added), on the basis of angle judgment, the length condition is introduced, that is, a chasing length is set manually, and the set length will be distinguished from the length of other objects as much as possible, so that the angle plus length will more stably achieve belt edge capture, for example Figure 5 Although the belt is very long, the chasing edge only grabs a small section of it in order to distinguish it from the guardrail next to it.

[0066] Step (3): Determine whether the coal conveyor belt is deviating by comparing the captured coal conveyor belt edge with the manually set coal conveyor belt edge. Figure 4 Line A is the manually set edge of the coal conveyor belt, and line B is the captured edge of the coal conveyor belt. If the captured edge of the coal conveyor belt exceeds the manually set edge of the coal conveyor belt, it is considered to be deviation.

[0067] Step (4): A pushing device is arranged on both sides of the coal conveyor belt. The pushing device may be a cylinder or an electric push rod. If it is determined that the coal conveyor belt is deviating, if it is deviating to the left, the pushing device on the left side will be controlled to push the coal conveyor belt to deviate to the right, and vice versa.

[0068] Step (5): Perform temperature scanning imaging on the designated area, calculate the temperature of the image block of the designated size, fit the characteristic temperature, and determine whether the temperature of the designated area is abnormal.

[0069] It should be noted that temperature scanning imaging can use an infrared thermal imager that supports multi-band infrared radiation detection to scan a specified area and convert the scanning result into a two-dimensional temperature distribution image (thermal map), where each pixel represents the temperature value at the corresponding position; the generated temperature distribution image is divided into image blocks of a specified size (for example, 8×8, 16×16, etc.), each image block contains a group of pixels, and each pixel corresponds to a temperature value; the characteristic temperature of the image block is calculated, and for each image block, the average temperature of all pixels in the image block is calculated, the maximum temperature value in the image block is taken, the minimum temperature value in the image block is taken, and the weighted average temperature is calculated according to a specific weight function (such as the distance from the center point); and statistical analysis is performed on the characteristic temperatures of all image blocks (such as average temperature, maximum temperature, etc.). Use a fitting algorithm (such as linear regression, polynomial fitting or Gaussian fitting) to generate a characteristic temperature distribution model for the entire area. The characteristic temperature distribution model can reflect the temperature change trend in the area. Set a normal temperature range based on historical data or empirical values. If the characteristic temperature of an image block exceeds the normal range, mark the area as abnormal, highlight the abnormal area on the thermal map, provide the specific temperature value, location information and possible cause analysis (such as overheating, insufficient cooling, etc.) of the abnormal area, and finally issue an early warning to remind the staff.

[0070] In addition, in the temperature abnormality warning system, some areas can be manually shielded. The background uses image frame pulling to shield non-warning areas, such as electric stoves, radiators and other areas with very high temperatures.

[0071] Step (6): Analyze the sampled images, calculate the characteristic values ​​of various faults, perform logical calculations on the characteristic values, perform logical comparisons on all characteristic values ​​with preset values, and finally decide whether to issue an early warning output based on the comparison results.

[0072] It should be noted that the image of the target area should be captured using a camera or industrial camera to ensure clear image quality and avoid the influence of factors such as lighting and noise on the analysis results; image preprocessing: denoising: use Gaussian filtering, median filtering and other methods to remove noise in the image, edge detection: use the Canny edge detection algorithm to extract edge information in the image, image segmentation: divide the image into different areas for subsequent analysis.

[0073] For each fault type, extract the corresponding eigenvalue. The following is the calculation method of the eigenvalue of common fault types:

[0074] (1) Material scattering: Material scattering usually manifests as uneven material distribution or abnormal accumulation; Characteristic value calculation: calculate the density distribution of materials in the area, extract the area ratio of the material area, and analyze the gray value or color distribution of the material area.

[0075] (2) Foreign matter: Foreign matter refers to foreign objects that are not working properly. Feature value calculation: Use background difference method or frame difference method to detect abnormal areas in the image, calculate the shape, size and position of the abnormal area, and extract the color and texture features of the abnormal area.

[0076] (3) Human intrusion, unauthorized entry into the monitoring area; Feature value calculation: Use human detection algorithms (such as HOG+SVM, YOLO, OpenPose, etc.) to identify the human body contour in the image, calculate the area, position and movement trajectory of the human body, and determine whether the human body appears in the preset restricted area.

[0077] (4) Failure of moving parts, such as stuck moving parts, abnormal speed or abnormal vibration; Characteristic value calculation: Use the optical flow method or inter-frame difference method to analyze the motion state of the moving parts, calculate the speed, acceleration and vibration frequency of the moving parts, and extract the trajectory characteristics of the moving parts.

[0078] The characteristic values ​​of various faults are logically combined to generate a comprehensive characteristic value. For example, if the characteristic value of material spreading exceeds the threshold, it is marked as "abnormal material spreading". If the characteristic value of foreign matter exceeds the threshold and the characteristic value of human intrusion is 0, it is marked as "foreign matter intrusion". If the characteristic value of moving part failure is abnormal and the characteristic value of material spreading is normal, it is marked as "moving part failure".

[0079] A preset threshold range is set for each fault type, and the calculated characteristic value is compared with the preset value: if the characteristic value exceeds the preset range, it is considered that a fault exists; if the characteristic value is within the normal range, the system is considered to be operating normally.

[0080] Early warning output: generates early warning signals based on the comparison results: No abnormality: the system operates normally and no alarm is required; Single-type abnormality: generates corresponding warnings according to the fault type (such as "material scattering abnormality", "foreign object invasion", etc.); Multiple-type abnormalities: If multiple characteristic values ​​are abnormal, a comprehensive warning is generated and the priority is marked.

[0081] Output form: Sound and light alarm: remind the operator through buzzer or warning light; Text prompt: display specific fault information on the monitoring interface; Data recording: store fault information in the database for subsequent analysis.

[0082] It is worth mentioning that in the present invention, the edge of the coal conveyor belt is captured by photographing the image of the coal conveyor belt, and whether it has deviated is determined by comparing the captured edge of the coal conveyor belt with the set edge of the coal conveyor belt. If it has deviated, the deviation is corrected by the pushing devices arranged on both sides of the coal conveyor belt, thereby realizing unmanned management of the coal conveyor belt, being able to detect the deviation of the coal conveyor belt at the first time, and automatically correcting the deviation at the first time, thereby improving efficiency and reducing the labor intensity of the staff.

[0083] Secondly, in the present invention, temperature scanning imaging is performed on the working area of the coal conveying belt to determine whether the temperature in the specified area is abnormal, thereby improving safety.

[0084] In addition, in the present invention, an image of the working area of the coal conveying belt is also taken to determine whether a fault has occurred. When a fault occurs, an early warning can be issued immediately, further improving safety.

[0085] Embodiment 2

[0086] Refer to Figure 3 , a belt unmanned management system based on multimodal fusion optoelectronic technology, which is used to implement a belt unmanned management method based on multimodal fusion optoelectronic technology in Embodiment 1, and includes a photographing device, a pushing device, an infrared spectral scanning device, an image spectral acquisition device, an early warning module, and a control system; the control system includes: a coal conveying belt edge capture module, a first judgment module, a second calculation module, a second judgment module, an analysis module, and a comparison module, wherein the coal conveying belt edge capture module includes an image binarization module, a finding module, a first exclusion module, a linear distribution module, a first calculation module, and a second exclusion module.

[0087] The photographing device is a high-resolution industrial camera or a surveillance camera, which is used to take an image of the coal conveying belt; the image binarization module is used to binarize the image of the coal conveying belt; the finding module is used to find out the light and dark distribution information of the image through the binarized coal conveying belt image; the first exclusion module is used to exclude all points with a color value higher than 127; the linear distribution module is used to linearly distribute all the remaining light and dark distribution points; the first calculation module is used to calculate the slope angles of all the lines and screen out the lines within the slope angle range that conforms to the coal conveying belt; the second exclusion module is used to exclude the lines longer than the set sampling length according to the set sampling length to obtain the edge of the coal conveying belt captured in real time; the first judgment module is used to judge whether the coal conveying belt has deviated by comparing the captured edge of the coal conveying belt with the set edge of the coal conveying belt; the pushing device is a cylinder or an electric push rod, which is arranged on both sides of the coal conveying belt and is used to correct the deviation of the coal conveying belt when it deviates.

[0088] The infrared spectral scanning device is an infrared thermal imager, which is used to perform temperature scanning imaging on a specified area; the second calculation module is used to calculate the temperature of an image block of a specified size and fit the characteristic temperature; the second judgment module is used to judge whether the temperature in the specified area is abnormal.

[0089] The image spectrum acquisition device is a camera or an industrial camera, which is used to collect on-site images; the analysis module is used to analyze the sampled images and calculate the characteristic values ​​of various faults (spilled materials, foreign objects, human intrusion, and moving parts failure); the comparison module is used to perform logical calculations on the characteristic values ​​and perform logical comparisons on all characteristic values ​​with preset values; the early warning module is used to output an early warning when the comparison result is abnormal or when the temperature in the specified area is abnormal.

[0090] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The unmanned belt management method based on multi-modal fusion photoelectric technology is characterized by: The method comprises: Take images of the coal conveyor belt; capturing the edge of the coal conveyor belt through the coal conveyor belt image; Determine whether the coal conveyor belt is deviated by comparing the captured edge of the coal conveyor belt with the set edge of the coal conveyor belt; If it deviates, it will be corrected by the pushing devices installed on both sides of the coal conveyor belt.

2. The unmanned belt management method based on multi-modal fusion optoelectronic technology according to claim 1 is characterized in that: Capturing the edge of the coal conveyor belt by using the belt image specifically includes: Binarizing the coal conveyor belt image; Find out the light and dark distribution information of the image through the binary coal conveyor belt image; Exclude all points with color values ​​higher than 127; Linearize all remaining light and dark distribution points; Calculate the slope angles of all lines and select the lines that meet the range of slope angles of the coal conveyor belt; The sampling length is set, and lines longer than the sampling length are excluded to obtain the edge of the coal conveyor belt captured in real time.

3. The unmanned belt management method based on multi-modal fusion optoelectronic technology according to claim 2 is characterized in that: The range slope angle of the coal conveyor belt is ±3.5° of the base angle of the coal conveyor belt.

4. The unmanned belt management method based on multi-modal fusion optoelectronic technology according to claim 1 is characterized in that: The method further comprises: Perform temperature scanning imaging on the designated area, calculate the temperature of the image block of the designated size, fit the characteristic temperature, and determine whether the temperature of the designated area is abnormal.

5. The unmanned belt management method based on multi-modal fusion optoelectronic technology according to claim 4 is characterized in that: The method further comprises: The sampled images are analyzed to calculate the characteristic values ​​of various faults, and the characteristic values ​​are logically calculated. All characteristic values ​​are logically compared with preset values, and finally a decision is made whether to output an early warning based on the comparison results.

6. An unmanned belt management system based on multi-modal fusion optoelectronic technology, used to implement the unmanned belt management method based on multi-modal fusion optoelectronic technology described in any one of claims 1 to 5, characterized in that: The system comprises: A photographing device, used for photographing the image of the coal conveyor belt; A coal conveyor belt edge capturing module, used for capturing the edge of the coal conveyor belt through the coal conveyor belt image; A first judgment module is used to judge whether the coal conveying belt is deviated according to the comparison between the captured edge of the coal conveying belt and the set edge of the coal conveying belt; The pushing device is arranged on both sides of the coal conveyor belt and is used to correct the deviation of the coal conveyor belt when it deviates.

7. The unmanned belt management system based on multi-modal fusion optoelectronic technology according to claim 6 is characterized in that: The coal conveyor belt edge capture module comprises: An image binarization module, used for binarizing the coal conveyor belt image; A finding module is used to find out the light and dark distribution information of the image through the binary coal conveyor belt image; The first exclusion module is used to exclude all points with color values ​​higher than 127; A linear distribution module is used to linearly distribute all remaining light and dark distribution points; The first calculation module is used to calculate the slope angles of all lines and select the lines within the range of slope angles that meet the coal conveyor belt; The second exclusion module is used to exclude lines longer than a set sampling length according to the sampling length, so as to obtain the edge of the coal conveyor belt captured in real time.

8. The unmanned belt management system based on multi-modal fusion optoelectronic technology according to claim 7 is characterized in that: The system further comprises: Infrared spectrum scanning equipment, used to perform temperature scanning imaging of a designated area; The second calculation module is used to calculate the temperature of the image block of a specified size and fit the characteristic temperature; The second judgment module is used to judge whether the temperature of the designated area is abnormal.

9. The unmanned belt management system based on multi-modal fusion optoelectronic technology according to claim 8 is characterized in that: The system further comprises: Image spectrum acquisition equipment, used to collect on-site images; The analysis module is used to analyze the sampled images and calculate the characteristic values ​​of various faults; A comparison module, used for performing logical calculation on the characteristic values ​​and performing logical comparison on all characteristic values ​​with preset values; The early warning module is used to output an early warning when the comparison result is abnormal or when the temperature in the specified area is abnormal.

10. The unmanned belt management system based on multi-modal fusion optoelectronic technology according to claim 6 is characterized in that: The pushing device is an electric push rod or a cylinder.

Images