Coal mine transport belt tear detection method, device and system based on binocular vision

Abstract

The present application discloses a method, device and system for detecting tearing of a coal mine conveyor belt based on binocular vision, which belongs to the field of coal mine transportation technology. A first camera is used to obtain a first image of the coal mine conveyor belt and an abnormality is identified for the first image. If the first image is abnormal, a second camera is used to obtain a second image of the coal mine conveyor belt and the first image and the second image are weightedly superimposed. The superimposed image is converted into a frequency value and the maximum and minimum values ​​of the difference between the frequency value and the reference frequency are calculated. The tearing area in the superimposed image is determined based on the maximum and minimum values ​​of the difference. Not only can the image be obtained only by the first camera when the coal mine conveyor belt is normal to reduce the amount of calculation, but also a binocular camera can be used to comprehensively obtain a more accurate tearing area of ​​the coal mine conveyor belt when the coal mine conveyor belt is abnormal, thereby improving the detection accuracy while reducing the amount of calculation as much as possible.

Description

Technical Field

[0001] The present application relates to the field of coal mine transportation technology, and in particular to a method, device and system for detecting tearing of a coal mine transportation belt based on binocular vision. Background Art

[0002] Conveyor belts are easy to operate, have low energy consumption, and can transport large quantities. Therefore, they are widely used in the production, transportation, and processing of coal and other minerals and ores. Belt conveyors are the main equipment for transporting coal underground in coal mines because of their high transportation efficiency. However, in the process of transporting coal underground, the conveyor belts can be easily torn and damaged due to the insertion of foreign objects such as anchor rods and drills, or the forward movement of the conveyor belts caused by the blockage of hard objects. If not discovered and handled in time, this damage will continue to expand, causing the transported objects to fall and causing production accidents.

[0003] With the continuous development of image processing technology, cameras are also used to capture image information of conveyor belts, and the images are analyzed to detect conditions such as tearing on the surface of the conveyor belts. However, the accuracy of image processing is related to its clarity, that is, if a high-definition image is to be obtained, the computing power will increase. Especially in scenes with complex working conditions such as coal mine transportation, the computing power required will be greater, which will increase the difficulty of the entire detection system. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method, device and system for detecting tearing of coal mine conveyor belts based on binocular vision.

[0005] According to one aspect of the present application, a method for detecting tearing of a coal mine conveyor belt based on binocular vision is provided, comprising: using a first camera to acquire a first image of a coal mine conveyor belt; identifying an abnormal area of ​​the first image to obtain an identification result; wherein the identification result indicates whether the first image is abnormal; if the identification result indicates that the first image is abnormal, using a second camera to acquire a second image of the coal mine conveyor belt after a preset time; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; weightedly superimposing the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; converting the pixel value of each pixel point in the superimposed image into a frequency value; wherein the conversion formula is: ;in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), M is the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit; calculate the frequency difference between the frequency value and the reference frequency; wherein the reference frequency is the frequency average of all pixels in the superimposed image; calculate the maximum difference and the minimum difference of the frequency difference in the same column or the same row; determine the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference.

[0006] In one embodiment, a light emitter is provided on the side of the coal mine conveyor belt away from the first camera and the second camera, and the light emitter emits a plurality of parallel line light beams to the coal mine conveyor belt; wherein, abnormal area identification is performed on the first image to obtain the identification result, including: calculating the brightness value of all pixels of the coal mine conveyor belt image in the first image; if the brightness value is greater than a reference brightness, determining that the first image is abnormal; wherein the reference brightness is the brightness value of the tear-free area of ​​the coal mine conveyor belt, and the reference brightness is updated regularly.

[0007] In one embodiment, abnormal areas of the first image are identified to obtain an identification result, including: binarizing the first image to obtain a binary image; segmenting the binary image to obtain a plurality of segmented areas; if there is a closed segmented area located in the middle of the first image among the plurality of segmented areas, and the pixel value of the closed segmented area is greater than the pixel value of the segmented area outside the closed segmented area, then the first image is determined to be abnormal.

[0008] In one embodiment, weighted superposition of the first image and the second image to obtain a superimposed image includes: aligning the first image and the second image based on the landmark points of the coal mine conveyor belt; weighted superposition of the closed segmented area in the first image and the corresponding area in the second image to obtain a superimposed closed area; enhancing the superimposed closed area to obtain a target superimposed area; weighted superposition of the external segmented area in the first image and the corresponding area in the second image to obtain a superimposed external area; and combining the target superimposed area and the superimposed external area to obtain the superimposed image.

[0009] In one embodiment, determining the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference includes: if the maximum difference and the minimum difference in the same row or column are adjacent, determining the corresponding pixel point as the boundary pixel point of the tearing area; and fitting the tearing area based on multiple boundary pixel points.

[0010] In one embodiment, the first image includes an empty image of the coal mine conveyor belt when empty and a loaded image of the coal mine conveyor belt when loaded; abnormal area identification is performed on the first image, and the identification result obtained includes: comparing the empty boundary position of the coal mine conveyor belt in the empty image with the loaded boundary position of the coal mine conveyor belt in the loaded image; if the difference between the empty boundary position and the loaded boundary position is greater than a preset difference, determining that the first image is abnormal.

[0011] In one embodiment, abnormal area identification is performed on the first image to obtain an identification result, including: inputting the first image into an image recognition model to obtain image feature information of the first image; if the image feature information includes a wrinkle feature, determining that the first image is abnormal.

[0012] In one embodiment, performing abnormal region identification on the first image to obtain an identification result includes: calculating the variance of all pixels between the first image and a standard image; if the variance is greater than a preset value, determining that the first image is abnormal.

[0013] According to another aspect of the present application, a coal mine conveyor belt tear detection device based on binocular vision is provided, comprising: a first image acquisition module, used to use a first camera to acquire a first image of the coal mine conveyor belt; an abnormal area recognition module, used to identify the abnormal area of ​​the first image to obtain an identification result; wherein the identification result indicates whether the first image is abnormal; a second image acquisition module, used to use a second camera to acquire a second image of the coal mine conveyor belt after a preset time if the identification result indicates that the first image is abnormal; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; an image weighted superposition module, used to weightedly superimpose the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; a pixel frequency calculation module, used to convert the pixel value of each pixel point in the superimposed image into a frequency value; wherein the conversion formula is: ; in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), M is the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit; a frequency difference calculation module, used to calculate the frequency difference between the frequency value and the reference frequency; wherein the reference frequency is the frequency average value of all pixels in the superimposed image; a frequency change calculation module, used to calculate the maximum difference and the minimum difference of the frequency difference in the same column or the same row; a tearing area determination module, used to determine the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference.

[0014] According to another aspect of the present application, a coal mine transport belt tear detection system based on binocular vision is provided, including a binocular camera and the above-mentioned coal mine transport belt tear detection device based on binocular vision.

[0015] The present application provides a method, device and system for detecting a tear in a coal mine conveyor belt based on binocular vision. The method adopts a first camera to obtain a first image of the coal mine conveyor belt; an abnormal area is identified in the first image to obtain an identification result; if the identification result indicates that the first image is abnormal, a second camera is used to obtain a second image of the coal mine conveyor belt after a preset time period; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time period is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; the first image and the second image are weightedly superimposed to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; the pixel value of each pixel point in the superimposed image is converted into a frequency value; and the frequency difference between the frequency value and the reference frequency is calculated; Among them, the reference frequency is the frequency average value of all pixels in the superimposed image; the maximum difference and the minimum difference of the frequency difference in the same column or the same row are calculated; the tearing area of ​​the superimposed image is determined based on the maximum difference and the minimum difference; that is, first use the first camera to obtain the first image of the coal mine conveyor belt and identify the abnormality of the first image. If the first image is abnormal, use the second camera to obtain the second image of the coal mine conveyor belt and weightedly superimpose the first image and the second image, convert the superimposed image into a frequency value, and calculate the maximum and minimum values ​​of the difference between the frequency value and the reference frequency. The tearing area in the superimposed image is determined based on the maximum difference and the minimum difference. Not only can the image be obtained only by the first camera when the coal mine conveyor belt is normal to reduce the amount of calculation, but also the binocular camera can be used to comprehensively obtain a more accurate tearing area of ​​the coal mine conveyor belt when the coal mine conveyor belt is abnormal, thereby improving the detection accuracy while reducing the amount of calculation as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 It is a flow chart of a method for detecting a tear in a coal mine transport belt using binocular vision provided by an exemplary embodiment of the present application.

[0018] Figure 2 It is a structural schematic diagram of a binocular vision coal mine conveyor belt tear detection device provided by an exemplary embodiment of the present application.

[0019] Figure 3It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0021] Figure 1 FIG. 1 is a flow chart of a method for detecting a tear in a coal mine conveyor belt using binocular vision provided by an exemplary embodiment of the present application. Figure 1 As shown, the coal mine conveyor belt tear detection method based on binocular vision includes the following steps:

[0022] Step 110: Use a first camera to acquire a first image of the coal mine conveyor belt.

[0023] The first camera is one of the binocular cameras. The present application first uses the first camera to obtain a first image of the coal mine conveyor belt in real time to monitor the coal mine conveyor belt.

[0024] Step 120: Identify the abnormal area of ​​the first image to obtain a recognition result.

[0025] Among them, the recognition result indicates whether there is an abnormality in the first image. The present application first performs a comprehensive but rough analysis and recognition of the coal mine conveyor belt. On the one hand, it can give priority to expanding the detection range, so as to achieve rapid detection, so as to improve the detection efficiency and reduce the amount of calculation. On the other hand, it can also use the first image to achieve a rough detection of the coal mine conveyor belt. Since the coal mine conveyor belt is in a normal state in most cases and most positions, that is, there is no tearing, only the analysis and recognition of the first image can realize the detection of the coal mine conveyor belt in most cases.

[0026] Step 130: If the recognition result indicates that the first image is abnormal, a second camera is used to obtain a second image of the coal mine conveyor belt after a preset time period.

[0027] Among them, the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time length is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt. The field of view of the first camera in the present application is larger than the field of view of the second camera, and the pixel value of the first camera is lower than the pixel value of the second camera. The present application sets the field of view and pixel value of the first camera and the second camera in the binocular camera to be different, that is, the first camera with a large field of view but a low pixel value is used to obtain the first image containing the coal mine conveyor belt, and the second camera with a small field of view but a high pixel value is used to obtain the second image containing the coal mine conveyor belt, thereby obtaining a global rough image (first image) and a local fine image (second image). When the first image is abnormal, the second camera is used to obtain a local fine image of the coal mine conveyor belt to improve the accuracy of tear detection. Since the relative position of the first camera and the second camera is fixed and the shooting angle is fixed, the present application can calculate the preset time length for the corresponding position of the first image to reach the shooting area of ​​the second camera according to the running speed of the coal mine conveyor belt, thereby improving the alignment accuracy of the first image and the second image.

[0028] Step 140: Perform weighted superposition on the first image and the second image to obtain a superimposed image.

[0029] Among them, the weight of the first image is equal to the ratio of the minimum pixel difference and the maximum pixel difference between the first image and the second image. Since the shooting environment of the first image and the second image (such as ambient brightness and camera parameters) is different, there is a certain difference between the first image and the second image. The present application fuses the first image and the second image by weighted superposition, thereby comprehensively determining the tear area of ​​the coal mine conveyor belt by combining the features of the first image and the second image to improve the detection accuracy. Since the clarity of the second image is relatively higher than that of the first image, the present application determines the weight of the first image by the ratio of the minimum pixel difference and the maximum pixel difference between the first image and the second image, that is, the greater the difference between the first image and the second image, the smaller the weight of the first image, so that when the difference between the first image and the second image is large, the second image can be preferred, thereby improving the authenticity and reliability of the image.

[0030] Step 150: Convert the pixel value of each pixel in the superimposed image into a frequency value.

[0031] The conversion formula is: ;in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), Mis the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit.

[0032] Step 160: Calculate the frequency difference between the frequency value and the reference frequency.

[0033] The reference frequency is the average frequency of all pixels in the superimposed image. The present application can obtain the relative value of the frequency value of each pixel by calculating the frequency difference between the frequency value of each pixel and the reference frequency.

[0034] Step 170: Calculate the maximum value and the minimum value of the frequency difference in the same column or the same row.

[0035] The present application calculates the maximum and minimum values ​​of the frequency difference in the same column or row to obtain the change point where the frequency in the same column or row changes from large to small or from small to large, so as to determine the dividing point in the image.

[0036] Step 180: Determine the torn area of ​​the superimposed image based on the maximum difference value and the minimum difference value.

[0037] After determining the demarcation points of each area, the present application can divide the superimposed image into regions based on all the demarcation points, thereby obtaining a closed area located in the middle of the coal mine conveyor belt, that is, a tearing area.

[0038] The present application provides a method for detecting a tear in a coal mine conveyor belt based on binocular vision, which uses a first camera to obtain a first image of the coal mine conveyor belt; identifies an abnormal area in the first image to obtain an identification result; if the identification result indicates that the first image is abnormal, uses a second camera to obtain a second image of the coal mine conveyor belt after a preset time; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; weightedly superimposes the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; converts the pixel value of each pixel point in the superimposed image into a frequency value; calculates the frequency difference between the frequency value and the reference frequency; wherein, The reference frequency is the frequency average of all pixels in the superimposed image; the maximum difference and the minimum difference of the frequency difference in the same column or the same row are calculated; the tearing area of ​​the superimposed image is determined based on the maximum difference and the minimum difference; that is, the first image of the coal mine conveyor belt is first obtained by using the first camera and the first image is identified as abnormal. If the first image is abnormal, the second camera is used to obtain the second image of the coal mine conveyor belt and the first image and the second image are weightedly superimposed, the superimposed image is converted into a frequency value, and the maximum and minimum values ​​of the difference between the frequency value and the reference frequency are calculated. The tearing area in the superimposed image is determined based on the maximum difference and the minimum difference. Not only can the image be obtained only by the first camera when the coal mine conveyor belt is normal to reduce the amount of calculation, but also the binocular camera can be used to comprehensively obtain a more accurate tearing area of ​​the coal mine conveyor belt when the coal mine conveyor belt is abnormal, thereby improving the detection accuracy while reducing the amount of calculation as much as possible.

[0039] In one embodiment, a light emitter is provided on the side of the coal mine conveyor belt away from the first camera and the second camera, and the light emitter emits a plurality of parallel line light beams to the coal mine conveyor belt; wherein, the specific implementation method of the above step 120 may be: calculating the brightness value of all pixels of the coal mine conveyor belt image in the first image; if the brightness value is greater than the reference brightness, it is determined that the first image is abnormal; wherein, the reference brightness is the brightness value of the tear-free area of ​​the coal mine conveyor belt, and the reference brightness is updated regularly.

[0040] The present application sets a light emitter on the side of the coal mine conveyor belt away from the binocular camera, and uses the light emitter to vertically emit multiple parallel line beams to the coal mine conveyor belt. On the one hand, it can realize the illumination in the mining area with dark ambient light to ensure the clarity of the image obtained by the binocular camera. On the other hand, it can also judge the transmittance of the corresponding position of the pixel point according to the brightness value of each pixel point in the second image captured by the second camera. Since the wear or tear of the coal mine conveyor belt usually causes the thickness of this position to be thinner than other positions, the transmittance of this position is significantly stronger than other positions. The present application calculates the brightness value of each pixel point in the target area. If the brightness value is greater than the reference brightness, it means that the transmittance of the corresponding position of the pixel point is high, that is, there is tearing or wear, etc. It should be understood that the present application can regularly update the reference brightness according to the use time of the coal mine conveyor belt. For example, the transmittance of the coal mine conveyor belt will become higher and higher during use. At this time, the reference brightness can be gradually increased to match its actual state.

[0041] In one embodiment, the specific implementation method of the above-mentioned step 120 may be: binarizing the first image to obtain a binary image; segmenting the binary image to obtain multiple segmented areas; if there is a closed segmented area located in the middle of the first image among the multiple segmented areas, and the pixel value of the closed segmented area is greater than the pixel value of the outer segmented area of ​​the closed segmented area, then the first image is determined to be abnormal.

[0042] Specifically, since the grayscale value of the coal mine conveyor belt is somewhat different from other objects in the environment, the present application can perform binarization processing on the first image to segment the image of the coal mine conveyor belt from the first image, so that all environmental objects can be excluded to reduce interference, and segmentation is performed on the basis of the binary image to obtain a closed area located in the middle of the first image and having a grayscale value different from other areas of the coal mine conveyor belt, so that the abnormal area can be obtained more accurately. It should be understood that the present application can first set a first grayscale value to distinguish the coal mine conveyor belt from the background image, and then set a second grayscale value to distinguish the normal area from the abnormal area in the coal mine conveyor belt image, and use two steps to improve the segmentation accuracy and avoid mutual interference between the abnormal area and the background area in a single segmentation.

[0043] In one embodiment, the specific implementation method of the above-mentioned step 140 may be: based on the landmark points of the coal mine conveyor belt, aligning the first image and the second image; weightedly superimposing the closed segmented area in the first image and the corresponding area in the second image to obtain a superimposed closed area; enhancing the superimposed closed area to obtain a target superimposed area; weightedly superimposing the external segmented area in the first image and the corresponding area in the second image to obtain a superimposed external area; combining the target superimposed area and the superimposed external area to obtain a superimposed image.

[0044] The present application can align and calibrate the first image and the second image according to the marking points of the coal mine transport belt (for example, marking points at fixed intervals on the edge of the belt), and after the alignment, weightedly superimpose the first image and the second image to determine the closed area in the superimposed image (corresponding to the abnormal area in the first image), and enhance the superimposed closed area to improve its contrast, thereby providing a clearer image for subsequent detection of the torn area.

[0045] In one embodiment, the specific implementation method of the above step 180 may be: if the maximum difference value and the minimum difference value in the same row or column are adjacent, the corresponding pixel point is determined to be the boundary pixel point of the tearing area; based on multiple boundary pixel points, the tearing area is fitted.

[0046] If there are adjacent maximum and minimum difference values ​​in the same row or column, it means that the corresponding pixel point is the dividing point of the two segmented areas (i.e., the boundary pixel point). The torn area can be fitted by combining all the boundary pixel points.

[0047] In one embodiment, the first image includes an empty image of the coal mine conveyor belt when it is empty and a loaded image of the coal mine conveyor belt when it is loaded; the specific implementation method of the above step 120 may be: comparing the empty boundary position of the coal mine conveyor belt in the empty image and the loaded boundary position of the coal mine conveyor belt in the loaded image; if the difference between the empty boundary position and the loaded boundary position is greater than the preset difference, the first image is determined to be abnormal.

[0048] The present application obtains an empty image of the coal mine conveyor belt when it is empty and a loaded image of the coal mine conveyor belt when it is loaded, and then compares the boundary position of the coal mine conveyor belt in the empty image with the boundary position of the coal mine conveyor belt in the loaded image to determine whether the coal mine conveyor belt is operating abnormally. Specifically, due to the pressure of the load, the boundary position of the coal mine conveyor belt will move closer to the middle, but if the boundary of the coal mine conveyor belt is too close to the middle position (that is, the difference between the empty boundary position and the loaded boundary position is greater than the preset difference), it means that the coal mine conveyor belt is too loose or otherwise abnormal. At this time, it can be determined that the coal mine conveyor belt is in an abnormal state, and an alarm can be issued to avoid long-term abnormal operation and increase the loss of the coal mine conveyor belt.

[0049] In one embodiment, the specific implementation method of the above step 120 may be: inputting the first image into an image recognition model to obtain image feature information of the first image; if the image feature information includes a wrinkle feature, determining that the first image is abnormal.

[0050] The present application can pre-set an image recognition model and use a large number of samples for training to obtain an image recognition model with high recognition accuracy. In the actual detection process, the image recognition model is used to identify the wrinkle features on the coal mine conveyor belt in real time. If the image feature information includes wrinkle features, it means that the coal mine conveyor belt is in a folded or twisted operating state. At this time, it can be determined that the state information of the coal mine conveyor belt is abnormal, and an alarm can be issued to avoid long-term abnormal operation and increase the loss of the coal mine conveyor belt.

[0051] In one embodiment, the specific implementation of the above step 120 may be: calculating the variance of all pixels between the first image and the standard image; if the variance is greater than a preset value, determining that the first image is abnormal.

[0052] The present application can determine the difference between the first image and the standard image by calculating the pixel variance between the first image and the standard image (the image of the coal mine conveyor belt under normal conditions). If the difference between the first image and the standard image is large, it is determined that the first image is abnormal.

[0053] Figure 2 Schematic diagram of the structure of a binocular vision coal mine conveyor belt tear detection device provided by an exemplary embodiment of the present application. Figure 2 As shown, the coal mine conveyor belt tear detection device 20 based on binocular vision includes: a first image acquisition module 21, which is used to use a first camera to acquire a first image of the coal mine conveyor belt; an abnormal area recognition module 22, which is used to identify the abnormal area of ​​the first image and obtain a recognition result; wherein the recognition result indicates whether the first image is abnormal; a second image acquisition module 23, which is used to use a second camera to acquire a second image of the coal mine conveyor belt after a preset time if the recognition result indicates that the first image is abnormal; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; an image weighted superposition module 24, which is used to weightedly superimpose the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference and the maximum pixel difference between the first image and the second image; a pixel frequency calculation module 25, which is used to convert the pixel value of each pixel point in the superimposed image into a frequency value; wherein the conversion formula is: ; in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), Mis the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit; a frequency difference calculation module 26 is used to calculate the frequency difference between the frequency value and the reference frequency; wherein the reference frequency is the frequency average value of all pixels in the superimposed image; a frequency change calculation module 27 is used to calculate the maximum difference and the minimum difference of the frequency difference in the same column or the same row; a tearing area determination module 28 is used to determine the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference.

[0054] The present application provides a binocular vision-based coal mine conveyor belt tear detection device, which uses a first camera to acquire a first image of the coal mine conveyor belt through a first image acquisition module 21; an abnormal area recognition module 22 recognizes the abnormal area of ​​the first image to obtain a recognition result; if the recognition result indicates that the first image is abnormal, the second image acquisition module 23 uses a second camera to acquire a second image of the coal mine conveyor belt after a preset time; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; an image weighted superposition module 24 performs weighted superposition on the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; a pixel frequency calculation module 25 converts the pixel value of each pixel point in the superimposed image into a frequency value; a frequency difference calculation module 26 calculates Calculate the frequency difference between the frequency value and the reference frequency; wherein the reference frequency is the frequency average value of all pixels in the superimposed image; the frequency change calculation module 27 calculates the maximum difference and the minimum difference of the frequency difference in the same column or the same row; the tearing area determination module 28 determines the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference; that is, first use the first camera to obtain the first image of the coal mine conveyor belt and perform abnormality identification on the first image, if the first image is abnormal, use the second camera to obtain the second image of the coal mine conveyor belt and weightedly superimpose the first image and the second image, convert the superimposed image into a frequency value and calculate the maximum and minimum values ​​of the difference between the frequency value and the reference frequency, and determine the tearing area in the superimposed image based on the maximum difference and the minimum difference. Not only can the image be obtained by only using the first camera when the coal mine conveyor belt is normal to reduce the amount of calculation, but also the binocular camera can be used to comprehensively obtain a more accurate tearing area of ​​the coal mine conveyor belt when the coal mine conveyor belt is abnormal, thereby improving the detection accuracy while reducing the amount of calculation as much as possible.

[0055] In one embodiment, a light emitter is provided on the side of the coal mine conveyor belt away from the first camera and the second camera, and the light emitter emits multiple parallel line light beams to the coal mine conveyor belt; wherein, the above-mentioned abnormal area identification module 22 can be further configured to: calculate the brightness value of all pixels of the coal mine conveyor belt image in the first image; if the brightness value is greater than the reference brightness, it is determined that the first image is abnormal; wherein, the reference brightness is the brightness value of the tear-free area of ​​the coal mine conveyor belt, and the reference brightness is updated regularly.

[0056] In one embodiment, the abnormal area identification module 22 can be further configured as follows: binarizing the first image to obtain a binary image; segmenting the binary image to obtain a plurality of segmented areas; if there is a closed segmented area located in the middle of the first image among the plurality of segmented areas, and the pixel value of the closed segmented area is greater than the pixel value of the segmented area outside the closed segmented area, then the first image is determined to be abnormal.

[0057] In one embodiment, the above-mentioned image weighted overlay module 24 can be further configured as follows: based on the landmark points of the coal mine conveyor belt, aligning the first image and the second image; weighted overlaying the closed segmented area in the first image and the corresponding area in the second image to obtain a superimposed closed area; enhancing the superimposed closed area to obtain a target superimposed area; weighted overlaying the external segmented area in the first image and the corresponding area in the second image to obtain a superimposed external area; and combining the target superimposed area and the superimposed external area to obtain a superimposed image.

[0058] In one embodiment, the tearing area determination module 28 may be further configured as follows: if the maximum difference value and the minimum difference value in the same row or column are adjacent, the corresponding pixel point is determined to be a boundary pixel point of the tearing area; and the tearing area is fitted based on multiple boundary pixel points.

[0059] In one embodiment, the first image includes an empty image of the coal mine conveyor belt when it is empty and a loaded image of the coal mine conveyor belt when it is loaded; the above-mentioned abnormal area identification module 22 can be further configured to: compare the empty boundary position of the coal mine conveyor belt in the empty image and the loaded boundary position of the coal mine conveyor belt in the loaded image; if the difference between the empty boundary position and the loaded boundary position is greater than the preset difference, the first image is determined to be abnormal.

[0060] In one embodiment, the abnormal region identification module 22 may be further configured to: input the first image into an image recognition model to obtain image feature information of the first image; if the image feature information includes a wrinkle feature, then determine that the first image is abnormal.

[0061] In one embodiment, the abnormal region identification module 22 may be further configured to: calculate the variance of all pixels between the first image and the standard image; if the variance is greater than a preset value, determine that the first image is abnormal.

[0062] The present application also provides a coal mine transport belt tear detection system based on binocular vision, including a binocular camera and the above-mentioned coal mine transport belt tear detection device based on binocular vision.

[0063] The present application provides a coal mine conveyor belt tear detection system based on binocular vision, which uses a first camera to obtain a first image of the coal mine conveyor belt; identifies an abnormal area of ​​the first image to obtain an identification result; if the identification result indicates that the first image is abnormal, uses a second camera to obtain a second image of the coal mine conveyor belt after a preset time; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; weightedly superimposes the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; converts the pixel value of each pixel point in the superimposed image into a frequency value; calculates the frequency difference between the frequency value and the reference frequency; wherein, The reference frequency is the frequency average of all pixels in the superimposed image; the maximum difference and the minimum difference of the frequency difference in the same column or the same row are calculated; the tearing area of ​​the superimposed image is determined based on the maximum difference and the minimum difference; that is, the first image of the coal mine conveyor belt is first obtained by using the first camera and the first image is identified as abnormal. If the first image is abnormal, the second camera is used to obtain the second image of the coal mine conveyor belt and the first image and the second image are weightedly superimposed, the superimposed image is converted into a frequency value, and the maximum and minimum values ​​of the difference between the frequency value and the reference frequency are calculated. The tearing area in the superimposed image is determined based on the maximum difference and the minimum difference. Not only can the image be obtained only by the first camera when the coal mine conveyor belt is normal to reduce the amount of calculation, but also the binocular camera can be used to comprehensively obtain a more accurate tearing area of ​​the coal mine conveyor belt when the coal mine conveyor belt is abnormal, thereby improving the detection accuracy while reducing the amount of calculation as much as possible.

[0064] Below, reference Figure 3 The electronic device according to the embodiment of the present application is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signal from them.

[0065] Figure 3 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0066] like Figure 3 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0067] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0068] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0069] In one example, the electronic device 10 may further include: an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0070] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector, which is used to receive the collected input signals from the first device and the second device.

[0071] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0072] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0073] Of course, to simplify, Figure 3 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0074] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0075] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0076] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0077] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0078] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0079] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0080] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0081] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0082] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for detecting tearing of coal mine conveyor belts based on binocular vision, characterized in that: include: Acquiring a first image of a coal mine conveyor belt using a first camera; Performing abnormal region recognition on the first image to obtain a recognition result; wherein the recognition result indicates whether the first image has an abnormality; If the recognition result indicates that the first image is abnormal, a second camera is used to obtain a second image of the coal mine conveyor belt after a preset time period; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time period is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; Performing weighted superposition on the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; The pixel value of each pixel in the superimposed image is converted into a frequency value; wherein the conversion formula is: ； in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), M is the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit; Calculating a frequency difference between the frequency value and a reference frequency; wherein the reference frequency is the frequency average value of all pixels in the superimposed image; Calculate the maximum value and the minimum value of the frequency difference in the same column or row; Determining a torn area of ​​the superimposed image based on the maximum difference value and the minimum difference value; A light emitter is provided on one side of the coal mine conveyor belt away from the first camera and the second camera, and the light emitter emits a plurality of parallel line beams to the coal mine conveyor belt; wherein, the abnormal area of ​​the first image is identified to obtain the identification result, including: calculating the brightness value of all pixels of the coal mine conveyor belt image in the first image; if the brightness value is greater than a reference brightness, determining that the first image is abnormal; wherein the reference brightness is the brightness value of the non-tear area of ​​the coal mine conveyor belt, and the reference brightness is updated regularly; Identifying an abnormal region of the first image to obtain an identification result includes: binarizing the first image to obtain a binary image; segmenting the binary image to obtain a plurality of segmented regions; if a closed segmented region located in the middle of the first image exists among the plurality of segmented regions, and a pixel value of the closed segmented region is greater than a pixel value of a segmented region outside the closed segmented region, determining that the first image is abnormal; The first image and the second image are weightedly superimposed to obtain a superimposed image, which includes: aligning the first image and the second image based on the landmarks of the coal mine conveyor belt; weighted superimposing the closed segmented area in the first image and the corresponding area in the second image to obtain a superimposed closed area; enhancing the superimposed closed area to obtain a target superimposed area; weighted superimposing the external segmented area in the first image and the corresponding area in the second image to obtain a superimposed external area; combining the target superimposed area and the superimposed external area to obtain the superimposed image; Determining the tearing area of ​​the superimposed image based on the maximum difference and the minimum difference includes: if the maximum difference and the minimum difference in the same row or the same column are adjacent, determining the corresponding pixel point as the boundary pixel point of the tearing area; and fitting the tearing area based on multiple boundary pixel points.

2. The method for detecting tearing of coal mine conveyor belt based on binocular vision according to claim 1 is characterized in that: The first image includes an empty image of the coal mine transport belt when it is empty and a loaded image of the coal mine transport belt when it is loaded; The abnormal region is identified on the first image, and the identification result obtained includes: Comparing the unloaded boundary position of the coal mine conveyor belt in the unloaded image with the loaded boundary position of the coal mine conveyor belt in the loaded image; If the difference between the unloaded boundary position and the loaded boundary position is greater than a preset difference, it is determined that the first image is abnormal.

3. The method for detecting tearing of coal mine conveyor belt based on binocular vision according to claim 1 is characterized in that: The abnormal region is identified on the first image, and the identification result obtained includes: Inputting the first image into an image recognition model to obtain image feature information of the first image; If the image feature information includes a wrinkle feature, it is determined that the first image is abnormal.

4. The method for detecting tearing of coal mine conveyor belt based on binocular vision according to claim 1 is characterized in that: The abnormal region is identified on the first image, and the identification result obtained includes: Calculating the variance of all pixels between the first image and the standard image; If the variance is greater than a preset value, it is determined that the first image is abnormal.

5. A coal mine conveyor belt tear detection device based on binocular vision, characterized in that: include: A first image acquisition module, used for acquiring a first image of a coal mine conveyor belt using a first camera; an abnormal region identification module, used to identify abnormal regions of the first image and obtain an identification result; wherein the identification result indicates whether the first image has an abnormality; A second image acquisition module, configured to acquire a second image of the coal mine conveyor belt using a second camera after a preset time period if the recognition result indicates that the first image is abnormal; wherein the first camera and the second camera are arranged on one side of the coal mine conveyor belt, and the second camera is arranged downstream of the first camera along the running direction of the coal mine conveyor belt, and the preset time period is determined according to the distance between the first camera and the second camera and the running speed of the coal mine conveyor belt; An image weighted superposition module, used for weighted superposition of the first image and the second image to obtain a superimposed image; wherein the weight of the first image is equal to the ratio of the minimum pixel difference to the maximum pixel difference between the first image and the second image; The pixel frequency calculation module is used to convert the pixel value of each pixel in the superimposed image into a frequency value; wherein the conversion formula is: ； in, P ( u,v ) is the frequency domain point ( u,v ), g ( x,y ) is the pixel point ( x,y ), M is the number of horizontal pixels in the first image, N is the number of vertical pixels in the first image, and i is an imaginary unit; A frequency difference calculation module, used to calculate the frequency difference between the frequency value and a reference frequency; wherein the reference frequency is the frequency average value of all pixels in the superimposed image; A frequency change calculation module, used for calculating the maximum value and the minimum value of the frequency difference in the same column or the same row; A tearing region determining module, configured to determine a tearing region of the superimposed image based on the maximum difference value and the minimum difference value; A light emitter is provided on one side of the coal mine conveyor belt away from the first camera and the second camera, and the light emitter emits a plurality of parallel line beams to the coal mine conveyor belt; wherein the abnormal area recognition module is further configured to: calculate the brightness value of all pixels of the coal mine conveyor belt image in the first image; if the brightness value is greater than a reference brightness, the first image is determined to be abnormal; wherein the reference brightness is the brightness value of the non-tear area of ​​the coal mine conveyor belt, and the reference brightness is updated regularly; The abnormal region identification module is further configured to: perform binarization processing on the first image to obtain a binarized image; segment the binarized image to obtain a plurality of segmented regions; if there is a closed segmented region located in the middle of the first image among the plurality of segmented regions, and the pixel value of the closed segmented region is greater than the pixel value of the segmented region outside the closed segmented region, then determine that the first image is abnormal; The image weighted superposition module is further configured to: align the first image and the second image based on the landmarks of the coal mine conveyor belt; weightedly superimpose the closed segmented area in the first image and the corresponding area in the second image to obtain a superimposed closed area; enhance the superimposed closed area to obtain a target superimposed area; weightedly superimpose the external segmented area in the first image and the corresponding area in the second image to obtain a superimposed external area; and combine the target superimposed area and the superimposed external area to obtain the superimposed image; The tearing area determination module is further configured to: if the maximum difference value and the minimum difference value in the same row or column are adjacent, determine the corresponding pixel point as the boundary pixel point of the tearing area; and fit the tearing area based on multiple boundary pixel points.

6. The coal mine conveyor belt tear detection system based on binocular vision is characterized by: include: A binocular camera and a coal mine transport belt tear detection device based on binocular vision as described in claim 5.

Images