Real-time image-based system and method for detecting foreign object on conveyor belt

Abstract

A real-time image-based system for detecting foreign object on conveyor belt is disclosed and includes: one or more than one optical imaging device, capturing images at more than one predetermined time points of a target area on a conveyor belt; an image recognition module, which transmits images to the image recognition module, and perform recognition by the object detection model through the image recognition module to generate a recognition result; and a classification module, used to classify the result by the object detection model to eliminate false positive results. The object detection model in this invention uses big data to learn from images, enabling it to identify abnormal areas by comparing pairs of images, thereby detecting various types of foreign objects in real-time and issuing alerts. Thus, reducing the risk of foreign objects cutting the belt, resulting in tangible and intangible losses; and the received image is recognized whether it contains foreign objects, thereby reducing the false alarm rate.

Description

Technical Field

[0001] This invention relates to a detection system and method. Specifically, it refers to a real-time foreign object detection system and method for conveyor belts that combines artificial intelligence with big data to determine whether received images contain foreign objects. Prior Technology

[0002] Currently known technologies based on artificial intelligence models include, for example, Chinese Invention Patent No. CN111507963, Chinese Invention Patent No. CN107043000A, Chinese Invention Patent No. CN107434147A, and Chinese Utility Model Patent No. CN203728106U, which are briefly described below.

[0003] Chinese Invention Patent No. CN111507963 (hereinafter referred to as the '963 Patent) discloses a method for removing foreign objects in coal mines based on target detection. The '963 Patent first performs image segmentation and then classifies the foreign objects using an artificial intelligence model. The '963 Patent targets the detection of two types of foreign objects (i.e., metal and plastic), mainly because coal mining often produces foreign objects such as plastic bags and mineral water bottles abandoned by miners, as well as metal wires generated during the production process.

[0004] Chinese Invention Patent No. CN107043000A (hereinafter referred to as '000 Patent) discloses a machine vision-based intelligent safety assurance system for belt conveyors. '000 Patent utilizes an artificial intelligence model to monitor giant anchor bolt foreign objects. Its main drawback is that the developed identification model can only detect a single specific anchor bolt foreign object.

[0005] Chinese Invention Patent No. CN107434147A (hereinafter referred to as '147 Patent) discloses a method for detecting high-hardness foreign objects on coal conveyor belts. '147 Patent uses a coal scale reader to measure the output weight of the conveyor belt (the normal output weight needs to be defined in advance or statistically analyzed based on historical data) to determine if a high-hardness foreign object may be present on the belt.

[0006] Chinese utility model patent CN203728106U (hereinafter referred to as '106 patent) discloses a device for preventing foreign objects in a conveyor belt chute from scratching the belt. The principle of '106 patent is to install a horizontally moving iron rod at the bottom of the chute. When the iron rod stops moving, it will trigger an alarm and stop the conveyor belt. '106 patent envisions that when a foreign object falls from the upper conveyor belt, it will get stuck on the conveyor belt at the bottom of the chute, causing the iron rod to move horizontally.

[0007] Regarding the aforementioned known patents, it is necessary to first define the types of foreign objects and raw materials, and then collect customized data for different scenarios. However, foreign object data is often very scarce, resulting in a significant time commitment for collection and delaying system launch. Furthermore, the types of foreign objects are numerous and may be constantly evolving, so defining them in advance could lead to omissions. Summary of the Invention

[0008] For the reasons mentioned above, the present invention provides an image-based real-time foreign object detection system and method for conveyor belts. The foreign object detection model combines big data with the received images to determine whether they contain foreign objects, thereby reducing the false alarm rate.

[0009] To achieve the above objectives, the present invention provides an image-based real-time foreign object detection system for conveyor belts, comprising: at least one optical imaging device for capturing an image set at least two predetermined time points of a fixed target area on a conveyor belt; an image recognition module for receiving the image set from the optical imaging device at the two predetermined time points of the fixed target area, and the image set being cross-referenced by the image recognition module using a foreign object detection model for identification, thereby generating an identification result; and a classification module for classifying foreign objects and materials according to the identification result output by the image recognition module.

[0010] In some embodiments, the foreign object detection model is based on artificial intelligence and is a model that learns a massive amount of images of various raw materials to form a normal texture feature set through offline pre-training.

[0011] In some embodiments, the image set includes a texture.

[0012] In some embodiments, the foreign object detection model is a classifier trained by a neural network, which includes at least machine learning, deep learning, and meta-learning.

[0013] The present invention further provides an image-based real-time foreign object detection method for conveyor belts, comprising: capturing an image set at two predetermined time points of a fixed target area on a conveyor belt using at least one optical imaging device, wherein the image set includes a texture; receiving the image set from the optical imaging device at the two predetermined time points of the fixed target area using an image recognition module, wherein the image set is cross-compared by the image recognition module using a foreign object detection model to identify and generate a foreign object identification result; and classifying the foreign object identification result into foreign object and non-foreign object categories using a classification module to filter out false positive identification results.

[0014] In some embodiments, the image recognition module is electrically connected to the optical photography device.

[0015] In some embodiments, the foreign object detection model is based on artificial intelligence and is a model pre-trained offline to identify different material textures.

[0016] In some embodiments, the image set includes a texture.

[0017] In some embodiments, the two predetermined time points include a first time point and a second time point. A first image and a second image are captured at the first time point and the second time point respectively by the optical imaging devices, and the first image and the second image are transmitted to the image recognition module. The foreign object detection model, which is pre-trained offline to identify the normal texture features of different raw materials, is then used to compare and analyze whether the textures of the first image and the second image are the same.

[0018] In some embodiments, the foreign object detection model is a classifier trained by a neural network, which includes at least machine learning, deep learning, and meta-learning.

[0019] The advantage of this invention is that the foreign object detection model learns from a normal image set through big data and cross-compares the image sets to find abnormal areas between them. Therefore, it can identify various types of foreign objects, thereby achieving real-time detection and warning of foreign objects, reducing the risk of foreign objects cutting belts and causing losses. Furthermore, the foreign object detection model will confirm the detected abnormal areas to ensure no identification errors, thereby reducing the false alarm rate.

[0020] The foregoing has provided a fairly broad overview of the technical features and advantages of this disclosure, so as to provide a better understanding of the detailed description of this disclosure that follows. Other technical features and advantages constituting the subject matter of this disclosure will be described below. Those skilled in the art to which this disclosure pertains will understand that the concepts and specific embodiments disclosed below can be readily used to modify or design other structures or processes to achieve the same purpose as this disclosure. Those skilled in the art to which this disclosure pertains will also understand that such equivalent constructions cannot depart from the spirit and scope of this disclosure as defined in the appended claims. Simple Explanation of the Diagram

[0021] Figure 1 shows a block diagram of the structure of an image-based real-time foreign object detection system for conveyor belts according to some embodiments of the present invention.

[0022] Figure 2 is a schematic flowchart of an image-based real-time foreign object detection method for conveyor belts according to some embodiments of the present invention.

[0023] Figure 3 shows a schematic diagram of the operation of the optical photography device in the image-based real-time foreign object detection system for conveyor belts according to some embodiments of the present invention, in which the image capture device performs image acquisition on the conveyor belt.

[0024] Figure 4 shows the texture image within the foreign object detection model of the image-based real-time foreign object detection system for conveyor belts according to some embodiments of the present invention.

[0025] Figure 5 shows a schematic diagram of the image recognition module in the image-based conveyor belt real-time foreign object detection system of some embodiments of the present invention interpreting different foreign object detection models.

[0026] Figure 6 shows a schematic diagram of the image recognition module in the image-based conveyor belt real-time foreign object detection system of some embodiments of the present invention when the foreign object detection model is used to determine that the images are identical.

[0027] Figure 7 shows a block diagram of the structure of an image-based real-time foreign object detection system for conveyor belts according to some other embodiments of the present invention.

[0028] Figure 8 shows a schematic diagram of the implementation of the image-based real-time foreign object detection system for conveyor belts shown in Figure 7.

[0029] Figure 9 shows a flowchart of the image-based real-time foreign object detection method for conveyor belts, as illustrated in Figure 7. Implementation

[0030] The specific embodiments of the present invention will be described in detail below with reference to specific circumstances.

[0031] Figure 1 shows a block diagram illustrating the structure of an image-based real-time foreign object detection system 100 for conveyor belts according to some embodiments of the present invention. Referring to Figure 1, the image-based real-time foreign object detection system 100 for conveyor belts of the present invention includes at least one optical imaging device 1 and an image recognition module 2. The present invention is illustrated using an optical imaging device 1 as an example, but is not limited thereto.

[0032] The optical imaging device 1 is used to capture an image set at two predetermined time points on a fixed target area 210 (refer to FIG. 4) on a conveyor belt 200. The image set is composed of multiple images captured by the optical imaging device 1, which will be described in detail later. In some embodiments, the optical imaging device 1 may be a camera, video camera, similar device, or other optical imaging device, but is not limited thereto.

[0033] The image recognition module 2 is electrically connected to the optical imaging device 1, and the image recognition module 2 can also communicate with the optical imaging device 1. The communication connection can include wired and wireless connections, but is not limited to these. The optical imaging device 1 transmits images of at least two predetermined time points (described in detail later) of the fixed target area 210 to the image recognition module 2, and the image set is identified by the image recognition module 2 using a foreign object detection model.

[0034] In some embodiments, the foreign object detection model is based on artificial intelligence and is a model that learns a set of normal texture features from a massive number of images of various raw materials through offline pre-training. Secondly, it is a classifier trained by a neural network, which includes at least machine learning, deep learning, and meta-learning. In some embodiments, the image set includes a texture, such as a material texture.

[0035] Figure 2 shows a flowchart of the image-based real-time foreign object detection method S100 for conveyor belts according to some embodiments of the present invention. Referring again to Figure 2, the image-based real-time foreign object detection method S100 for conveyor belts of the present invention includes: step S110, which involves using an optical imaging device 1 to capture image sets at two predetermined time points on a fixed target area 210 on the conveyor belt 200; and step S120, which involves using an image recognition module 2 to receive image sets from the optical imaging device 1 at two predetermined time points on the fixed target area 210, and the image sets are cross-compared by the image recognition module 2 using a foreign object detection model for identification, thereby generating an identification result. In other words, the image-based real-time foreign object detection method S100 for conveyor belts of the present invention is divided into image capture based on the optical imaging device 1 (step S110) and foreign object identification by the image recognition module 2 (step S120). The details will be described in detail below.

[0036] Figure 3 illustrates the operation of the optical imaging device 1 in the image-based real-time foreign object detection system 100 of some embodiments of the present invention, where the device captures images on the conveyor belt 200. Referring to Figure 3, generally, the material 300 is conveyed continuously for a short period of time, rather than in fragmented manner. As shown in Figure 6, a material 300 will remain in space for a certain length without intersecting or alternating with other material 300s. Furthermore, the length of the foreign object 400 is finite. Considering these two conditions, step S110 is described in detail below.

[0037] Referring to Figure 3, the optical imaging device 1 captures the first image IM1 at the first time point (i.e., time point t), and the optical imaging device 1 captures the second image IM2 at the second time point (i.e., any time point within the time interval (tc) and (t+c)). The first image IM1 and the second image IM2 can be transmitted to the image recognition module 2 for the identification of foreign objects 400. When the total number of identical identifications is greater than a certain threshold, it is determined that time point t is normal (i.e., there are no foreign objects); otherwise, it is determined that there are foreign objects 400. When the following conditions are met, the present invention can achieve 100% detection of foreign objects 400 without false alarms.

[0038] Please refer to Figure 3 again. Assume the speed of the conveyor belt 200 is v (meter / sec), the maximum length of the foreign object 400 is la, the minimum length of the conveyor belt 200 or the material 300 is lc, and the image acquisition time interval is c. When lc > 2la and c > (2la / v), there exists a critical value of 50la / vc, ensuring that the image acquisition in step S110 is correct. In practice, the speed v is fixed, so c > (2la / v) can be ensured by adjusting the image acquisition time interval c. Lc > 2la is related to the characteristics of the conveyor belt 200. Generally speaking, a single material 300 conveying will typically maintain a length of at least tens of meters, while the length of the foreign object 400 usually does not exceed two meters.

[0039] Figure 4 shows the texture image within the foreign object detection model of the image-based real-time foreign object detection system for conveyor belts according to some embodiments of the present invention.

[0040] When humans discover foreign objects 400, they typically do so based on significant differences in the material and texture of the object compared to its surrounding background. For example, metallic objects and minerals have a very different texture from the conveyor belt 200. Texture is defined as an attribute of an object's surface or structure (corresponding to images formed by reflection or penetration), and it is widely used and visually and intuitively obvious. Please refer to Figure 4, which shows images with different textures. Texture images can be big data.

[0041] Based on the logic of human detection of foreign objects 400, step S120 in the image-based real-time foreign object detection method S100 of the present invention uses a foreign object detection model to analyze the texture of the material 300. The input of the foreign object detection model is two images, and the output is whether the two images belong to the same texture.

[0042] Figure 5 shows a schematic diagram of the image recognition module 2 in the image-based real-time foreign object detection system 100 of the conveyor belt according to some embodiments of the present invention, when the foreign object detection model is used to determine that they are different. Figure 6 shows a schematic diagram of the image recognition module 2 in the image-based real-time foreign object detection system 100 of the present invention, when the foreign object detection model is used to determine that they are the same. Referring to Figures 5 and 6, two images at two time points (e.g., t and t+c) on the conveyor belt 200 are input into the foreign object detection model. If the first image IM1 at the first time point (t) belongs to the foreign object 400 and the second image IM2 at the second time point (t+c) belongs to the conveyor belt 200 or the material 300, then the foreign object detection model outputs "different" because of the different textures of the foreign object 400 and the conveyor belt 200 or the material 300, as shown in Figure 5. If the first image IM1 at the first time point (t) and the second image IM2 at the second time point (t+c) belong to the material 300, then the foreign object detection model outputs "same" because of the same texture, as shown in Figure 6.

[0043] If the image recognition module 2 with the foreign object detection model determines that the texture is the same, it is considered normal (i.e., there is no foreign object 400); if it determines that the texture is different, it is considered abnormal (i.e., there is a foreign object 400). This achieves the detection and interpretation effect. Combining the results of the above step S110 and this step S120 can further reduce the false alarm rate and improve the accuracy.

[0044] The results of comparing the conventional technology with the image-based real-time foreign object detection system 100 and image-based real-time foreign object detection method S100 of the present invention are shown in Table 1, which compares the foreign object detection capabilities of the present invention and the conventional technology without collecting customized data. Table 2 compares the recognition rate with a fixed number of false alarms. Table 1 compares the number of false alarms under the premise that the detection rate of the present invention and the conventional technology is 100%. It can be found that the number of false alarms of the present invention is significantly reduced compared with the conventional technology. Table 2 shows that the detection rate obtained by the present invention is better under this condition than that of the conventional technology, which also shows that the present invention can achieve rapid deployment without customized data.

[0045] Table 1: Comparison of foreign object detection capabilities of the present invention and prior art without collecting customized data (fixed detection rate). Detection rate Number of false alarms (in four hours) Learned technology 100% 872 This invention 100% 6

[0046] Table 2: Comparison of foreign object detection capabilities between the present invention and conventional technology without collecting customized data (fixed number of false alarms). Detection rate Number of false alarms (in four hours) Learned technology 6% 0 This invention 88% 0

[0047] Figure 7 shows a block diagram illustrating the structure of an image-based real-time foreign object detection system 100' for conveyor belts according to some other embodiments of the present invention. Referring to Figure 7, the structure of the image-based real-time foreign object detection system 100' for conveyor belts in this embodiment is similar to that shown in Figure 1, except that the image-based real-time foreign object detection system 100' in this embodiment further includes a classification module 3. The classification module 3 is electrically connected to the image recognition module 2.

[0048] Figure 8 shows a schematic diagram of the implementation of the image-based real-time foreign object detection system 100' for conveyor belts shown in Figure 7. The image-based real-time foreign object detection system 100 for conveyor belts shown in Figure 1 compares images from different time points to identify abnormal images with significant differences. However, the detection results are often accompanied by many false positives (i.e., system error warnings). These false positives (i.e., system error warnings) are mainly caused by factors such as the interface between the raw material and the belt, belt stains, and uneven distribution of raw materials, as shown in the lower left box B1 in Figure 8.

[0049] Figure 9 shows a flowchart of the image-based real-time foreign object detection method S100' of the image-based conveyor belt real-time foreign object detection system 100' in Figure 7. Referring to Figures 7 to 9, the image-based real-time foreign object detection method S100' of this embodiment is similar to the image-based real-time foreign object detection method S100 of Figure 2, except that it also includes step S130, that is, classifying the identification result into foreign objects and materials by the classification module 3.

[0050] To reduce the aforementioned false positive results (i.e., system error alerts), the classification module 3 of the image-based real-time foreign object detection system 100' shown in Figure 7 and step S130 of the image-based real-time foreign object detection method S100' shown in Figures 8 and 9 can be used. The main purpose is to confirm the identification results of the image recognition module 2 to filter out false positive results (i.e., system error alerts). The implementation flow of this classification module 3 is shown on the right side of Figure 8. That is, when the image recognition module 2 identifies a group of images with a high degree of abnormality, the classification module 3 is used to judge two (square) images at different positions in this group of images, classifying them as belt, raw material, or foreign object 400, thereby filtering out erroneous detection results where both are raw material or belt.

[0051] Using the aforementioned system and method, the present invention provides an image-based real-time foreign object detection system 100 and method S100 for conveyor belts, comprising an optical imaging device 1 and an image recognition module 2 with a foreign object detection model. The optical imaging device 1 captures real-time images of the scene on the conveyor belt 200 and transmits them to the image recognition module 2. The foreign object detection module in this invention learns from a large dataset of images and, when cross-referencing the images, can identify abnormal areas between them. Therefore, it can identify various types of foreign objects 400, achieving real-time detection and warning of foreign objects 400, thereby reducing the tangible and intangible losses caused by the risk of foreign objects 400 cutting the conveyor belt 200. Furthermore, the abnormal areas detected by the foreign object detection model are input into the classification module 3 to classify the abnormal areas as foreign objects 400 and materials, performing further confirmation and thereby reducing the false alarm rate.

[0052] 100: Real-time Foreign Object Detection System for Conveyor Belts (Image-based) 100': Real-time Foreign Object Detection System for Conveyor Belts (Image-based) 1: Optical photography equipment 2: Image Recognition Module 200: Conveyor belt 210: Fixed Target Area 300: Materials 400: Foreign object 3: Classification Module B1: Box IM1: First Image IM2: Second Image S100: Imaging-based Real-time Foreign Object Detection Method for Conveyor Belts S100': Real-time Foreign Object Detection Method for Image-Based Conveyor Belts S110: Steps S120: Steps S130: Steps

Claims

1. A real-time foreign object detection system for an image-based conveyor belt, comprising: At least one optical imaging device is used to capture an image set at least two predetermined time points of a fixed target area on a conveyor belt; an image recognition module is used to receive the image set from the fixed target area at the two predetermined time points from the optical imaging device, and the image set is cross-compared by the image recognition module using a foreign object detection model to identify it, thereby generating an identification result, and the image recognition module determines whether the texture is the same based on the identification result. If the texture is different, the identification result is considered to have a foreign object; if the texture is the same, the identification result is considered to be normal and there is no foreign object; and a classification module is used to classify the identification result output by the image recognition module into foreign objects and materials, and classifies them into a conveyor belt, a raw material and a foreign object based on two image areas in the identification result.

2. The system as described in claim 1, wherein, This foreign object detection model is based on artificial intelligence and is a model that learns a massive amount of images of various raw materials to form a normal texture feature set through offline pre-training.

3. The system as described in claim 1, wherein, The image set includes a texture.

4. The system as described in claim 1, wherein, The foreign object detection model is a classifier trained by a neural network, which includes at least machine learning, deep learning, and meta-learning.

5. A method for real-time foreign object detection on an image-based conveyor belt, comprising: The method involves: acquiring an image set at two predetermined time points of a fixed target area on a conveyor belt using at least one optical imaging device, the image set including a texture; receiving the image set from the optical imaging device at the two predetermined time points of the fixed target area using an image recognition module, and cross-referencing the image set using a foreign object detection model to identify a foreign object; classifying the foreign object identification result into foreign objects and non-foreign objects using a classification module to filter out false positives, and classifying the two image regions in the foreign object identification result into a conveyor belt, a raw material, and a foreign object. The two predetermined time points include a first time point and a second time point. A first image and a second image are captured at the first time point and the second time point respectively through the optical imaging equipment. The first image and the second image are transmitted to the image recognition module. The foreign object detection model, which is pre-trained offline to identify the normal texture features of different raw materials, compares and analyzes whether the textures of the first image and the second image are the same. If they are different textures, it is considered that there is a foreign object. If they are the same texture, it is considered normal and there is no foreign object.

6. The method as described in claim 5, wherein, The image recognition module is electrically connected to these optical photography devices.

7. The method as described in claim 5, wherein, The foreign object detection model is based on artificial intelligence and is a model that is pre-trained offline to identify different material textures.

8. The method as described in claim 5, wherein, The foreign object detection model is a classifier trained by a neural network, which includes at least machine learning, deep learning, and meta-learning.

Images