Transport state detection device and transport state detection method

Abstract

The conveyance state of the material on the belt can be automatically detected. A conveyance state detection device (21) detects the conveyance state of the material (2) on each belt (3) of a plurality of belt conveyors (1) that convey the material (2), and includes an imaging device that captures an image of the belt (3) that conveys the material from above, and a conveyance state detection unit (10) that detects the conveyance state of the material (2) on the belt (3) by processing the image captured by the imaging device.

Description

[Technical field]

[0001] The present invention relates to a technology for automatically detecting, by image processing, the transport state of an object on each of a plurality of belt conveyors that transport objects. [Background technology]

[0002] The raw materials (transported items) stored in the raw materials yard are transported to the next process, such as sintering, coking, or blast furnace plants, depending on the steelworks' production. For this reason, the raw materials transporting capacity of the raw materials yard has a significant impact on the steelworks' production. Numerous belt conveyors are installed in the raw materials yard to transport the raw materials. They transport raw materials of different types and brands. For this reason, various raw materials are transported to their destinations by changing between multiple belt conveyors at transfer sections that use drop chutes or the like.

[0003] At this time, when the raw materials are loaded onto the belt at the transfer point of the belt conveyor, the raw materials may be loaded to one side in the width direction of the belt. In this case, falling of the raw materials may occur during transportation. Falling of the raw materials refers to the falling of the raw materials from the belt. In particular, the risk of falling increases as the amount of raw materials transported increases. The amount of raw materials transported refers to the amount loaded onto the belt. Falling of the raw materials also increases mechanical troubles and cleaning costs in the belt conveyor and surrounding facilities.

[0004] Conventionally, an operator would view images captured by a camera on a screen to monitor the state of the raw material being transported on the belt. However, it is difficult to constantly monitor the state of transport by a belt conveyor with the human eye. For this reason, conventionally, in order to avoid the risk of falling ore, the amount of raw material loaded on the belt was operated on the safe side. In other words, the belt conveyor was operated with a large margin of raw material transport capacity relative to the belt width. This is one of the factors that limits the raw material transport capacity in the raw material yard. For this reason, in recent years, there has been a demand for a method for collectively monitoring the transport states of a plurality of belt conveyors without relying on human visual monitoring.

[0005] Here, as a technique for monitoring an abnormality in a belt of a belt conveyor, there is a technique described in, for example, Patent Documents 1 and 2. In Patent Document 1, in order to manage a belt conveyor that transports goods, images of the belt are continuously acquired. Furthermore, in Patent Document 1, the position of the belt is determined from the positional relationship of the continuously acquired images. Patent Document 1 also discloses that the risk of an abnormality occurring in the belt conveyor is determined from the determined positional relationship of the belt. In addition, in Patent Document 2, image data of the belt width direction end region is input to a deterioration level determination model to determine the deterioration level of the belt width direction end region, and Patent Document 2 discloses that information on the determined belt position and information on the deterioration level for that belt position are output. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-18999 A [Patent Document 2] Patent Publication No. 2021-17296 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the methods for monitoring belt abnormalities described in Patent Documents 1 and 2 can monitor abnormal conditions in the belt itself, such as meandering of the running belt, but the methods described in Patent Documents 1 and 2 cannot directly detect abnormalities in the transported goods on the belt, such as the load (transport amount) of the transported goods being too small.

[0008] The present invention has been made in view of the above-mentioned points, and has an object to make it possible to automatically detect the transport state of an object on a belt. [Means for solving the problem]

[0009] In order to solve the problem, one aspect of the present invention is a conveying condition detection device that detects the conveying condition of an object on each of a plurality of belt conveyors that convey the object, and includes an imaging device that images the belt of each conveyor from above, and a conveying condition detection unit that detects the conveying condition of the object on the belt for each of the belts by image processing the image captured by the imaging device.

[0010] Another aspect of the present invention is a conveying state detection method for detecting the conveying state of an object conveyed by the belt of each of a plurality of belt conveyors that convey the object, in which the belt of each belt conveyor is imaged from above and the captured image is processed to determine the conveying state of the object on the belt for each of the belts. Effect of the Invention

[0011] According to an aspect of the present invention, for example, in a facility in which an object is transported by a plurality of belt conveyors, an image of the belt of each belt conveyor is image-processed. As a result, according to the aspect of the present invention, it becomes possible to automatically detect the transport state of the object on the belt of each belt conveyor individually. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing an example of the configuration of a belt conveyor and a transport state detection device according to a first embodiment of the present invention. [Diagram 2] FIG. 4 is a view of the upper belt as seen from the belt running direction. [Diagram 3] FIG. 4 illustrates an example of the configuration of a transport state detection unit. [Figure 4] 1A and 1B are diagrams showing examples of captured images according to a first embodiment of the present invention, in which (a) shows an image captured by an imaging unit (camera), and (b) shows an image obtained by cutting out a partial area from the image captured by the imaging unit (camera). [Diagram 5] FIG. 1 is a diagram showing an image of the input ((a)) and output ((b)) of a trained individual model. [Figure 6] 13 is a flowchart showing an example of processing in a transport state detection unit and a determination information output unit. FIG. [Figure 7] 10 is a schematic diagram showing an example of the configuration of a belt conveyor and a transport state detection device according to a second embodiment of the present invention. FIG. [Figure 8] 10A and 10B are diagrams showing examples of captured images according to a second embodiment of the present invention, in which (a) shows an image captured by an imaging unit (camera), and (b) shows an image obtained by cutting out a partial area from an image captured by the imaging unit (camera). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] "First embodiment" Next, a first embodiment of the present invention will be described with reference to the drawings. Here, multiple belt conveyors are installed in the raw material yard. Raw materials are transported to the raw material yard by the multiple belt conveyors and stored as raw material piles. The raw materials are also supplied to the next process by the belt conveyors as needed. In the raw material yard, raw materials of different types and brands are transported by changing between multiple belt conveyors at a transfer section that uses a drop-type chute or the like.

[0014] In this embodiment, the present disclosure will be described assuming application to a plurality of belt conveyors used in a raw material yard. However, the present disclosure can also be applied to a conveying facility having a plurality of belt conveyors for conveying other conveyed objects. In the present embodiment, the material to be transported by the belt is described as being in the form of loose raw materials. The material to be transported is not limited to being in the form of loose raw materials. In the following description, the transported goods will also be referred to as raw materials. Here, the present disclosure is a technology suitable for multiple belt conveyors that convey objects made of the same type of material. The destinations of the objects on each belt conveyor may be different. However, the present disclosure is a technology suitable for detecting the conveying state on the belts of multiple belt conveyors in a conveying facility where the objects are conveyed by transferring between multiple belt conveyors.

[0015] (Overall composition) The present embodiment includes a conveying facility and a conveying status detection device. The conveying facility includes a plurality of belt conveyors for conveying objects. The conveying status detection device is a device for detecting the conveying status of the objects on the belts of each of the belt conveyors. In this embodiment, a case where a plurality of belt conveyors used in the above-mentioned raw material yard are used will be described as an example. Also, this embodiment is a case where a conveying state detection device 21 is provided for each belt conveyor.

[0016] FIG. 1 is a schematic diagram showing one belt conveyor 1 of the present embodiment and an individual conveyance state detection device 21 corresponding to the one belt conveyor 1. As shown in FIG. Here, a plurality of belt conveyors are installed in the raw material yard. Generally, however, each belt conveyor that transports raw materials has the same configuration. However, in this embodiment, each belt conveyor may have a different configuration. That is, the belt conveyor 1 shown below is an example, and a belt conveyor having a different configuration may also be used. Generally, in a raw material yard, the basic configuration of each belt conveyor and the conveying state detection device corresponding to that belt conveyor are similar to each other, so the following will only explain one belt conveyor 1 and its corresponding conveying state detection device 21.

[0017] <Conveyor Belt 1> The belt conveyor 1 includes an endless belt 3 and a pulley 7. The belt 3 runs on a plurality of rollers 8 with a predetermined tension due to the rotation of the pulley 7, which is composed of a driving pulley 7a and a plurality of driven pulleys 7b. The driving pulley 7a is rotationally driven by an electric motor 5 via a reduction gear 6. In this embodiment, the rollers 8 that support the upward belt 3a are configured as a combination of a central roller 8a and left and right rollers 8b, as shown in FIG. 2. The central roller 8a is disposed at the center in the belt width direction (left and right direction in FIG. 2). The left and right rollers 8b are disposed on the left and right of the central roller 8a, and are rollers with inclined axes. This allows the rollers 8 to support the upward belt 3a in a trapezoidal state. The roller 8 that supports the downward belt 3b side is configured as a single roller. However, the support configuration of the belt 3 by the rollers 8 is not limited to this.

[0018] In this embodiment, a raw material input device 20 consisting of a chute or the like is located upstream in the conveying direction of the upward belt 3a. Raw materials 2 are loaded onto the belt 3 from the raw material input device 20. The raw materials 2 are then conveyed by the running belt 3. The raw materials 2 are then conveyed downstream in the conveying direction by the upward belt 3a, and are sequentially input into the chute 4.

[0019] <Transportation state detection device 21> The conveyance state detection device 21 is a device that detects the conveyance state of the raw material 2 loaded on the belt 3 of the corresponding belt conveyor 1. The conveyance state of the raw material 2 is, for example, the placement state of the raw material 2 on the running belt 3. 1, the transport state detection device 21 includes an imaging device 9 and a transport state detection section 10. Reference numeral 11 denotes a database serving as an image storage device for storing captured images.

[0020] <Imaging device 9> The imaging device 9 of this embodiment is composed of a visible camera. The imaging device 9 captures an image of the upward moving belt 3a of the belt 3 of the belt conveyor 1 from above to obtain a captured image. Fig. 4 is a schematic diagram showing an example of a captured image. The imaging device 9 is adjusted and installed so that at least both ends of the belt 3 in the width direction are captured. In the example of Fig. 1, the imaging axis of the imaging device 9 is installed at an angle to the running direction of the belt 3. The installation location of the imaging device 9 must ensure a sufficient field of view to monitor the belt 3. Also, Fig. 1 illustrates a case where one imaging device 9 is provided for one belt conveyor 1. Two or more imaging devices 9 may be provided along the extension direction (travel direction) of the upward belt 3a. In this case, falling ore or the like can be detected on the way based on the difference in the conveying state at two points along the conveying direction.

[0021] <Transportation state detection unit 10> As shown in FIG. 3, the transport state detection unit 10 includes an image processing unit 10A, a state determination unit 10B, and a determination information output unit 10C.

[0022] <Image processing unit 10A> The image processing unit 10A performs image processing on the captured image captured by the imaging device 9. The image processing unit 10A then acquires at least the transported item area information out of the transported item area information and the belt area information. The transported item area information is transported item area information that is information on the area of ​​the raw materials 2 on the belt 3. The belt area information is information on the area of ​​the belt 3 on which the raw materials 2 are not placed. The part of the belt 3 on which the raw materials 2 are not placed is the part where the upper surface of the belt 3 is exposed. The transported goods area information may be, for example, position information of the raw materials 2 placed thereon. The image processing unit 10A may be configured to perform binarization or other enhancement processing on the acquired image as the image processing. In this case, the image processing unit 10A detects the width direction end position of the belt 3 in the image and the outer circumferential contour position of the raw material 2 placed on the belt 3. Then, the belt area information and the transported object area information are obtained from the detected positions.

[0023] The belt conveyor 1 in the raw material yard is often placed outdoors. In this case, the surrounding environment may affect the accuracy of detecting the width direction end positions of the belt 3 in the captured image, depending on the enhancement process applied to the image. Here, the surrounding environment refers to the season, weather, etc. Furthermore, the enhancement process may affect the accuracy of detecting the outer circumferential contour position of the raw material 2 placed on the belt 3. For this reason, in this embodiment, a trained model 12 is generated in advance by machine learning such as deep learning using supervised data (see FIG. 3). The trained model 12 is a model that receives an image of the belt 3 of one target belt conveyor 1 as an input, and outputs transported object region information of the region where the raw materials 2 placed on the belt 3 are placed. The image processing unit 10A of this embodiment receives a captured image as input data and acquires transported object region information using the trained model 12 described above. A known learning method may be applied to machine learning for obtaining the trained model 12. An example of the machine learning for obtaining the trained model 12 is semantic segmentation.

[0024] Consider the case where semantic segmentation is adopted. In this case, teacher image data is created by labeling each pixel constituting a previously captured image for learning into one of three area classes. The three areas are the area of ​​the raw material 2 on the belt 3, the area of ​​the belt 3 excluding the raw material position, and the other area. In other words, the image is divided into the area of ​​the raw material 2 on the belt 3, the area of ​​the belt 3, and the other area. Then, the teacher image data is created by labeling each pixel in the image into which area (class) it is classified. Then, a large number of teacher image data are trained by a convolutional neural network to generate a trained model 12. The trained model 12 receives a captured image (input data) as input and outputs information about at least the area of ​​the raw material 2 in the captured image.

[0025] Here, it is advisable to prepare multiple sets of teacher image data, each captured by the imaging device 9 in different time periods and under different weather conditions, and generate a model using the prepared multiple sets of teacher image data. In this case, even if the image of the belt conveyor 1 is captured outdoors, the area of ​​the raw material 2 can be detected stably and accurately. As a result, automatic monitoring of the conveying state of the raw material 2 becomes possible with high accuracy. The output information of the trained model 12 is, for example, data that includes information on which area in the captured image is the area of ​​the raw material 2 and which area is the area of ​​the belt 3. Note that the area of ​​the raw material 2 may be composed of one area or may be divided into multiple areas.

[0026] When the trained model 12 using semantic segmentation is used, the output is as follows. That is, image data is output in which identification information (transported goods area information) indicating that the area is raw material 2 is assigned to each pixel in the area of ​​raw material 2 in the output image. That is, in the image data that is the output value of the trained model 12 using semantic segmentation, identification information is assigned to each pixel. The assigned identification information is information on whether the pixel is raw material 2, belt 3, or other. This makes it possible to identify which area is raw material 2 and which area is belt 3. For example, the trained model 12 outputs image data in which the pixel color is changed for each class. Then, the image processing unit 10A uses the trained model 12 described above to input the captured image captured by the imaging device 9, and outputs image data including transported object region information.

[0027] Here, the captured image input to the trained model 12 does not need to be the image captured by the imaging device 9 itself. Consider the case where the imaging device 9 is set to capture a predetermined length in the longitudinal direction (traveling direction) of the belt 3 as shown in FIG. 4(a). In this case, an image of a region as shown in FIG. 4(b), which is a part of the longitudinal direction cut out from the captured image, may be input to the trained model 12. The region to be cut out is only a specific region where the conveying state of the image data needs to be detected. The specific region is, for example, a region where the state of the raw material 2 is stable, or conversely, a region where the belt 3 is likely to meander in the width direction. Two or more regions may be cut out. In this case, image processing may be performed for each cut out region. Here, the reference symbol 3a refers to an exposed portion of the belt 3 on which the raw material 2 is not placed. The same applies to FIG. 5(a).

[0028] Note that images captured by the imaging device 9 and cut-out images are sequentially stored in the database 11. Then, an update process of the trained model 12 may be appropriately performed using the stored images. In this embodiment, a conveying state detection device 21 is provided for each belt conveyor 1. Therefore, a trained model 12 is generated for each belt conveyor. Then, the trained model 12 set for each belt conveyor is used to acquire conveyed object region information.

[0029] <State Determination Unit 10B> The state determining unit 10B determines the conveying state of the raw material 2 based on the information acquired by the image processing unit 10A. Here, the conveying state of the raw material 2 is, for example, information on the placement state of the raw material 2 placed on the belt 3. Furthermore, the conveying state of the raw material 2 is information on the presence or absence of the raw material 2 on the belt 3, information on the amount of the raw material 2 conveyed by the belt 3, and information on the width direction position of the raw material 2 present on the belt 3. When belt area information is also detected, the state of the relative position of the raw material 2 with respect to the belt 3 can also be identified as the conveying state. Also, the conveying state may be determined indirectly from the belt area information. The state determination unit 10B of this embodiment determines the conveying state of the raw material 2 from the information obtained by the image processing unit 10A. The state determination unit 10B determines the conveying state of the raw material 2 from an image to which conveyed object area information and belt area information output by the trained model 12 are added, for example.

[0030] <Transport amount calculation unit 10Ba> The state determination unit 10B of this embodiment includes a transport amount calculation unit 10Ba. The conveying amount calculation unit 10Ba calculates the area of ​​the raw material 2 present on the belt 3 when viewed from above. The conveying amount calculation unit 10Ba calculates the amount of raw material 2 conveyed by the belt 3 per unit length of the belt 3 from the calculated area of ​​the raw material 2. The conveying amount constitutes an example of a conveying state.

[0031] The conveyance amount calculation unit 10Ba, for example, calculates the number of pixels of the raw material 2 in the image data output by the trained model 12. The number of pixels of the raw material 2 corresponds to the area occupied by the raw material 2. The conveyance amount calculation unit 10Ba then calculates the area occupied by the raw material 2 by multiplying the coefficient corresponding to the size of the pixel. The conveyance amount calculation unit 10Ba then calculates the conveyance amount per unit length of the raw material 2 loaded on the belt 3 from the calculated area occupied by the raw material 2. The conveyance amount can be calculated from the area of ​​the raw material 2. For example, a formula for calculating the theoretical conveyance amount of a trough-type belt conveyor is used. The formula is publicly known, as described in "Mashima Utaro: Conveyor Calculation Method, Kogaku Tosho Co., Ltd., Tokyo, (1979), p. 14" and the like. In this case, the volume of the raw material 2, that is, the conveyance amount, can be calculated from the number of pixels (area) of the raw material 2 calculated from the specified pixels. At this time, the speed and density of the belt 3 are applied from the operating conditions.

[0032] <Judgment information output section 10C> The determination information output unit 10C outputs information such as the determination information obtained by the state determination unit 10B, the transport amount obtained by the transport amount calculation unit 10Ba, and the transport state determined from such information. The determination information output unit 10C notifies the operator of the determination information, for example, by outputting the information. An example of a process flow of the state determination unit 10B and the determination information output unit 10C will be described with reference to Fig. 6. This process is repeatedly executed every time an output from the image processing unit 10A is input, for example. First, the state determination unit 10B receives output data (image data) of the trained model 12 in step S10.

[0033] In this embodiment, the trained model 12 inputs a captured image as shown in FIG. 5(a), and outputs image data as shown in FIG. 5(b). The image of the image data shown in FIG. 5(b) shows the region R1 of the raw material 2, the region R2 of the belt 3, and the other region R3 in different displays. As described above, identification information is assigned to each pixel. Here, the identification information of the pixel may be changed for each type of raw material 2. In FIG. 5(b), R1 indicates the region of the raw material 2, R2 indicates the region of the belt 3 where the raw material 2 is not present, and R3 indicates the other regions. The same identification information is assigned to pixels in the same region.

[0034] Next, in step S20, the pixels of the raw material 2 (transported object) in the image data acquired in step S10 are identified, and the number of pixels of the raw material 2 is calculated. The number of pixels is proportional to the area S occupied by the raw material 2. The calculated number of pixels is multiplied by a coefficient that converts the size of one pixel into the actual size on the belt 3. In this way, the area S occupied by the raw material 2 is calculated. Next, in step S30, it is determined whether the number of pixels of the raw material 2 obtained in step S20 is equal to or greater than a preset threshold value A. Instead of the number of pixels of the raw material 2, it may be determined whether the area S occupied by the raw material 2 is equal to or greater than a preset threshold value. If it is equal to or greater than the threshold value A, the process proceeds to step S50, and if it is less than the threshold value A, the process proceeds to step S40. The threshold value A may be set based on the operating conditions.

[0035] In step S40, it is determined that "there is no raw material 2 on the belt 3," and the determination result is output and the process ends. The determination result is notified, for example, by emitting an alarm sound, sending an email, displaying on a screen, etc. This allows the operator to know that there is no raw material 2 on the belt 3. If there is raw material 2 on the belt 3, the process proceeds to step S50. Then, the transport amount V of the raw material 2 within the cut-out image area is calculated. The transport amount V can be calculated from the area S of the raw material 2 by a known method. Then, the calculated transport amount V is output to the display unit.

[0036] Next, in step S60, it is determined whether the transport amount V calculated in step S50 is within a preset allowable range (V1 or more and V2 or less). If it is within the allowable range, the process ends. If it is outside the allowable range, the process proceeds to step S70. It may be possible to determine whether or not the size of the exposed portion of the belt in the belt width direction is within the allowable range from the belt region information. The conveyance amount is outside the allowable range if it is too large or too small. If the conveyance amount is too large, (V>V2). If it is too small, (V <V1)となっている。 That is, the appropriate range of the transport amount (lower limit value V1 and upper limit value V2) is determined according to the transport conditions, and this range is set as the allowable range.

[0037] In step S70, the result of the abnormality determination is output. The abnormality determination is notified, for example, by emitting an alarm sound, sending an email, displaying on a screen, etc. The operator can adjust the increase or decrease of the amount of raw material 2 conveyed by the belt 3 based on the output determination result and the conveyance amount. In addition, the presence or absence of ore falling from Belt 3 will be estimated. Furthermore, the operator may adjust the amount of raw material 2 loaded onto the belt 3 based on the history of the transport amount that is continuously acquired. Here, when the state determination unit 10B determines that the raw material 2 is loaded on the belt 3, it may execute a process of detecting the position of the raw material 2 with respect to the belt 3 as a conveying state.

[0038] For example, the positions of both ends of belt 3 in the width direction or the center position of belt 3 in the width direction are obtained from the pixels representing the area of ​​belt 3. Furthermore, the center of gravity of the area of ​​raw material 2 is obtained from the positions of the pixels of raw material 2. Then, from the deviation in the belt width direction of the center of gravity of the area of ​​raw material 2 from the center of belt 3 in the width direction, the offset in the belt width direction of raw material 2 loaded on belt 3 is detected. Also, for example, the distance in the belt width direction between the ends of belt 3 in the width direction and the outline of the area of ​​raw material 2 may be obtained as the conveying state of raw material 2.

[0039] The distance in the belt width direction between the end of the belt 3 in the width direction and the outline of the area of ​​the raw material 2 serves as a criterion for determining the limit of the amount of raw material 2 that can be transported loaded on the belt 3. Here, this embodiment is an example in which a conveying state detection device 21 is provided for each belt conveyor 1. In this case, multiple locations in the longitudinal direction on the belt 3 of one belt conveyor 1 may be locations for detecting the conveying state individually. In this case, each location may have the trained model 12 described above, or each belt conveyor 1 may have a trained model 12 common to the multiple locations.

[0040] (Operation etc.) In this embodiment, a visible camera (imaging device 9) is installed for each belt conveyor at a position where the belt 3 of the belt conveyor 1 can be monitored. The camera constantly monitors the belt 3 that transports the raw materials 2 (transported goods). In this embodiment, the images captured by the camera are processed, making it possible to automatically detect the transport state of the raw materials 2 on the belt 3 for each belt conveyor. Since this embodiment uses automatic detection, it becomes possible to efficiently monitor the transport states of multiple belt conveyors 1 all at once without relying on human eyes. At this time, the operator can grasp the presence or absence of raw materials 2 on the belt 3, the width and conveying amount of raw materials 2 on the belt 3, etc. Then, by appropriately adjusting the conveying amount of raw materials 2 by each belt conveyor 1 to an appropriate value, it is possible to prevent ores from falling off the belt 3. Also, efficient measures can be taken before equipment trouble occurs. Therefore, in this embodiment, it is possible to reduce equipment troubles caused by ores falling and cleaning time and cleaning costs.

[0041] In addition, it is possible to efficiently detect whether the proper maximum amount of raw materials is being transported without dropping. Therefore, by maximizing the transport amount for the equipment specifications of the belt conveyor 1, the transport amount is suppressed, which leads to a reduction in demurrage charges. Furthermore, in this embodiment, the conveyance state is detected for each belt conveyor using an individual trained model 12. Therefore, even if the present disclosure is applied to an outdoor belt conveyor 1, it is possible to detect the conveyance state with high accuracy.

[0042] In this way, the present disclosure is useful for monitoring a raw material yard that transports and stores raw materials 2 such as iron ore and coal. The present disclosure is a technique particularly suitable for detecting uneven distribution of raw materials 2. It should be noted that the present disclosure does not indirectly detect the conveyance state of the raw material 2 conveyed by the belt 3 from the state of the belt 3 itself as in the conventional case. The present disclosure detects the conveyance state, which is the state of the raw material 2 on the belt 3. And, from the detection of the conveyance state, it is possible to determine conveyance abnormalities of the belt 3 in some cases.

[0043] "Second embodiment" Next, a second embodiment of the present invention will be described. The same components as those in the first embodiment are denoted by the same reference numerals, and detailed description of the components will be omitted. In the first embodiment, a case where a conveying state detection device 21 is provided for each belt conveyor 1 is exemplified. In the first embodiment, a trained model 12 is generated for each belt conveyor. In the first embodiment, the trained model 12 generated for each belt conveyor is used to acquire conveyed object area information on the belt of each belt conveyor. In contrast, in this embodiment, a trained model common to multiple belt conveyors is generated. Then, in this embodiment, the common trained model is used to acquire transported object region information. This point differs from the first embodiment. The other configurations may be configured similarly to the first embodiment.

[0044] In this embodiment, for example, as shown in Fig. 7, an image of each belt 3 of a plurality of belt conveyors 1 is acquired by each imaging device 9. As shown in Fig. 7, an imaging device 9 may be installed for each belt 3, or one imaging device 9 may be installed for a plurality of belts 3. The imaging device 9 captures an image of each belt 3 (upward belt 3a) of the corresponding belt conveyor from above, and acquires the image. The imaging device 9 supplies the acquired image to the conveyance state detection unit 10.

[0045] Then, similarly to the first embodiment, the transport state detection unit 10 performs processing in the image processing unit 10A to input the captured images captured by each imaging device 9 and output image data including transported object region information, using the common trained model 12. The common trained model 12 may be generated using semantic segmentation or the like.

[0046] <About trained models> In the first embodiment, the teacher image data of the belt 3 of each belt conveyor 1 is used to generate an individual trained model 12 for each belt conveyor. In contrast, in this embodiment, multiple teacher image data for multiple belts 3 captured on multiple belt conveyors are combined and trained using a convolutional neural network to generate a trained model 12 common to multiple belt conveyors.

[0047] Here, the captured images of the belt 3 captured by the multiple imaging devices 9 each often have a different field of view. For this reason, if the captured images are used as teacher image data to learn and generate the trained model 12, the detection accuracy of the conveyance state may be reduced. Also, if the captured images are input to the trained model 12 as they are, the detection accuracy of the conveyance state may be reduced. For this reason, as shown in Fig. 8, it is preferable to perform a process (normalization process) for aligning the angles of view for each of the captured images of the multiple belts 3 captured by the multiple imaging devices 9. Projective transformation process is an example of the process for aligning the angles of view. By performing projective transformation process for each captured image, captured images with different fields of view (imaging angles) can be aligned to the same angle of view (imaging angle). Note that when one imaging device 9 is installed for multiple belts 3, that is, when multiple belts 3 exist in one captured image, normalization process (projective transformation process) is performed individually for each belt 3 in the captured image.

[0048] In this manner, in this embodiment, for captured images of multiple belts 3, one common trained model 12 is generated using captured images with the same angle of view. Then, the normalized captured images are input to this common trained model 12. This makes it possible to accurately detect the conveying state of each belt 3 of multiple belt conveyors 1 using one common trained model 12. That is, the conveying states of multiple belt conveyors 1 can be monitored using one conveying state detection unit 10. Therefore, monitoring efficiency can be improved compared to a configuration in which a conveying state detection unit 10 is provided for each belt conveyor. In addition, a plurality of belt conveyors may be divided into a plurality of groups, and each group may have one common trained model 12 and one conveying state detection unit 10. In this case, instead of providing a common trained model 12 for each group, a single common trained model 12 may be used for all the groups.

[0049] In addition, the above common trained model 12 may be generated for each of two or more belt conveyors selected from the multiple target belt conveyors. Other configurations and operations are similar to those of the first embodiment.

[0050] (others) The present disclosure may also have the following configuration. (1) A conveyance state detection device for detecting a conveyance state of an object on a belt of each of a plurality of belt conveyors for conveying an object, comprising: An imaging device that images the belt of each belt conveyor from above; a conveyance state detection unit that detects a conveyance state of an object on each of the belts by performing image processing on an image captured by the imaging device; A transport state detection device comprising: (2) The transport state detection unit detects, for each of the belts, at least one of information regarding the presence or absence of an object on the belt and information regarding the amount of the object being transported by the belt, as the transport state of the object. (3) the conveying state detection unit is an image processing unit that acquires conveyed object area information, which is information about an area of ​​the conveyed object present on each belt, by image processing for each of the belts; a state determination unit that determines the conveying state, which is the state of the conveyed object loaded on the belt, based on the acquired conveyed object area information; Equipped with. (4) The image processing unit acquires the transported object area information from the image captured by the imaging device using a trained model generated by learning, with a convolutional neural network, teacher image data in which each pixel of the previously captured image is labeled to indicate which class the pixel falls into (the area of ​​the transported object on the belt, the area of ​​the belt, or other areas). (5) The trained model is provided separately for each of the conveyor belts. (6) The trained model is a trained model generated by learning using the teacher image data of the multiple belt conveyors, and the trained model is a trained model common to the multiple belt conveyors. (7) The image processing unit performs a projective transformation process on the captured image captured by the imaging device, and acquires the transported object region information from the captured image after the projective transformation. (8) The transport state detection unit includes a transport amount calculation unit that calculates an area of ​​the transported object present on the belt for each of the belts and determines the transport amount of the transported object from the calculated area of ​​the transported object as transport state information. (9) The transport state detection unit includes a determination information output unit that determines whether the transport amount calculated by the transport amount calculation unit is within a preset allowable range, and outputs the determination result when it determines that the transport amount is outside the allowable range. (10) A conveyance state detection method for detecting a conveyance state of an object conveyed by a belt of each of a plurality of belt conveyors for conveying the object, comprising the steps of: The belt of each belt conveyor is imaged from above, and the captured image is processed to determine the transport state of the article on the belt for each belt. (11) acquiring, by the image processing, transported object area information which is information regarding an area of ​​the transported object present on the belt; Based on the acquired transported object area information, the transport state, which is the state of the transported object loaded on the belt, is determined for each belt.

[0051] The entire contents of Japanese Patent Application No. 2023-135730 (filed on August 23, 2023), from which this application claims priority, are incorporated herein by reference. Although the present application has been described with reference to a limited number of embodiments, the scope of the rights is not limited thereto, and modifications of each embodiment based on the above disclosure would be obvious to one skilled in the art. [Explanation of symbols]

[0052] 1. Conveyor belt 2 Raw materials (example of transported goods) 3 Belt 9. Imaging device 10. Transport state detection unit 10A Image processing unit 10B Status determination section 10Ba Transport amount calculation unit 10C Judgment information output section 12 Pre-trained models 21 Transport status detection device

Claims

1. A conveyance state detection device for detecting a conveyance state of an object on a belt of each of a plurality of belt conveyors for conveying an object, comprising: An imaging device that images the belt of each belt conveyor from above; a conveyance state detection unit that detects a conveyance state of an object on each of the belts by performing image processing on an image captured by the imaging device; Equipped with the conveying state detection unit is an image processing unit that acquires conveyed object area information, which is information regarding an area of ​​the conveyed object present on each belt, by image processing for each of the belts; a state determination unit that determines the conveying state, which is the state of the conveyed object loaded on the belt, based on the acquired conveyed object area information; Equipped with The image processing unit acquires the transported object region information, including identification information indicating that the region is a transported object region on a belt, which is assigned to each pixel, from the image captured by the imaging device, using a trained model generated by training a convolutional neural network on teacher image data in which each pixel of the captured image captured in advance is labeled as to which region the image is classified into, i.e., the region of the transported object on the belt, the region of the belt, or other regions; the conveyance state detection unit includes a conveyance amount calculation unit that calculates an area of ​​the conveyed object present on the belt in a top view for each of the belts, and obtains a conveyance amount of the conveyed object from the calculated area of ​​the conveyed object as conveyance state information. Transport status detection device.

2. the conveyance state detection unit detects, for each of the belts, at least one of information on the presence or absence of an object on the belt and information on the conveyance amount of the object by the belt as the conveyance state of the object; 2. A conveying state detection device according to claim 1.

3. The trained model is provided for each of the belt conveyors.

2. A conveying state detection device according to claim 1.

4. The trained model is a trained model generated by learning using the teacher image data of the plurality of belt conveyors, and the trained model is a trained model common to the plurality of belt conveyors.

2. A conveying state detection device according to claim 1.

5. The image processing unit performs a projective transformation process on the captured image captured by the imaging device, and acquires the transported object region information from the captured image after the projective transformation. The transport state detection device according to any one of claims 1 to 4.

6. the transport state detection unit includes a determination information output unit that determines whether the transport amount calculated by the transport amount calculation unit is within a preset allowable range, and outputs a determination result when the transport amount is determined to be outside the allowable range. The transport state detection device according to any one of claims 1 to 4.

7. A conveyance state detection method for detecting a conveyance state of an object conveyed by a belt of each of a plurality of belt conveyors for conveying an object, comprising: an image of each belt conveyor is captured from above, and the captured image is processed for each belt to obtain transported object area information, which is information regarding the area of ​​the transported object present on each belt; A conveying state, which is a state of the conveyed object loaded on the belt, is determined based on the acquired conveyed object area information; The image processing acquires the transported object region information from the captured image, including identification information assigned to each pixel indicating that the pixel is a transported object region on the belt, the belt region, or other regions, using a trained model generated by training the captured image using teacher image data in which each pixel of the captured image is labeled to indicate which pixel is classified as a region of the transported object on the belt, the belt region, or other regions; The determination of the conveying state includes calculating an area of ​​the belt in a top view of the conveying object, for each of the belts, and determining a conveying amount of the conveying object from the calculated area of ​​the conveying object as conveying state information. A method for detecting the transport state.

8. By the image processing, conveyed object area information is obtained, which is information regarding an area of ​​the conveyed object present on the belt; determining, for each of the belts, the conveying state, which is the state of the conveyed objects loaded on the belt, based on the acquired conveyed object area information; 8. The method for detecting a conveying state according to claim 7.

Images