Conveyance state detection device and conveyance state detection method

Abstract

The present invention makes it possible to automatically detect a conveyance state of an article to be conveyed on a belt. Provided is a conveyance state detection device (21) that detects conveyance states of a raw material (2) on belts (3) of a plurality of belt conveyors (1) which convey the raw material (2). The conveyance state detection device (21) comprises: an imaging device that images, from above, the belts (3) which convey articles to be conveyed; and a conveyance state detection unit (10) that carries out image processing on the image captured by the imaging device to detect conveyance states of the raw material (2) on the belts (3).

Description

Transport state detection device and transport state detection method

[0001] The present invention relates to a technology for automatically detecting, by image processing, the transport state of an object on each of a plurality of belt conveyors that transport the object.

[0002] The raw materials (transported items) stored in the raw material yard are transported to the next process, such as sintering, coking, or blast furnace, depending on the steelworks' production. For this reason, the raw material transport capacity of the raw material yard has a significant impact on the steelworks' production. Numerous belt conveyors are installed within the raw material yard to transport the raw materials. Different types and brands of raw materials are transported. For this reason, various types of raw materials are transported to their destinations by changing between multiple belt conveyors at transfer points using drop chutes or other devices.

[0003] At this time, when loading raw materials onto the belt at the transfer point of the belt conveyor, the raw materials may be unevenly distributed across the width of the belt. In this case, falling of raw materials may occur during transport. Falling of raw materials refers to the falling of raw materials from the belt. In particular, the risk of falling increases as the amount of raw materials being transported increases. The amount of raw materials being transported refers to the amount loaded onto the belt. Falling of raw materials also increases mechanical troubles and cleaning costs for the belt conveyor and surrounding equipment.

[0004] Conventionally, an operator would view images captured by a camera on a screen to monitor the transport status of raw materials on the belt. However, it is difficult to constantly monitor the transport status of a belt conveyor by human eyes. For this reason, in the past, to avoid the risk of falling ore, the amount of raw materials loaded on the belt was operated on the safe side. In other words, the belt conveyor was operated with a large margin of raw material transport capacity relative to the belt width. This is one of the factors that limits the raw material transport capacity in raw material yards. For this reason, in recent years, there has been a demand for a system that can monitor the transport status of multiple belt conveyors simultaneously without relying on human eyes.

[0005] Here, there are technologies described in, for example, Patent Documents 1 and 2 as technologies for monitoring belt abnormalities in a belt conveyor. In Patent Document 1, images of the belt are continuously acquired to manage the belt conveyor that transports goods. Furthermore, Patent Document 1 determines the position of the belt from the positional relationship of a group of continuously acquired images. Patent Document 1 then discloses determining the risk of belt conveyor abnormality occurrence from the determined positional relationship of the belt. Furthermore, Patent Document 2 discloses inputting image data of the widthwise end region of the belt into a deterioration level determination model to determine the deterioration level of the widthwise end region of the belt. Then, Patent Document 2 discloses outputting information on the determined belt position and information on the deterioration level for that belt position.

[0006] JP 2019-18999 A JP 2021-17296 A

[0007] However, the methods for monitoring belt abnormalities described in Patent Documents 1 and 2 can monitor abnormal conditions of the belt itself, such as meandering of the running belt, but cannot directly detect abnormalities of the transported objects on the belt, such as an excessively small load (transport amount) of the transported objects.

[0008] The present invention has been made in view of the above points, and has as its object to enable automatic detection of the conveyance state of an object on a belt.

[0009] In order to solve the problem, one aspect of the present invention is a conveying state detection device that detects the conveying state of an object on the belt of each belt conveyor in a plurality of belt conveyors that convey the object, and includes an imaging device that images the belt of each belt conveyor from above, and a conveying state detection unit that detects the conveying state of the object on the belt for each belt by image processing the image captured by the imaging device.

[0010] Another aspect of the present invention is a conveying state detection method for detecting the conveying state of an object being conveyed by the belt of each belt conveyor in a plurality of belt conveyors that convey the object, in which the belt of each belt conveyor is imaged from above and the captured image is processed to determine the conveying state of the object on the belt for each of the belts.

[0011] According to an aspect of the present invention, for example, in a facility where multiple belt conveyors transport objects, image processing is performed on captured images of the belts of each belt conveyor, and as a result, according to an aspect of the present invention, it becomes possible to automatically detect the transport state of the objects on the belts of each belt conveyor individually.

[0012] 1 is a schematic diagram showing an example of the configuration of a belt conveyor and a transport state detection device according to a first embodiment of the present invention. FIG. 2 is a diagram showing an upward belt as viewed from the belt running direction. FIG. 3 is a diagram showing an example of the configuration of a transport state detection unit. FIG. 4 is a diagram showing an example of a captured image according to a first embodiment based on the present invention. (a) shows an image captured by an imaging unit (camera). (b) shows an image obtained by cutting out a partial area from an image captured by the imaging unit (camera). FIG. 5 is a diagram showing an image of input ((a)) and output ((b)) in a trained individual model. A flow chart showing an example of processing in a transport state detection unit and a determination information output unit. FIG. 6 is a schematic diagram showing an example of the configuration of a belt conveyor and a transport state detection device according to a second embodiment of the present invention. FIG. 7 is a diagram showing an example of a captured image according to a second embodiment based on the present invention. (a) shows an image captured by an imaging unit (camera). (b) shows an image obtained by cutting out a partial area from an image captured by the imaging unit (camera).

[0013] "First embodiment" Next, a first embodiment of the present invention will be described with reference to the drawings. Here, a plurality of belt conveyors are installed in a raw material yard. Raw materials are transported to the raw material yard by the plurality of belt conveyors and stored as raw material piles. Furthermore, the raw materials are supplied to the next process by the belt conveyors as needed. In the raw material yard, raw materials of different types and brands are transported by changing between the plurality of belt conveyors at a transfer section using a drop-type chute or the like.

[0014] In this embodiment, the present disclosure will be described assuming application to multiple belt conveyors used in a raw material yard. However, the present disclosure can also be applied to conveyance facilities having multiple belt conveyors for conveying other conveyed objects. Furthermore, in this embodiment, the conveyed object conveyed by the belt will be described using an example of a raw material made of loose materials. The conveyed object is not limited to loose materials. Note that in the following description, the conveyed object will also be referred to as raw materials. Here, the present disclosure is a technology suitable for multiple belt conveyors that convey objects made of the same type of material. The destination of the object on each belt conveyor may be different. However, the present disclosure is a technology suitable for detecting the conveyance status on the belts of multiple belt conveyors in a conveyance facility where the object is conveyed via multiple belt conveyors.

[0015] (Overall Configuration) This embodiment includes a conveying facility and a conveying status detection device. The conveying facility includes multiple belt conveyors that convey the objects. The conveying status detection device is a device that detects the conveying status of the objects on the belt of each belt conveyor. This embodiment will be described using as an example a case where multiple belt conveyors used in the above-mentioned raw material yard are targeted. This embodiment also illustrates a configuration in which a conveying status detection device 21 is provided for each belt conveyor.

[0016] FIG. 1 is a schematic diagram showing one belt conveyor 1 and an individual conveyance state detection device 21 corresponding to that belt conveyor 1 in this embodiment. Here, multiple belt conveyors are installed in the raw material yard. However, generally, each belt conveyor that conveys raw materials has the same configuration. However, in this embodiment, each belt conveyor may have a different configuration. That is, the belt conveyor 1 shown below is an example, and a belt conveyor with a different configuration may also be used. Here, generally, in a raw material yard, the basic configuration of each belt conveyor and the conveyance state detection device corresponding to that belt conveyor is the same for each belt conveyor. For this reason, the following will only describe one belt conveyor 1 and its corresponding conveyance state detection device 21.

[0017] <Belt Conveyor 1> The belt conveyor 1 includes an endless belt 3 and a pulley 7. The belt 3 runs over a plurality of rollers 8 with a predetermined tension due to the rotation of the pulley 7, which is composed of a drive pulley 7a and a plurality of driven pulleys 7b. The drive pulley 7a is driven to rotate by an electric motor 5 via a reducer 6. In this embodiment, among the rollers 8, the rollers supporting the upper belt 3a are configured as a combination of a central roller 8a and left and right rollers 8b, as shown in FIG. 2. The central roller 8a is disposed at the center in the width direction of the belt (the left and right direction in FIG. 2). The left and right rollers 8b are disposed on the left and right of the central roller 8a and have inclined axes. This allows the roller 8 to support the upper belt 3a in a trapezoidal shape. The roller 8 supporting the lower belt 3b is configured as a single roller. However, the support configuration of the belt 3 by the rollers 8 is not limited to this.

[0018] In this embodiment, a raw material charging device 20, such as a chute, is located upstream of the upward belt 3a in the conveying direction. Raw materials 2 are loaded onto the belt 3 from the raw material charging device 20. The raw materials 2 are then conveyed by the running belt 3. The raw materials 2 are then conveyed downstream in the conveying direction by the upward belt 3a and are sequentially introduced into the chute 4.

[0019] <Transportation Status Detection Device 21> The transport status detection device 21 is a device that detects the transport status of the raw materials 2 loaded on the belt 3 of the corresponding belt conveyor 1. The transport status of the raw materials 2 is, for example, the placement state of the raw materials 2 on the running belt 3. As shown in Fig. 1, the transport status detection device 21 includes an imaging device 9 and a transport status detection unit 10. Reference numeral 11 denotes a database that serves as an image storage device that stores captured images.

[0020] <Imaging Device 9> The imaging device 9 in this embodiment is composed of a visible camera. The imaging device 9 captures an image of the upward belt 3a of the belt 3 of the belt conveyor 1 from above to obtain a captured image. FIG. 4 is a schematic diagram showing an example of a captured image. The imaging device 9 is adjusted and installed so that at least both widthwise ends of the belt 3 are captured. In the example of FIG. 1, the imaging axis of the imaging device 9 is installed at an angle to the running direction of the belt 3. Note that the installation location of the imaging device 9 must ensure a field of view that allows sufficient monitoring of the belt 3. FIG. 1 also illustrates a case where one imaging device 9 is installed for one belt conveyor 1. Two or more imaging devices 9 may be installed along the extension direction (running direction) of the upward belt 3a. In this case, ore falling during the conveyance can be detected from the difference in the conveyance state at two points along the conveyance direction.

[0021] <Transportation Status Detection Unit 10> As shown in FIG. 3, the transport status detection unit 10 includes an image processing unit 10A, a status determination unit 10B, and a determination information output unit 10C.

[0022] <Image Processing Unit 10A> The image processing unit 10A performs image processing on the captured image captured by the imaging device 9. The image processing unit 10A then acquires at least the transported object area information from the transported object area information and the belt area information. The transported object area information is transported object area information, which is information on the area of ​​the raw materials 2 on the belt 3. The belt area information is information on the area of ​​the belt 3 on which the raw materials 2 are not placed. The portion of the belt 3 on which the raw materials 2 are not placed is the portion where the upper surface of the belt 3 is exposed. Examples of the transported object area information include position information of the raw materials 2 placed thereon. The image processing unit 10A may be configured to perform binarization or other enhancement processing on the acquired image. In this case, the image processing unit 10A detects the width direction end positions of the belt 3 and the outer circumferential contour position of the raw materials 2 placed on the belt 3 in the image. The belt area information and the transported object area information are then acquired from the detected positions.

[0023] Belt conveyors 1 in raw material yards are often placed outdoors. In this case, the surrounding environment may affect the accuracy of detecting the widthwise edge positions of the belt 3 in the captured image, depending on the image enhancement process. Here, the surrounding environment refers to the season, weather, etc. Furthermore, enhancement processes may also affect the accuracy of detecting the outer contour positions of the raw materials 2 placed on the belt 3. For this reason, in this embodiment, a trained model 12 is generated in advance by machine learning, such as deep learning, using supervised data (see FIG. 3 ). The trained model 12 is a model that takes a captured image of the belt 3 of a target belt conveyor 1 as input and outputs transported object area information for the area where the raw materials 2 are placed on the belt 3. The image processing unit 10A of this embodiment uses the captured image as input data and acquires transported object area information using the trained model 12. A known learning method may be used for the machine learning to obtain the trained model 12. An example of machine learning for obtaining the trained model 12 is semantic segmentation.

[0024] Consider the case where semantic segmentation is employed. In this case, teacher image data is created by labeling each pixel constituting a previously captured image for learning into one of three area classes. The three areas are the area of ​​raw material 2 on belt 3, the area of ​​belt 3 excluding the raw material position, and other areas. In other words, the image is divided into the area of ​​raw material 2 on belt 3, the area of ​​belt 3, and other areas. Then, teacher image data is created by labeling each pixel in the image according to the area (class) into which it is classified. A large number of teacher image data are then trained using a convolutional neural network to generate a trained model 12. The trained model 12 receives the captured image (input data) as input and outputs information regarding at least the area of ​​raw material 2 in the captured image.

[0025] Here, it is advisable to prepare multiple pieces of teacher image data for images taken by the imaging device 9 at different times and in different weather conditions, and generate a model using the prepared multiple pieces of teacher image data. In this case, the area of ​​the raw material 2 can be detected stably and accurately even in an image of the belt conveyor 1 taken outdoors. As a result, automatic monitoring of the transport status of the raw material 2 becomes possible with high accuracy. The output information of the trained model 12 is, for example, data that includes information on which area in the captured image is the area of ​​the raw material 2 and which area is the area of ​​the belt 3. Note that the area of ​​the raw material 2 may consist of a single area, or may be divided into multiple areas.

[0026] When the trained model 12 using semantic segmentation is used, the output is as follows. That is, image data is output in which identification information (transported object area information) indicating that the pixel is a raw material 2 area is assigned to each pixel in the output image for the pixels in the raw material 2 area. That is, in the image data that is the output value of the trained model 12 using semantic segmentation, identification information is assigned to each pixel. The assigned identification information is information indicating whether the pixel is raw material 2, belt 3, or other. This makes it possible to identify which area is raw material 2 and which area is belt 3. For example, the trained model 12 outputs image data in which the pixel color is displayed for each class. Then, using the trained model 12 described above, the image processing unit 10A receives the captured image captured by the imaging device 9 as input and outputs image data including transported object area information.

[0027] Here, the captured image input to the trained model 12 does not need to be the exact image captured by the imaging device 9. Consider a case where the imaging device 9 is set to capture a predetermined length of the image in the longitudinal direction (travel direction) of the belt 3, as shown in FIG. 4(a). In this case, an image of a region, such as that shown in FIG. 4(b), obtained by cutting out a portion of the captured image in the longitudinal direction, may be used as the captured image input to the trained model 12. The region to be cut out is limited to a specific region where the conveying state of the image data needs to be detected. A specific region is, for example, a region where the state of the raw material 2 is stable, or, conversely, a region where the belt 3 is prone to meandering in the width direction. Two or more regions may also be cut out. In this case, image processing may be performed for each cut-out region. Here, reference numeral 3a denotes an exposed portion of the belt 3 on which the raw material 2 is not placed. The same applies to FIG. 5(a).

[0028] Note that the captured images captured by the imaging device 9 and the extracted images are sequentially stored in the database 11. Then, the stored images may be used to perform an update process on the trained model 12 as appropriate. In this embodiment, a conveyance state detection device 21 is provided for each belt conveyor 1. Therefore, a trained model 12 is generated individually for each belt conveyor. Then, the individual trained model 12 set for each belt conveyor is used to acquire transported object area information.

[0029] <State Determination Unit 10B> The state determination unit 10B determines the conveyance state of the raw material 2 based on the information acquired by the image processing unit 10A. Here, the conveyance state of the raw material 2 is, for example, information regarding the placement state of the raw material 2 placed on the belt 3. Furthermore, the conveyance state of the raw material 2 is information regarding the presence or absence of the raw material 2 on the belt 3, information regarding the amount of raw material 2 conveyed by the belt 3, and information regarding the width direction position of the raw material 2 present on the belt 3. If belt area information is also detected, the relative position of the raw material 2 with respect to the belt 3 can also be identified as the conveyance state. Furthermore, the conveyance state may be determined indirectly from the belt area information. The state determination unit 10B of this embodiment determines the conveyance state of the raw material 2 from the information obtained by the image processing unit 10A. The state determination unit 10B determines the conveyance state of the raw material 2, for example, from an image to which conveyed object area information and belt area information are added, output by the trained model 12.

[0030] <Conveyance Amount Calculation Unit 10Ba> The state determination unit 10B of this embodiment includes a conveyance amount calculation unit 10Ba. The conveyance amount calculation unit 10Ba calculates the area of ​​the raw material 2 present on the belt 3 when viewed from above. The conveyance amount calculation unit 10Ba calculates the conveyance amount of the raw material 2 by the belt 3 per unit length of the belt 3 from the calculated area of ​​the raw material 2. The conveyance amount constitutes an example of the conveyance state.

[0031] The conveyance amount calculation unit 10Ba, for example, calculates the number of pixels of the raw material 2 in the image data output by the trained model 12. The number of pixels of the raw material 2 corresponds to the area occupied by the raw material 2. The conveyance amount calculation unit 10Ba then calculates the occupied area of ​​the raw material 2 by multiplying the pixel size by a coefficient. The conveyance amount calculation unit 10Ba then calculates the conveyance amount per unit length of the raw material 2 loaded on the belt 3 from the calculated occupied area of ​​the raw material 2. The conveyance amount can be calculated from the area of ​​the raw material 2. For example, a formula for calculating the theoretical conveyance amount of a trough-type belt conveyor is used. Note that this calculation formula is publicly known, as described in, for example, "Mashima Utaro: Conveyor Calculation Method," Kogaku Tosho Co., Ltd., Tokyo, (1979), p. 14." In this case, the volume of the raw material 2, i.e., the conveyance amount, can be calculated from the number of pixels (area) of the raw material 2 calculated from the identified pixels. At this time, the speed and density of the belt 3 are applied from the operating conditions.

[0032] <Determination Information Output Unit 10C> The determination information output unit 10C outputs information such as the determination information determined by the state determination unit 10B, the transport amount determined by the transport amount calculation unit 10Ba, and the transport state determined from such information. The determination information output unit 10C notifies the operator of the determination information, for example, by outputting the information. An example of a processing flow of the state determination unit 10B and the determination information output unit 10C will be described with reference to FIG. 6. This processing is repeatedly executed, for example, each time output from the image processing unit 10A is input. First, in step S10, the state determination unit 10B inputs output data (image data) of the trained model 12.

[0033] In this embodiment, the trained model 12 inputs, for example, a captured image such as that shown in FIG. 5( a) and outputs image data such as that shown in FIG. 5( b). The image data shown in FIG. 5( b) shows a region R1 of raw material 2, a region R2 of belt 3, and other regions R3, each displayed differently. As described above, identification information is assigned to each pixel. Here, the identification information of a pixel may be changed for each type of raw material 2. In FIG. 5( b), R1 indicates the region of raw material 2, R2 indicates the region of belt 3 where raw material 2 is not present, and R3 indicates other regions. The same identification information is assigned to pixels in the same region.

[0034] Next, in step S20, the pixels of raw material 2 (the transported object) are identified in the image data acquired in step S10, and the number of pixels of raw material 2 is determined. The number of pixels is proportional to the area S occupied by raw material 2. The determined number of pixels is multiplied by a coefficient that converts the size of one pixel to its actual size on belt 3. This determines the area S occupied by raw material 2. Next, in step S30, it is determined whether the number of pixels of raw material 2 determined in step S20 is equal to or greater than a preset threshold A. Instead of the number of pixels of raw material 2, it may also be determined whether the area S occupied by raw material 2 is equal to or greater than a preset threshold. If it is equal to or greater than threshold A, the process proceeds to step S50, and if it is less than threshold A, the process proceeds to step S40. The threshold A may be set based on the operating conditions.

[0035] In step S40, it is determined that "raw material 2 is not on belt 3," and the determination result is output, ending the process. The determination result is notified, for example, by emitting an alarm, sending an email, or displaying on a screen. This allows the operator to know that raw material 2 is not on belt 3. If raw material 2 is present on belt 3, the process proceeds to step S50. Then, the transport amount V of raw material 2 within the cut-out image area is calculated. The transport amount V can be calculated from the area S of raw material 2 using known means. The calculated transport amount V is then output to the display unit.

[0036] Next, in step S60, it is determined whether the transport amount V calculated in step S50 is within a preset allowable range (V1 or more and V2 or less). If it is within the allowable range, the process ends. If it is outside the allowable range, the process proceeds to step S70. Whether it is within the allowable range may also be determined from the belt area information, i.e., the size of the exposed portion of the belt in the belt width direction. Note that outside the allowable range includes cases where the transport amount is too large or too small. If the transport amount is too large, (V>V2). If it is too small, (V<V1). In other words, the appropriate range of transport amounts (lower limit V1 and upper limit V2) can be calculated according to the transport conditions, and this range can be set as the allowable range.

[0037] In step S70, the result of the abnormality determination is output. The abnormality determination is notified, for example, by generating an alarm sound, sending an email, or displaying on a screen. The operator can adjust the amount of raw material 2 being transported by belt 3 based on the output determination result and the transport amount. The presence or absence of ore falling from belt 3 is also estimated. The operator may also adjust the amount of raw material 2 loaded on belt 3 based on the history of continuously acquired transport amounts. Here, if the state determination unit 10B determines that raw material 2 is loaded on belt 3, it may execute a process to detect the position of raw material 2 relative to belt 3 as a transport state.

[0038] For example, the positions of both widthwise ends of the belt 3 or the widthwise center of the belt 3 are determined from the pixels representing the area of ​​the belt 3. Furthermore, the center of gravity of the area of ​​the raw material 2 is determined from the positions of the pixels of the raw material 2. Then, the deviation of the center of gravity of the area of ​​the raw material 2 from the center of the belt 3 in the widthwise direction of the belt is detected. Furthermore, for example, the distance in the widthwise direction of the belt between the end of the widthwise direction of the belt 3 and the outline of the area of ​​the raw material 2 may be determined as the conveyance state of the raw material 2.

[0039] The distance in the belt width direction between the width direction edge of the belt 3 and the outline of the area of ​​the raw material 2 is used to determine the limit of the transport amount of the raw material 2 loaded on the belt 3. Here, this embodiment is an example in which a transport state detection device 21 is individually provided for each belt conveyor 1. In this case, multiple locations in the longitudinal direction on the belt 3 of one belt conveyor 1 may be used as locations for individually detecting the transport state. In this case, each location may have its own trained model 12, or each belt conveyor 1 may have a trained model 12 common to the multiple locations.

[0040] (Operation and Others) In this embodiment, a visible camera (imaging device 9) is installed for each belt conveyor 1 at a position where it can monitor the belt 3 of the belt conveyor 1. The camera constantly monitors the belt 3, which conveys the raw materials 2 (transported objects). In this embodiment, the image captured by the camera is processed to automatically detect the transport status of the raw materials 2 on the belt 3 for each belt conveyor. Because this embodiment employs automatic detection, the transport status of multiple belt conveyors 1 can be efficiently monitored collectively without relying on human eyes. At this time, an operator can grasp the presence or absence of raw materials 2 on the belt 3, the width and transport amount of raw materials 2 on the belt 3, etc. By appropriately adjusting the transport amount of raw materials 2 on each belt conveyor 1 to an appropriate value, it is possible to prevent ore from falling from the belt 3. Furthermore, efficient countermeasures can be taken before equipment trouble occurs. Therefore, this embodiment can reduce equipment trouble caused by ore falling, as well as cleaning time and costs.

[0041] Furthermore, it is possible to efficiently detect whether the proper maximum amount of raw materials is being transported without dropping. Therefore, by maximizing the transport volume relative to the equipment specifications of the belt conveyor 1, the transport volume can be reduced, leading to a reduction in demurrage fees. Furthermore, in this embodiment, the transport status is detected using an individual trained model 12 for each belt conveyor. Therefore, even when the present disclosure is applied to an outdoor belt conveyor 1, it is possible to accurately detect the transport status.

[0042] As described above, the present disclosure is useful for monitoring raw material yards where raw materials 2 such as iron ore and coal are transported and stored. The present disclosure is a technology particularly suitable for detecting imbalance of the raw materials 2. Note that the present disclosure does not indirectly detect the transport state of the raw materials 2 transported by the belt 3 from the state of the belt 3 itself, as in the conventional technology. The present disclosure detects the transport state, which is the state of the raw materials 2 on the belt 3. Then, from the detection of the transport state, it is possible to determine a transport abnormality of the belt 3 in some cases.

[0043] Second Embodiment Next, a second embodiment of the present invention will be described. Components similar to those in the first embodiment will be denoted by the same reference numerals, and detailed description of those components will be omitted. In the first embodiment, a case in which a conveyance state detection device 21 is individually provided for each belt conveyor 1 was exemplified. In the first embodiment, a trained model 12 was individually generated for each belt conveyor. In the first embodiment, the trained model 12 generated for each belt conveyor was used to acquire transported object area information on the belt of each belt conveyor. In contrast, in this embodiment, a trained model common to multiple belt conveyors is generated. Then, in this embodiment, transported object area information is acquired using that common trained model. This is what makes this embodiment different from the first embodiment. Other components may be configured similarly to the first embodiment.

[0044] In this embodiment, for example, as shown in Fig. 7 , an image of each belt 3 of a plurality of belt conveyors 1 is acquired by each imaging device 9. As shown in Fig. 7 , an imaging device 9 may be installed for each belt 3, or one imaging device 9 may be installed for a plurality of belts 3. The imaging device 9 images each belt 3 (upward belt 3a) of the corresponding belt conveyor from above, and acquires each captured image. The imaging device 9 supplies the acquired captured image to the conveyance state detection unit 10.

[0045] Then, similarly to the first embodiment, the transport status detection unit 10 uses the common trained model 12 in the image processing unit 10A to input the captured images captured by each imaging device 9 and output image data including transported object region information. The common trained model 12 may be generated using semantic segmentation or the like.

[0046] <Regarding the Trained Model> In the first embodiment, an individual trained model 12 is generated for each belt conveyor by using teacher image data of the belt 3 of each belt conveyor 1. In contrast, in the present embodiment, a trained model 12 common to multiple belt conveyors is generated by combining multiple sets of teacher image data for each of multiple belts 3 captured on multiple belt conveyors and training the combined data using a convolutional neural network.

[0047] Here, the captured images of the belt 3 captured by the multiple imaging devices 9 often have different fields of view. Therefore, if the captured images are used as training image data to generate the trained model 12, the accuracy of detecting the conveyance state may be reduced. Furthermore, if the captured images are input as they are to the trained model 12, the accuracy of detecting the conveyance state may be reduced. For this reason, as shown in FIG. 8 , it is preferable to perform a process (normalization process) to align the angles of view for each of the captured images of the belts 3 captured by the multiple imaging devices 9. Projective transformation processing is an example of the process for aligning the angles of view. By performing projective transformation processing on each captured image, captured images with different fields of view (imaging angles) can be aligned to the same angle of view (imaging angle). Note that when one imaging device 9 is installed for multiple belts 3, i.e., when multiple belts 3 exist in one captured image, the normalization process (projective transformation process) is performed individually for each belt 3 in the captured image.

[0048] As described above, in this embodiment, a single common trained model 12 is generated using captured images of multiple belts 3 with the same angle of view. Then, the normalized captured images are input to this common trained model 12. This enables the single common trained model 12 to accurately detect the conveyance states of the belts 3 of the multiple belt conveyors 1. That is, the conveyance states of the multiple belt conveyors 1 can be monitored by a single conveyance state detection unit 10. This improves monitoring efficiency compared to a configuration in which a conveyance state detection unit 10 is provided for each belt conveyor. Note that the multiple belt conveyors may be divided into multiple groups, and each group may have one common trained model 12 and one conveyance state detection unit 10. In this case, instead of providing a common trained model 12 for each group, a single common trained model 12 may be used for all of the multiple groups.

[0049] Furthermore, the common trained model 12 may be generated for each of two or more belt conveyors selected from the plurality of target belt conveyors. Other configurations and operations are the same as those of the first embodiment.

[0050] (Other) The present disclosure may also have the following configurations. (1) A conveyance status detection device for detecting the conveyance status of an object on each belt conveyor of a plurality of belt conveyors that convey the object, comprising: an imaging device that captures an image of the belt of each belt conveyor from above; and a conveyance status detection unit that detects the conveyance status of the object on each belt by image processing the image captured by the imaging device. (2) The conveyance status detection unit detects, for each belt, at least one of information on the presence or absence of an object on the belt and information on the amount of the object being conveyed by the belt, as the conveyance status of the object. (3) The conveyance status detection unit includes: an image processing unit that acquires, for each belt, conveyance area information that is information on the area of ​​the object present on each belt by image processing; and a state determination unit that determines the conveyance status, which is the state of the object loaded on the belt, based on the acquired conveyance area information. (4) The image processing unit acquires the transported object region information from the captured image captured by the imaging device using a trained model generated by training a convolutional neural network on teacher image data in which each pixel of the captured image captured in advance is labeled as to which class the region of the transported object on the belt, the belt, or other regions belongs to. (5) The trained model is provided individually for each of the belt conveyors. (6) The trained model is generated by training using the teacher image data of the multiple belt conveyors, and the trained model is used as a trained model common to the multiple belt conveyors. (7) The image processing unit performs a projective transformation process on the captured image captured by the imaging device and acquires the transported object region information from the projectively transformed captured image. (8) The transport status detection unit includes a transport amount calculation unit that calculates the area of ​​the transported object present on the belt for each of the belts and determines the transport amount of the transported object from the calculated area of ​​the transported object as transport status information. (9) The transport state detection unit is provided with a determination information output unit that determines whether the transport amount calculated by the transport amount calculation unit is within a preset tolerance range, and outputs the determination result if it is determined to be outside the tolerance range.(10) A conveyance state detection method for detecting the conveyance state of an object conveyed by the belt of each of a plurality of belt conveyors that conveys the object, comprising: capturing an image of the belt of each of the belt conveyors from above, and processing the captured image to determine the conveyance state of the object on the belt for each of the belts; (11) By the image processing, object area information, which is information regarding the area of ​​the object present on the belt, is obtained; and based on the obtained object area information, the conveyance state, which is the state of the object loaded on the belt, is determined for each of the belts.

[0051] The entire contents of Japanese Patent Application No. 2023-135730 (filed August 23, 2023), from which this application claims priority, are incorporated herein by reference. While the present application has described a limited number of embodiments, the scope of the invention is not limited thereto, and modifications of each embodiment based on the above disclosure would be obvious to one skilled in the art.

[0052] REFERENCE SIGNS LIST 1 Belt conveyor 2 Raw material (an example of a conveyed object) 3 Belt 9 Imaging device 10 Conveyance state detection unit 10A Image processing unit 10B State determination unit 10Ba Conveyance amount calculation unit 10C Determination information output unit 12 Trained model 21 Conveyance state detection device

Claims

1. A conveying condition detection device for detecting the conveying condition of an object on each of a plurality of belt conveyors that convey objects, comprising: an imaging device that images the belt of each conveyor from above; and a conveying condition detection unit that detects the conveying condition of the object on each of the belts by processing the images captured by the imaging device.

2. A conveying state detection device as described in claim 1, wherein the conveying state detection unit detects at least one of the following information for each belt as the conveying state of the conveyed object: the presence or absence of the conveyed object on the belt, and information on the conveying amount of the conveyed object by the belt.

3. A conveying state detection device as described in claim 1 or claim 2, wherein the conveying state detection unit comprises: an image processing unit that acquires, for each belt, conveyed object area information, which is information relating to the area of ​​the conveyed object present on each belt, by image processing; and a state determination unit that determines the conveying state, which is the state of the conveyed object loaded on the belt, based on the acquired conveyed object area information.

4. The transport state detection device of claim 3, wherein the image processing unit acquires the transported object area information from the image captured by the imaging device using a trained model generated by learning, with a convolutional neural network, teacher image data in which each pixel of the previously captured image is labeled to indicate which area it is classified into (the area of ​​the transported object on the belt, the area of ​​the belt, or other areas).

5. The conveyance state detection device according to claim 4, wherein the trained model is provided for each of the belt conveyors.

6. A conveying condition detection device as described in claim 4, wherein the trained model is a trained model generated by learning using the teacher image data of the multiple belt conveyors, and the trained model is a trained model common to the multiple belt conveyors.

7. A transport state detection device as described in any one of claims 3 to 6, wherein the image processing unit performs a projective transformation process on the image captured by the imaging device, and acquires the transported object area information from the captured image after projective transformation.

8. A transport state detection device as described in any one of claims 1 to 7, wherein the transport state detection unit is provided with a transport amount calculation unit that calculates the area of ​​the transported object present on the belt for each of the belts and determines the transport amount of the transported object from the calculated area of ​​the transported object as information on the transport state.

9. A transport state detection device as described in claim 8, wherein the transport state detection unit judges whether the transport amount calculated by the transport amount calculation unit is within a preset allowable range, and includes a judgment information output unit which outputs the judgment result when it is judged to be outside the allowable range.

10. A conveying condition detection method for detecting the conveying condition of an object conveyed by each of a plurality of belt conveyors, the method comprising: capturing an image of each of the belt conveyors from above; and processing the captured image to determine the conveying condition of the object on each of the belts.

11. A method for detecting a conveying state as described in claim 10, further comprising the steps of: acquiring conveyed object area information, which is information relating to the area of ​​the conveyed object present on the belt, by the image processing; and determining the conveying state, which is the state of the conveyed object loaded on the belt, for each belt based on the acquired conveyed object area information.

Images