Transport error processing device, transport error processing method, transport error processing program

Abstract

To provide a conveyance abnormality processing device or the like that can effectively treat a conveyance abnormality in a conveyance device.SOLUTION: A conveyance abnormality processing device 5 includes: an image abnormality registration part 54 that registers an image abnormality determined based on an image taken at a monitoring target site in a conveyance device 2 that conveys an object 3 to be conveyed; a sensor data acquisition part 55 that acquires sensor data measured by a sensor 42 that measures at least a part of the conveyance device 2 in a time zone when the image including the image abnormality was taken; a data abnormality specification part 56 that specifies a data abnormality in the sensor data; and a related registration part 57 that registers the data abnormality in association with the image abnormality. The conveyance abnormality processing device 5 further includes a conveyance abnormality estimation part 58 that monitors the sensor data and estimates the occurrence of a conveyance abnormality corresponding to the image abnormality registered by the related registration part 57, when an abnormality is found in the data abnormality registered by the related registration part 57.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a conveyance abnormality processing apparatus and the like. [Background Art]

[0002] Patent Document 1 discloses an image inspection technique capable of inspecting an inspection object with high accuracy based on an image obtained by capturing the inspection object. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2020-3452 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] According to the technique of Patent Document 1, an abnormality of an inspection object captured in an image can be detected, but there is a possibility that the cause of the abnormality cannot be identified. For example, when the inspection object has undergone conveyance and other processing, and the abnormality occurs during the processing, it is difficult to identify the cause of the abnormality from an image of the inspection object after the processing.

[0005] The present invention has been made in view of such circumstances, and an object of the present invention is to provide a conveyance abnormality processing apparatus and the like that can effectively process a conveyance abnormality in a conveyance device. [Means for Solving the Problem]

[0006] To solve the above problems, a transport anomaly processing device according to one aspect of the present invention includes: an image anomaly registration unit that registers image anomalies determined based on images taken at a monitored part of a transport device that transports an object to be transported; a sensor data acquisition unit that acquires sensor data measured by a sensor that measures at least a part of the transport device during the time period in which an image containing an image anomaly was taken; a data anomaly identification unit that identifies data anomalies in the sensor data; and an association registration unit that registers data anomalies in association with image anomalies.

[0007] In this embodiment, image anomalies (transport anomalies) determined based on images captured at the monitored part of the transport device are associated with data anomalies in sensor data measured during the same time period, allowing the transport anomaly processing device to accurately grasp the causal relationship between the two. It should be noted that "anomalies" in this invention are not limited to serious anomalies requiring immediate shutdown of the transport device, but also include minor deficiencies in transport quality that can be improved even while the transport device is operating. Therefore, the transport anomaly processing device according to the present invention can also be used for detecting and improving transport quality in a transport device.

[0008] Another aspect of the present invention is a method for processing transport anomalies. This method comprises: an image anomaly registration step of registering an image anomaly determined based on an image taken at a monitored part of a transport device that transports an object to be transported; a sensor data acquisition step of acquiring sensor data measured by a sensor that measures at least a part of the transport device during the time period in which an image containing the image anomaly was taken; a data anomaly identification step of identifying a data anomaly in the sensor data; and an association registration step of registering the data anomaly in association with the image anomaly.

[0009] Furthermore, any combination of the above components, as well as methods, apparatus, systems, recording media, computer programs, etc., derived from these representations, are also included in the present invention. [Effects of the Invention]

[0010] According to the present invention, transport abnormalities in a transport device can be effectively handled. [Brief explanation of the drawing]

[0011] [Figure 1] A schematic diagram of a conveying device that transports objects is shown. [Figure 2] This diagram schematically shows a list of images that the image display unit presents to the user. [Figure 3] A schematic example of the data anomaly detection unit's processing is shown below. [Modes for carrying out the invention]

[0012] The following describes in detail the embodiments (hereinafter also referred to as "models") for carrying out the present invention, with reference to the drawings. In the description and / or drawings, identical or equivalent components, members, processes, etc., are denoted by the same reference numerals, and redundant descriptions are omitted. The scale and shape of the illustrated parts are set for convenience to simplify the description and are not to be interpreted restrictively unless otherwise specified. The embodiments are illustrative and do not limit the scope of the present invention in any way. Not all features or combinations thereof described in the embodiments are necessarily essential to the present invention.

[0013] Figure 1 schematically shows a conveying device 2 that conveys an object to be conveyed 3. Examples of objects to be conveyed 3 include linear objects such as strings and wires, and planar objects such as paper, cloth, film, foil, and rubber. In this embodiment, a roll-to-roll conveying device 2 that conveys a planar substrate as the object to be conveyed 3 in the conveying direction (left-right direction in Figure 1) will be described. The conveying device 2 may also be part of a device that performs any processing on the conveyed object, such as a coater or coating device that applies a coating to the conveyed object, a printing machine that prints on the conveyed object, or a stretching device that applies tension to the conveyed object to stretch it. The conveying control device 1 controls the conveying operation of the conveying device 2.

[0014] The conveying device 2 comprises a conveying roller group 20, a dancer 24, an unwinding roller 26, and a winding roller 27. The conveying roller group 20 comprises multiple conveying rollers that serve as multiple conveying units for conveying the object to be conveyed 3. In the example in Figure 1, the conveying roller group 20 comprises three pairs of conveying rollers arranged in series along the conveying direction of the object to be conveyed 3. The multiple conveying rollers are adjacent to each other in the conveying direction. Each pair of conveying rollers comprises drive rollers 211 to 213 that are rotationally driven by the drive units 111 to 113 described later, and driven rollers 221 to 223 that sandwich the object to be conveyed 3 between themselves and the drive rollers 211 to 213 and rotate in conjunction with the drive rollers 211 to 213. The three pairs of conveying rollers 211 / 221 to 213 / 223 and the three drive units 111 to 113 and three speed control units 121 to 123 provided in the conveying control device 1 corresponding to each can be configured similarly to one another. Therefore, the first conveyor roller pair 211 / 221, the first drive unit 111, and the first speed control unit 121 will be described below, and redundant explanations of the other conveyor roller pairs 212 / 222, 213 / 223, the other drive units 112, 113, and the other speed control units 122, 123 will be omitted. Note that the number of conveyor roller pairs provided in the conveyor roller group 20 may be arbitrary (any integer of 1 or more). Also, some or all of the conveyor rollers may not be provided with drive units. In this case, the conveyor rollers become driven rollers that rotate in accordance with the conveyed object 3.

[0015] The drive roller 211 and driven roller 221 are conveying rollers as one aspect of the conveying unit that conveys the object to be conveyed 3, and are rotatable around a rotation axis in a direction perpendicular to the conveying direction (left-right direction in Figure 1) (a direction perpendicular to the plane of the paper in Figure 1). The drive roller 211 is rotationally driven by a drive unit 111 such as a motor in accordance with a rotational speed command generated by the speed control unit 121. When the conveying direction of the object to be conveyed 3 in Figure 1 is to the right, the drive roller 211 is rotationally driven clockwise by the drive unit 111, and the driven roller 221 rotates counterclockwise in conjunction with the drive roller 211. By individually rotating each of the drive rollers 211 to 213 constituting the conveying roller group 20 by each of the drive units 111 to 113 of the conveying control device 1, the speed and tension of each part of the object to be conveyed 3 can be precisely controlled, thereby optimizing the conveying operation of the object to be conveyed 3 by the conveying device 2 and the processing by various devices on which the conveying device 2 is installed.

[0016] The Dansa 24 is installed, for example, upstream (left side in Figure 1) in the conveying direction of the conveying roller group 20. The Dansa 24 applies tension to the conveyed object 3 in the conveying direction. The Dansa 24 comprises a pair of rollers 241 and 242 installed on the conveying path of the conveyed object 3 (the left-right path in which the conveyed object 3 extends in Figure 1), and a Dansa roller 243 installed between the pair of rollers 241 and 242, at a position deviating from the conveying path of the conveyed object 3. The Dansa roller is also called a Dansa roll.

[0017] The dancer roller 243 is provided so as to be movable between its upper end 243A and lower end 243B in a direction perpendicular to the transport path of the object to be transported 3 (vertical direction in Figure 1). The air cylinder 244, acting as a thrust-adding unit, generates a thrust that biases or pressurizes the dancer roller 243 away from the transport path of the object to be transported 3 (downward in Figure 1). This thrust is based on the air pressure of the air cylinder 244 connected to the dancer roller 243 via a piston rod or connecting rod. The air pressure of the air cylinder 244 is generated by a thrust control unit 17, which consists of an electro-pneumatic regulator or the like that controls the air pressure electrically. Generally, a substantially constant voltage is applied to the electro-pneumatic regulator (thrust control unit 17), and the air pressure of the air cylinder 244, i.e., the thrust of the dancer roller 243, is controlled to be substantially constant. Alternatively, instead of the air cylinder 244, a thrust-adding unit that applies thrust to the dancer roller 243 based on another principle (for example, a linear motor that applies thrust to the dancer roller 243 based on electricity) may be provided.

[0018] The dancer roller 243, biased or pressurized downward by the thrust from the air cylinder 244, applies tension to the conveyed object 3 by pulling it away from the conveying path. At this time, the downward thrust received by the dancer roller 243 from the air cylinder 244 balances the upward tension received from the conveyed object 3. As mentioned above, generally, the downward thrust received by the dancer roller 243 from the air cylinder 244 is maintained or controlled to be approximately constant, so the tension that the dancer roller 243 applies to the conveyed object 3 is approximately constant. Typically, the dancer 24 is adjusted so that the downward thrust and upward tension applied to the dancer roller 243 are balanced when the vertical position of the dancer roller 243 is approximately in the middle of the upper end 243A and the lower end 243B. Conversely, the vertical position of the dancer roller 243 can indirectly represent the tension of the conveyed object 3.

[0019] The position of the dancer roller 243 in the thrust direction (the vertical direction in FIG. 1) is detected as an electrical signal by the position detection unit 245 or a position sensor, and provided to the subtracter 14 of the conveyance control device 1. In addition, a position command for the dancer roller 243 in the thrust direction generated by the position command generation unit 13 of the conveyance control device 1 is input to the subtracter 14. As described above, since the position of the dancer roller 243 substantially corresponds to the tension of the conveyed object 3, the position command for the dancer roller 243 generated by the position command generation unit 13 substantially corresponds to a tension command for the conveyed object 3. The speed control unit 15 of the conveyance control device 1 generates a speed command for reducing the deviation of the position of the dancer roller 243 or the tension of the conveyed object 3 provided from the subtracter 14. This speed command is a command for the conveyance speed of the conveyed object 3, and specifically is a command for the rotation speed of the drive roller 251 described below.

[0020] The drive unit 16 of the conveyance control device 1 rotationally drives the drive roller 251 provided immediately adjacent to the dancer 24 in accordance with the speed command provided from the speed control unit 15. The drive roller 251 is a conveyance roller rotatable around a rotation axis orthogonal to the conveyance direction of the conveyed object 3. When the drive roller 251 is rotationally driven in the clockwise direction in FIG. 1, the driven roller 252 rotates in the counterclockwise direction in conjunction with the drive roller 251. Also, immediately before the dancer 24, a drive roller 231 similar to the drive roller 251 and a driven roller 232 similar to the driven roller 252 are provided. The drive roller 231 is rotationally driven at a constant rotation speed, for example, by a drive unit not shown in the drawings. In contrast, the rotation speed of the drive roller 251 is adaptively controlled in accordance with the position and / or tension deviation. In this way, while conveying the conveyed object 3 sandwiched between the drive roller 251 and the driven roller 252, they apply a desired tension corresponding to the position command for the dancer roller 243 generated by the position command generation unit 13, that is, the tension command for the conveyed object 3, to the conveyed object 3. In the example of FIG. 1 in which the dancer 24 is provided immediately before the conveyance roller group 20 in the conveyance direction, the tension of the conveyed object 3 at the inlet portion (the left end in FIG. 1) of the conveyance roller group 20 can be controlled to a desired value.

[0021] In addition to or instead of the dancer 24, a tension detector or tension sensor capable of directly detecting the tension of the conveyed object 3 may be provided. The unwinding roller 26, provided at the starting point of the conveyed object 3 and / or the conveying device 2, unwinds the conveyed object 3 along the conveying direction. The unwinding roller 26 is rotated clockwise in Figure 1 by a drive unit 114 such as a motor in accordance with a rotation speed command generated by the speed control unit 124. The winding roller 27, provided at the end point of the conveyed object 3 and / or the conveying device 2, winds up the conveyed object 3. The winding roller 27 is rotated clockwise in Figure 1 by a drive unit 115 such as a motor in accordance with a rotation speed command generated by the speed control unit 125.

[0022] In order to effectively handle the transport abnormalities in the transport device 2 and the resulting abnormalities that appear in the transported object 3, a camera group 41 consisting of one or more cameras that photograph various parts of the transport device 2 and / or the transported object 3 is provided, a sensor group 42 consisting of one or more sensors that measure various parts of the transport device 2 and / or the transported object 3 is provided, and a transport abnormality processing device 5 is provided that processes abnormalities in the transport device 2 and / or the transported object 3 based on image data acquired by the camera group 41 and sensor data acquired by the sensor group 42. The camera group 41 and the sensor group 42 are collectively referred to as camera group / sensor group 4. In addition, one or more cameras that make up the camera group 41 are also referred to as camera 41, and one or more sensors that make up the sensor group 42 are also referred to as sensor 42.

[0023] Each of the cameras 41 constituting the camera group 41 is installed so as to be capable of photographing any monitoring target site in the conveying device 2 and / or the conveyed object 3. As the monitoring target sites in the example of FIG. 1, from left to right, there are listed an unwinding roller 26 that unwinds the conveyed object 3, a driving unit 114 as a motor that drives the unwinding roller 26, a dancer 24 that applies tension to the conveyed object 3, a driving roller 251 as a conveying roller that conveys the conveyed object 3, a driving unit 16 as a motor that drives the driving roller 251, driving rollers 211 to 213 and / or driven rollers 221 to 223 as conveying rollers that convey the conveyed object 3, driving units 111 to 113 as motors that drive the driving rollers 211 to 213, a winding roller 27 that winds the conveyed object 3, and a driving unit 115 as a motor that drives the winding roller 27.

[0024] However, the monitoring target sites by the camera group 41 are not limited to these, and may be any sites in the conveying device 2 and / or the conveyed object 3. Since the camera group 41 is provided for photographing or detecting an abnormality in the conveying device 2 and / or the conveyed object 3, it is preferable that the camera group 41 is installed at a site where an abnormality occurs frequently. As such monitoring target sites, mechanical elements of the conveying device 2 (the unwinding roller 26, the dancer 24, the winding roller 27, the driving units 16, 111 to 115, the driving rollers 251, 211 to 213, the driven rollers 221 to 223, etc.) that are involved in the mechanical conveying operation of the conveyed object 3 and listed above are suitable. Further, when the conveying device 2 is a part of an apparatus (a coater, a printing press, a stretching apparatus, etc.) that performs any treatment on the conveyed object 3, an abnormality is also likely to occur in a treatment unit responsible for the treatment (for example, a printing unit in a printing press). Therefore, in order to set the whole or a part of the treatment unit as a monitoring target site, it is preferable that one or a plurality of cameras 41 for photographing the same are provided.

[0025] Each sensor 42 constituting the sensor group 42 is installed to measure any measurement point on the conveying device 2 and / or the conveyed object 3 in any manner. In the example in Figure 1, the measurement points include the unwinding roller 26, the dancer 24, the winding roller 27, the thrust control unit 17, the drive units 16, 111-115, the drive rollers 251, 211-213, the driven rollers 221-223, and the processing units such as the printing unit, similar to the areas monitored by the camera group 41 described above. Thus, the areas monitored by the camera group 41 and the measurement points by the sensor group 42 may overlap. However, even if the camera 41 and the sensor 42 are installed in the same location, there is a difference in that the camera 41 generates image data, while the sensor 42 generates measurement data (non-image data) according to its method.

[0026] The type of sensor 42 and the type of sensor data are arbitrary. For example, a sensor 42 attached to the unwinding roller 26 can acquire sensor data such as the rotational position and rotational speed of the unwinding roller 26 and the diameter (unwinding diameter) of the conveyed object 3 wound around the unwinding roller 26. A sensor 42 attached to the winding roller 27 can acquire sensor data such as the rotational position and rotational speed of the winding roller 27 and the diameter (winding diameter) of the conveyed object 3 wound around the winding roller 27. Sensors 42 attached to the drive rollers 251, 211-213 and / or driven rollers 221-223 as conveying rollers can acquire sensor data. Sensor data such as the rotational position and rotational speed of each unit can be acquired. Sensor data such as speed, current, and torque related to each rotational drive can be acquired by sensors 42 attached to the drive units 16, 111-115 as motors. Sensor data such as the tension of the conveyed object 3 and the position of the dancer roller 243 can be acquired by the dancer 24 and tension detector, which themselves function as sensors 42. Sensor data such as the thrust (air pressure of the air cylinder 244) can be acquired by sensors 42 attached to the thrust control unit 17.

[0027] Furthermore, if the transport device 2 is installed in a printing press that performs multi-color printing on the transported object 3, a mark sensor (for example, one that detects the spacing between register marks printed on the transported object 3 by each color printing unit) may be used as the sensor 42 for registration control to reduce registration errors, which are misalignments between printing or printing plates. In addition, sensors 42 such as thermometers and hygrometers may be installed at any location to acquire sensor data related to the environment, such as temperature and humidity, for each part of the transport device 2. Other sensors such as vibration sensors and inertia sensors that can acquire sensor data such as vibrations of motors and rollers may also be used as the sensor 42.

[0028] The transport anomaly processing device 5 comprises an image data acquisition unit 51, an image presentation unit 52, a selection operation reception unit 53, an image anomaly registration unit 54, a sensor data acquisition unit 55, a data anomaly identification unit 56, a related registration unit 57, a transport anomaly estimation unit 58, and a transport anomaly notification unit 59. Some of these functional blocks can be omitted as long as the transport anomaly processing device 5 can realize at least some of the operations and / or effects described below. These functional blocks are realized through the cooperation of hardware resources such as the central processing unit, memory, input devices, output devices, and peripheral devices connected to the computer, and software that runs using them. Regardless of the type or location of the computer, each of the above functional blocks may be realized with the hardware resources of a single computer, or with a combination of hardware resources distributed across multiple computers.

[0029] The image data acquisition unit 51 acquires image data captured by the camera group 41 at each monitored part of the transport device 2 and / or the transported object 3. The image data acquisition unit 51 may acquire real-time image data directly from the camera group 41, or it may read image data that has been captured by the camera group 41 in the past and stored in storage not shown. Each image data acquired by the image data acquisition unit 51 is accompanied by shooting time information, such as a timestamp indicating the shooting time or shooting time period.

[0030] The image presentation unit 52 presents the user with images taken by each camera 41 at each monitored area, based on the image data acquired by the image data acquisition unit 51. For example, the image presentation unit 52 presents the user with a list of multiple images taken by one camera 41 at different times or time periods at a single monitored area. The selection operation reception unit 53 receives the user's selection operation for each image in the list presented by the image presentation unit 52.

[0031] Figure 2 schematically shows a list of images presented to the user by the image presentation unit 52. In this example, three images ("Image 1" to "Image 3") are schematically shown as a list of images of the unwinding roller 26, which is the part to be monitored. Note that the part to be monitored for which the list of images should be displayed is not limited to the unwinding roller 26, and the user can arbitrarily select any other part. The user visually inspects each image displayed by the image presentation unit 52 and determines whether or not there is an abnormality in each image. For images in which no abnormality is observed ("Image 1" and "Image 3" in the example of Figure 2), the user checks the "Good" checkbox (which constitutes the selection operation reception unit 53) to indicate this (or the "Good" checkbox may be checked by default). Note that the selection operation reception unit 53, which receives the user's selection operation, is not limited to checkboxes, but can be implemented by any input means or selection means on the screen.

[0032] On the other hand, for images in which an abnormality is observed (e.g., "Image 2" in the example in Figure 2), the user checks a "Bad" checkbox (which constitutes the selection operation reception unit 53) to indicate this. In this way, in the example in Figure 2, the user's judgment (selection operation) that "Image 2" is "abnormal" is input on the screen through the checkbox in the selection operation reception unit 53. As shown in Figure 2, each image is accompanied by shooting time information such as the time of shooting, so the transport abnormality processing device 5 can recognize that an abnormality occurred in the unwinding roller 26, which is the monitored part, at the time or time period of shooting of "Image 2" in response to the user's selection operation through the selection operation reception unit 53. Note that the selection or judgment of such abnormal images ("Image 2") may be performed autonomously by artificial intelligence equipped with machine learning capabilities, without user intervention. In this case, the image presentation unit 52 and / or the selection operation reception unit 53 for user interface are unnecessary.

[0033] The image anomaly registration unit 54 registers image anomalies determined based on image data acquired by the image data acquisition unit 51 (images taken by the camera 41 at the monitored area). In the example in Figure 2, the image anomaly registration unit 54 registers an image anomaly in the image selected by the user from the list of images presented ("Image 2"). If artificial intelligence detects an anomaly in the image on behalf of the user, the image anomaly registration unit 54 registers the image anomaly based on the anomaly detection result.

[0034] The sensor data acquisition unit 55 acquires sensor data measured by the sensor group 42 during the time period in which an image containing an image anomaly registered by the image anomaly registration unit 54 was taken. The sensor data acquisition unit 55 may acquire real-time sensor data directly from the sensor group 42, or it may read sensor data that has been measured by the sensor group 42 in the past and stored in storage (not shown). Each sensor data acquired by the sensor data acquisition unit 55 is accompanied by measurement time information, such as a timestamp indicating the measurement time or measurement time period. Therefore, the sensor data acquisition unit 55 can accurately extract sensor data from the same time period as the image anomaly registered by the image anomaly registration unit 54. As will be described later, the sensor data acquisition unit 55 may further acquire normal sensor data measured by the sensor group 42 during normal time periods in which images that do not contain the image anomaly registered by the image anomaly registration unit 54 were taken.

[0035] The data anomaly identification unit 56 identifies data anomalies in the sensor data acquired by the sensor data acquisition unit 55. Figure 3 schematically shows an example of the processing of the data anomaly identification unit 56. The data anomaly identification unit 56 comprehensively analyzes the sensor data group measured by the sensor group 42 during the time period before and after the time period in which the image anomaly registered by the image anomaly registration unit 54 occurred (the time period in which the image containing the image anomaly was captured by the camera 41), which is shown as the "image anomaly occurrence period" in Figure 3.

[0036] It is preferable for the data anomaly identification unit 56 to analyze not only the sensor data from sensors 42 installed near the location where the camera 41 that captured the image anomaly is installed, but also the sensor data from sensors 42 installed at a distance from the location where the camera 41 that captured the image anomaly is installed. For example, in the example in Figure 3, even if the image anomaly was captured near "motor 1", the data anomaly identification unit 56 will also analyze sensor data such as the rotation speed (rotational velocity) of "motor 2", which is located away from "motor 1". This is because the anomaly (image anomaly) confirmed in "motor 1" may be caused by a malfunction of "motor 2", etc.

[0037] The data anomaly identification unit 56 may identify data anomalies in the sensor data based on predetermined thresholds for one or more sensor data. In the example in Figure 3, the rotation speed of "motor 1" increases or decreases sharply before and after the "image anomaly occurrence period". In this case, the data anomaly identification unit 56 applies predetermined thresholds to the absolute value, increase amount, increase rate, decrease amount, decrease rate, variability (standard deviation, etc.) of the rotation speed of "motor 1". For example, the data anomaly identification unit 56 identifies an anomaly (data anomaly) in the rotation speed of "motor 1" when the increase amount of the rotation speed of "motor 1" exceeds a predetermined threshold. Note that in Figure 3, for simplicity, only the rotation speed of "motor 1" fluctuates during the same time period as the "image anomaly occurrence period", but in reality, it is conceivable that one image anomaly may occur as a result of fluctuations in multiple sensor data. In such cases, the data anomaly identification unit 56 treats multiple sensor data for which fluctuations synchronized with the image anomaly have been confirmed as a group. In other words, the data anomaly identification unit 56 may identify a complex data anomaly spanning multiple sensor data for a single image anomaly.

[0038] The data anomaly identification unit 56 may identify a data anomaly in the sensor data if the difference between the sensor data from the time period in which an image containing an image anomaly, as shown in Figure 3, was taken, and the normal sensor data measured by the sensor group 42 during a normal time period in which an image without an image anomaly (for example, "Image 1" or "Image 3" in Figure 2) was taken, is greater than or equal to a threshold. In other words, the data anomaly identification unit 56 can use the sensor data group from a normal time period in which no image anomalies were confirmed as a reference to identify data anomalies in the sensor data group from an abnormal time period in which an image anomaly was confirmed with high accuracy. In particular, by making the threshold for identifying data anomalies small, even slight fluctuations in sensor data (for example, slight differences between a normal time period and an abnormal time period, or slight fluctuations before and after an abnormal time period as shown in Figure 3) can be identified as data anomalies that may be related to image anomalies without being overlooked.

[0039] The related registration unit 57 associates data anomalies in one or more sensor data identified by the data anomaly identification unit 56 with one image anomaly (one image captured by the camera 41) registered by the image anomaly registration unit 54. In other words, the related registration unit 57 registers or stores a causal relationship in which a data anomaly in one or more sensor data identified by the data anomaly identification unit 56 caused one image anomaly registered by the image anomaly registration unit 54. The causal relationship between data anomalies and image anomalies registered by the related registration unit 57 in this way may be used as training data or learning data for training or machine learning of artificial intelligence equipped with machine learning capabilities. Such artificial intelligence can be provided in the transport anomaly estimation unit 58, which will be described next, to improve the accuracy of transport anomaly estimation.

[0040] The transport anomaly estimation unit 58 monitors the sensor data group measured in real time by the sensor group 42 based on the causal relationship between data anomalies and image anomalies registered by the related registration unit 57. Specifically, when the transport anomaly estimation unit 58 detects a data anomaly (an abnormal fluctuation or behavior in one or more sensor data, as shown in Figure 3) registered by the related registration unit 57 in one or more sensor data measured in real time by the sensor group 42, it estimates the occurrence of a transport anomaly corresponding to an image anomaly (for example, an image anomaly included in "Image 2" in Figure 2) registered by the related registration unit 57. The transport anomaly notification unit 59 notifies users of the transport anomaly estimation unit 58, such as the transport control device 1, transport device 2, and transport anomaly processing device 5, of the transport anomaly estimated by the transport anomaly estimation unit 58 through screen display, alarm, light, etc.

[0041] According to this embodiment, an image anomaly (transportation anomaly) determined based on images captured by the camera group 41 at the monitored parts of the transport device 2 and / or the transported object 3 is associated with a data anomaly in the sensor data measured by the sensor group 42 during the same time period. As a result, the transport anomaly processing device 5 can accurately grasp the causal relationship between the two. Furthermore, the transport anomaly estimation unit 58, which is configured with artificial intelligence or the like in the transport anomaly processing device 5, can autonomously estimate the occurrence of a transport anomaly corresponding to an image anomaly by monitoring the sensor data group in real time based on the causal relationship between the accumulated image anomalies and data anomalies.

[0042] The present invention has been described above based on embodiments. Various modifications are possible for each component and each combination of processes in the exemplary embodiments, and it will be obvious to those skilled in the art that such modifications are included within the scope of the present invention.

[0043] The configuration, operation, and function of each device and method described in the embodiments can be realized by hardware resources or software resources, or by the cooperation of hardware resources and software resources. Hardware resources include, for example, processors, ROMs, RAMs, and various integrated circuits. Software resources include, for example, operating systems and application programs. [Explanation of symbols]

[0044] 1 Transport control device, 2 Transport device, 3 Transported object, 4 Camera group / sensor group, 5 Transport anomaly processing device, 41 Camera, 42 Sensor, 51 Image data acquisition unit, 52 Image presentation unit, 53 Selection operation reception unit, 54 Image anomaly registration unit, 55 Sensor data acquisition unit, 56 Data anomaly identification unit, 57 Related registration unit, 58 Transport anomaly estimation unit, 59 Transport anomaly notification unit.

Claims

1. An image anomaly registration unit that registers image anomalies determined based on images taken at the monitored part of a transport device that transports objects, A sensor data acquisition unit acquires sensor data measured by a sensor that measures at least a part of the transport device during the time period in which the image containing the aforementioned image abnormality was taken. A data anomaly identification unit that identifies data anomalies in the sensor data, A related registration unit that registers the aforementioned data anomaly in association with the aforementioned image anomaly, A transport anomaly estimation unit monitors the sensor data and, when a data anomaly registered by the related registration unit is detected, estimates the occurrence of a transport anomaly corresponding to the image anomaly registered by the related registration unit. A transport abnormality processing device equipped with the following:

2. The transport anomaly processing apparatus according to claim 1, further comprising a transport anomaly notification unit that notifies the transport anomaly estimated by the transport anomaly estimation unit.

3. The system further includes an image display unit that presents images taken at the monitored area to the user. The image anomaly registration unit registers the image anomalies determined by the user based on the presented image. A transport abnormality processing device according to claim 1 or 2.

4. The image display unit presents the user with a list of multiple images taken at different times on the monitored area. The image anomaly registration unit registers image anomalies based on the image selected by the user from the list. The transport abnormality processing device according to claim 3.

5. The transport anomaly processing device according to claim 1 or 2, wherein the data anomaly identification unit identifies a data anomaly in the sensor data based on a predetermined threshold for the sensor data.

6. The sensor data acquisition unit further acquires normal sensor data measured by the sensor during normal time periods when images without image abnormalities are captured. The data anomaly identification unit identifies a data anomaly in the sensor data if the difference between the sensor data and the normal sensor data during the time period in which the image containing the image anomaly was taken is greater than or equal to the threshold. The transport abnormality processing device according to claim 5.

7. An image anomaly registration step, which registers image anomalies determined based on images taken at a monitored part of a transport device that transports an object to be transported, A sensor data acquisition step involves acquiring sensor data measured by a sensor that measures at least a part of the transport device during the time period in which the image containing the aforementioned image anomaly was taken, A data anomaly identification step for identifying data anomalies in the sensor data, A related registration step involves registering the aforementioned data anomaly in association with the aforementioned image anomaly, A transport anomaly estimation step that monitors the sensor data and, if a data anomaly registered in the associated registration step is detected, estimates the occurrence of a transport anomaly corresponding to the image anomaly registered in the associated registration step. A method for handling transport abnormalities, comprising:

8. An image anomaly registration step, which registers image anomalies determined based on images taken at a monitored part of a transport device that transports an object to be transported, A sensor data acquisition step involves acquiring sensor data measured by a sensor that measures at least a part of the transport device during the time period in which the image containing the aforementioned image anomaly was taken, A data anomaly identification step for identifying data anomalies in the sensor data, A related registration step involves registering the aforementioned data anomaly in association with the aforementioned image anomaly, A transport anomaly estimation step that monitors the sensor data and, if a data anomaly registered in the associated registration step is detected, estimates the occurrence of a transport anomaly corresponding to the image anomaly registered in the associated registration step. A transport error handling program that instructs a computer to execute.

Images