State determination device, state determination method, and program

Abstract

To provide a state determination device, state determination method and program which can reduce and save a load on operation personnel of a facility operation.SOLUTION: A state determination device comprises: an image acquisition unit which acquires an image obtained by imaging an object from an imaging device that is provided so as to be able to image a state where the object is conveyed in a conveyance facility that conveys the object from a front-stage facility to a subsequent-stage facility; a conveyance amount acquisition unit which acquires a conveyance amount of the object in the image on the basis of the image acquired by the image acquisition unit; a motion amount acquisition unit which acquires a motion amount of the object in the image on the basis of the plurality of images captured at different times by the image acquisition unit; and a state determination unit which determines whether or not the state of the object satisfies a requested condition on the basis of the conveyance amount of the object acquired by the conveyance amount acquisition unit and the motion amount of the object acquired by the motion amount acquisition unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a state determination device, a state determination method, and a program. [Background technology]

[0002] 2. Description of the Related Art In a facility that includes a process in which an object is transported, it is important to know the state of the object in order to control the operation state of the facility. For example, in a sewage treatment plant, solids in sewage are dehydrated in a dehydration facility to become dehydrated sludge (an example of a target object), and the dehydrated sludge is incinerated in an incineration facility or dried in a drying facility. The dehydrated sludge is transported from the dehydration facility to the incineration facility or drying facility by a transport facility such as a belt conveyor or a screw conveyor. When the dewatered sludge is transported, a worker may visually judge whether the state of the dewatered sludge is suitable for downstream processing and, depending on the result of the judgment, decide whether the dewatered sludge can be supplied to the downstream equipment. When the worker makes a visual judgment, the accuracy of the judgment may vary depending on the experience of the worker. In response to this, various techniques have been proposed to improve the accuracy of judgment by determining the condition of an object using methods other than visual inspection by an operator.

[0003] For example, Patent Document 1 listed below discloses a technique in which slurry sludge (an example of a target object) discharged from the outlet of a dehydrator is continuously sampled, compressed, and then the moisture content is measured using microwaves.

[0004] Furthermore, Patent Document 2 listed below discloses a technology for evaluating the condition of activated sludge (an example of a target object) contained in sludge water by flowing sludge water from a water treatment tank into a transport flow path, capturing an image of the sludge water as it is transported through the transport flow path, and performing image processing on the obtained image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-1019 A [Patent Document 2] International Publication No. 2017 / 030138 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the technology of Patent Document 1, it is necessary to provide many facilities to grasp the state of the object, such as facilities for collecting the slurry sludge and facilities for compacting it. In addition, in the technology of Patent Document 1, it is necessary to compact the slurry sludge and measure it, and the sludge after measurement may become lumps and be transported to the equipment of the downstream process, which may affect the operation of the equipment of the downstream process. In the technology of Patent Document 2, the evaluation object is activated sludge, and it is difficult to judge the state of the slurry sludge. In this way, due to differences in equipment, costs, evaluation objects, etc., there were cases where the operating personnel had to judge the state of the slurry sludge by visual inspection. In this case, it was necessary to prepare an operator who visually judges the state of the slurry sludge, and the judgment results may differ depending on the difference in operating experience.

[0007] In view of the above-mentioned problems, an object of the present invention is to provide a state determination device, a state determination method, and a program that are capable of reducing the burden on equipment operation personnel and saving labor. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, a state determination device according to one aspect of the present invention includes an image acquisition unit that acquires an image of an object from an imaging device that is provided in a transportation facility that transports the object from a facility in a previous stage to a facility in a subsequent stage, the image acquisition unit acquiring an image of the object from an imaging device that is provided in such a manner that the object is being transported, and a state determination unit that determines a state of the object in the image based on the image acquired by the image acquisition unit. A quantity indicating the amount of the object being transported by the transport facility. Transport amount As Based on a conveyance amount acquisition unit that acquires the conveyance amount and a plurality of images captured at different times by the image acquisition unit, A value obtained by quantifying the movement of the object between two images indicates a value obtained by quantifying the movement of the object being transported by the transport equipment. momentum As An exercise amount acquisition unit for acquiring the exercise amount; Using a machine learning model that has learned a correspondence relationship between the transport amount of the object, the motion amount of the object, and the state of the object, The transport amount of the object acquired by the transport amount acquisition unit and the motion amount of the object acquired by the motion amount acquisition unit The state of the corresponding object is suitable for subsequent processing. and a state determination unit that determines whether or not the state is correct.

[0009] A state determination method according to one aspect of the present invention includes a step of: an image acquisition unit acquiring an image of an object from an imaging device provided in a conveying facility that conveys the object from a preceding facility to a succeeding facility, the imaging device being capable of imaging a state in which the object is conveyed; and a conveyance amount acquisition unit calculating a distance of the object in the image based on the image acquired by the image acquisition unit. The amount indicates the amount of the object being transported by the transport facility. Transport amount As and a motion amount acquisition unit acquires a motion amount based on a plurality of images captured at different times by the image acquisition unit. A value obtained by quantifying the movement of the object between two images indicates a value obtained by quantifying the movement of the object being transported by the transport equipment. momentum As The process of acquiring the information and the state determination unit Using a machine learning model that has learned a correspondence relationship between the transport amount of the object, the motion amount of the object, and the state of the object, The transport amount of the object acquired by the transport amount acquisition unit and the motion amount of the object acquired by the motion amount acquisition unit The state of the corresponding object is suitable for subsequent processing. and determining whether or not

[0010] A program according to one aspect of the present invention includes a computer, which is configured to control a computer to execute a program for executing ... A quantity indicating the amount of the object being transported by the transport facility. Transport amount As Based on a conveyance amount acquisition means for acquiring the conveyance amount and a plurality of images captured at different times by the image acquisition means, A value obtained by quantifying the movement of the object between two images indicates a value obtained by quantifying the movement of the object being transported by the transport equipment. momentum As An exercise amount acquisition means for acquiring the exercise amount; Using a machine learning model that has learned a correspondence relationship between the transport amount of the object, the motion amount of the object, and the state of the object, The transport amount of the object acquired by the transport amount acquisition means and the momentum of the object acquired by the momentum acquisition means The state of the corresponding object is suitable for subsequent processing. The state determination means determines whether or not the state is correct. Effect of the Invention

[0011] According to the present invention, it is possible to reduce the burden on the personnel operating the equipment and to save labor. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of a configuration of a state determination system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of a state determination device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of a dehydrated sludge image according to an embodiment of the present invention. [Figure 4] FIG. 11 is a diagram showing an example of a classification image according to an embodiment of the present invention. [Diagram 5] 4 is a flowchart showing a process flow in the state determination system according to the embodiment of the present invention. [Figure 6] FIG. 13 is a diagram showing an F value of a determination result according to an embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing a confusion matrix of a determination result according to an embodiment of the present invention. [Figure 8] FIG. 13 is a graph showing a determination result when a moving average value according to an embodiment of the present invention is used. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] <1. Overview> The present invention relates to a state determination system for determining whether or not a state of a specific object satisfies a condition required by a destination device in a transport facility for transporting the object. Examples of the object include any substance having a certain viscosity, such as a resin material, a cosmetic cream, a food, sludge, a high molecular weight polymer, a gel electrolyte, etc. However, the object is not limited to the above-mentioned substances.

[0015] The destination device in the present invention may be any device that performs some kind of treatment on the object or stores it, and is appropriately selected depending on the use of the object. Examples include, but are not limited to, drying devices, mixing devices, separation devices, filtration devices, incinerators, roasting furnaces, etc.

[0016] The condition required for the state of the object is, for example, whether the property of the object is suitable for the subsequent processing. The conditions for determining whether the state is suitable are, specifically, whether the moisture content of the object is suitable, whether the size of the object is suitable (presence or absence of large lumps), etc. Whether or not the state of the object satisfies the required conditions may affect the processing of the subsequent process. Therefore, it is necessary to determine whether or not the state of the object satisfies the required conditions. For example, a facility manager or operator can determine whether or not to supply the object to a downstream facility based on the result of the determination of the object's state, and control the operation of each facility.

[0017] An example of application of the condition determination system according to this embodiment to a water treatment plant will be described below. Specifically, the condition determination system is applied to a process in which dewatered sludge (an example of a target object) is transported from a dehydrator (an example of an upstream equipment) to a dryer (an example of a downstream equipment) in a water treatment plant. In this process, the condition determination system determines the condition of the dewatered sludge transported from the dehydrator to the dryer by a screw conveyor (an example of a transport equipment). It should be noted that the target of the state determination system, the upstream equipment, the downstream equipment, the transport equipment, and the target object are not limited to these examples.

[0018] <2. Configuration of the condition determination system> The outline of the present invention has been described above. Next, the configuration of a condition determination system according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of a condition determination system 1 according to an embodiment of the present invention. 1, the condition determination system 1 includes an imaging device 10, a condition determination device 20, and an output device 30. Dehydrated sludge discharged from a dehydrator 2 is transported to a dryer 4 by a screw conveyor 3.

[0019] The imaging device 10 is a camera that captures an image (still image or video) of the dewatered sludge being transported (transport state). As an example, the imaging device 10 according to the present embodiment captures a video of the dewatered sludge being transported. The frame rate of the video is, for example, 30 fps (frames per second), but is not limited to this example. The imaging device 10 is provided so as to be able to capture an image of the dewatered sludge being transported on the screw conveyor 3 that transports the dewatered sludge from the dehydrator 2 to the dryer 4. The position at which the imaging device 10 is provided may be any position as long as it is possible to capture an image of the dewatered sludge being transported. For example, the imaging device 10 may be provided independently of the screw conveyor 3, or may be attached to the screw conveyor 3. When the imaging device 10 is attached to the screw conveyor 3, the imaging device 10 may be provided inside or outside the screw conveyor. The imaging device 10 is communicatively connected to the state determination device 20. The imaging device 10 transmits to the state determination device 20 an image of the dehydrated sludge that it has captured (hereinafter, also referred to as a “dehydrated sludge image”).

[0020] The state determining device 20 is a device that determines whether or not the state of the dewatered sludge satisfies a required condition. The state determining device 20 is, for example, a terminal such as a PC (Personal Computer) or a server. The state determination device 20 receives the dehydrated sludge images captured by the imaging device 10 from the imaging device 10. The state determination device 20 according to the present embodiment determines the state of the dehydrated sludge based on the dehydrated sludge images received from the imaging device 10, as an example. The status determination device 20 is communicatively connected to the output device 30. The status determination device 20 transmits output information to the output device 30. As an example, the status determination device 20 according to the present embodiment transmits, as output information, determination result information indicating the determination result of the state of the dewatered sludge to the output device 30.

[0021] The output device 30 is a device that outputs output information received from the status determination device 20. The output device 30 is, for example, a terminal such as a PC, a smartphone, a tablet terminal, or a terminal provided in each facility. The output device 30 outputs the output information by a display device such as a display or an audio output device such as a speaker provided in the terminal. As an example, the output device 30 according to the present embodiment displays the determination result information received from the status determination device 20 on a display device.

[0022] <3. Functional configuration of the state determination device> The above has described the configuration of the state determination system 1. Next, the functional configuration of the state determination device 20 will be described with reference to Fig. 2 to Fig. 4. Fig. 2 is a block diagram showing an example of the functional configuration of the state determination device 20 according to an embodiment of the present invention. As shown in FIG. 2, the state determining device 20 includes a communication unit 200, a control unit 210, and a storage unit 220.

[0023] (1) Communications Department 200 The communication unit 200 has a function of communicating with an external device. The communication unit 200 outputs information received from the external device to the control unit 210. The external device is a device provided outside the state determination device 20. The external device includes, for example, the imaging device 10, the output device 30, and a device to be controlled.

[0024] The communication unit 200 outputs information received from an external device to the control unit 210. For example, the communication unit 200 receives a dewatered sludge image captured by the imaging device 10 from the imaging device 10, and outputs the image to the image acquisition unit 211 of the control unit 210.

[0025] Furthermore, the communication unit 200 transmits information input from the control unit 210 to an external device. For example, the communication unit 200 transmits output information input from the output processing unit 215 of the control unit 210 to the output device 30.

[0026] The communication by the communication unit 200 may be realized by either wired communication or wireless communication.

[0027] (2) Control unit 210 The control unit 210 has a function of controlling the overall operation of the state determination device 20. The control unit 210 is realized, for example, by causing a CPU (Central Processing Unit) provided as hardware in the state determination device 20 to execute a program. As shown in FIG. 2, the control unit 210 includes an image acquisition unit 211 , a transport amount acquisition unit 212 , an exercise amount acquisition unit 213 , a state determination unit 214 , and an output processing unit 215 .

[0028] (2-1) Image acquisition unit 211 The image acquisition unit 211 has a function of acquiring dehydrated sludge images obtained by capturing images of dehydrated sludge being transported on the screw conveyor 3. For example, the image acquisition unit 211 acquires the dehydrated sludge images from the imaging device 10 via the communication unit 200. Specifically, the image acquisition unit 211 acquires a plurality of dehydrated sludge images captured by the imaging device 10 at regular time intervals over a predetermined period of time. In this embodiment, the plurality of dehydrated sludge images are still images each constituting a video captured by the imaging device 10. The image acquisition unit 211 outputs the acquired dehydrated sludge images to the transport amount acquisition unit 212 and the momentum acquisition unit 213.

[0029] Here, a dehydrated sludge image according to an embodiment of the present invention will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of a dehydrated sludge image according to the present embodiment. In FIG. 3, one frame of a plurality of frames constituting a video captured by the imaging device 10 is shown as a dewatered sludge image 50. The dewatered sludge image 50 includes dewatered sludge 51, a blade 52, a blade 53, a non-target object 54, and a background 55. The dewatered sludge 51 is sludge being transported by the screw conveyor 3. The blade 52 and the blade 53 are parts of the screw of the screw conveyor 3. The non-target object 54 is an object other than the dewatered sludge 51, the blade 52, and the blade 53 being transported. For example, the non-target object 54 is dewatered sludge attached to the inner wall of the screw conveyor 3. The background 55 is an area that does not correspond to any of the dewatered sludge 51, the blade 52, the blade 53, and the non-target object 54. For example, it is an area in the dewatered sludge image 50 where no object is captured.

[0030] (2-2) Transport amount acquisition unit 212 The conveying amount acquisition unit 212 has a function of acquiring the amount of dehydrated sludge (conveying amount) being conveyed by the screw conveyor 3. For example, the conveying amount acquisition unit 212 acquires the amount of dehydrated sludge in the image as the conveying amount based on the dehydrated sludge image acquired by the image acquisition unit 211. Specifically, the conveying amount acquisition unit 212 first classifies the area of ​​the dehydrated sludge image by elements included in the image. Here, the elements included in the image are, for example, the dehydrated sludge 51, the blade 52, the blade 53, the non-target object 54, and the background 55 in the dehydrated sludge image 50 shown in FIG. 3. Note that the elements included in the image are not limited to such an example. The image in which the conveying amount acquisition unit 212 classifies the area of ​​the dehydrated sludge image by elements is also referred to as a "classified image" below.

[0031] Here, a classification image according to an embodiment of the present invention will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a classification image according to an embodiment of the present invention. FIG. 4 shows a classification image 60 in which the regions of the dewatered sludge image 50 shown in FIG. 3 have been classified by element. Five regions, region 61, region 62, region 63, region 64, and region 65, are shown in classification image 60. Region 61 is the region of dewatered sludge 51 in the dewatered sludge image 50 of FIG. 3. Region 62 is the region of blade 52 in the dewatered sludge image 50 of FIG. 3. Region 63 is the region of blade 53 in the dewatered sludge image 50 of FIG. 3. Region 64 is the region of non-target object 54 in the dewatered sludge image 50 of FIG. 3. Region 65 is the region of background 55 in the dewatered sludge image 50 of FIG. 3.

[0032] After classifying the areas of the dehydrated sludge image by element, the transport amount acquisition unit 212 calculates (quantifies) the transport amount of the dehydrated sludge based on the number of pixels included in the areas where the elements are dehydrated sludge. Specifically, the transport amount acquisition unit 212 calculates the number of pixels included in the dehydrated sludge area of ​​the dehydrated sludge image as the transport amount. The transport amount acquisition unit 212 may also calculate the proportion of pixels in the dehydrated sludge area to all pixels of the dehydrated sludge image as the transport amount.

[0033] The transport amount acquisition unit 212 acquires the transport amount of the dewatered sludge using, for example, a machine learning model. The machine learning model is a model that learns the correspondence between each pixel of the dewatered sludge image and the element indicated by each pixel. When the dewatered sludge image acquired by the image acquisition unit 211 is input, the machine learning model outputs a classification image. The transport amount acquisition unit 212 can calculate the transport amount of the dewatered sludge by counting the pixels in the area of ​​the dewatered sludge indicated by the classification image output by the machine learning model.

[0034] The machine learning model used by the conveyance amount acquisition unit 212 is, for example, a model (hereinafter, also referred to as an "SS model") obtained by performing machine learning using semantic segmentation (SS). Semantic segmentation is a deep learning algorithm that associates a label or a category with every pixel in an image.

[0035] The SS model uses dehydrated sludge images as learning data during learning. When a dehydrated sludge image is input as input data, the trained SS model (hereinafter also referred to as a "trained SS model") outputs a result of analogy based on the input dehydrated sludge image as output data. The SS model of this embodiment trains, for example, to use a dehydrated sludge image as input data and output a classification image as output data.

[0036] In generating a trained SS model, for example, supervised learning is performed. In supervised learning, a learning model is made to learn using a learning dataset. The dataset is a set of input data (explanatory variables) and training data (objective variables) corresponding to the input data. The input data is data that is input during learning. The input data in this embodiment is, for example, a dehydrated sludge image. The training data is data indicating the correct answer of the output data that is output based on the input data. The training data in this embodiment is, for example, an image in which information indicating the correct answer of the element indicated by each pixel is added to each pixel of the dehydrated sludge image that is the input data. The correct answer indicated by the training data can be indicated, for example, by coloring the dehydrated sludge image according to each element.

[0037] In addition, the transport amount acquisition unit 212 may calculate a moving average (MA) of the transport amount of dehydrated sludge obtained for each of multiple dehydrated sludge images acquired during a specified period, and use the calculated moving average as the transport amount to be finally output.

[0038] (2-3) Momentum acquisition unit 213 The momentum acquisition unit 213 has a function of acquiring the momentum of the dewatered sludge being transported by the screw conveyor 3. The momentum acquisition unit 213 acquires the momentum of the dewatered sludge, for example, by using optical flow. Optical flow is a technique for quantifying the movement of an object between two images (for example, between frames of a video). In this embodiment, the momentum acquisition unit 213 quantifies the direction and displacement of the movement of the dewatered sludge by using optical flow, and calculates the average and deviation of the quantified direction and displacement. The analysis results by optical flow are determined by an optical flow algorithm and are calculated for each pixel.

[0039] Specifically, the momentum acquisition unit 213 acquires the momentum of the dehydrated sludge in the image based on multiple dehydrated sludge images captured at different times and acquired by the image acquisition unit 211. More specifically, the momentum acquisition unit 213 compares a first dehydrated sludge image acquired by the image acquisition unit 211 with a second dehydrated sludge image acquired before the first dehydrated sludge image, and calculates the movement direction and movement amount of the feature point. When there are multiple combinations of the first dehydrated sludge image and the second dehydrated sludge image among the multiple dehydrated sludge images acquired during a predetermined period, the momentum acquisition unit 213 calculates the movement direction and movement amount of the dehydrated sludge for each combination. Then, the momentum acquisition unit 213 calculates the average and deviation of the multiple obtained movement directions and movement amounts as the momentum to be output.

[0040] Among the elements included in the dehydrated sludge image, in addition to the dehydrated sludge being transported, the momentum of the blade may also change. In this embodiment, it is assumed that the rotation speed of the blade is kept constant. In this case, the momentum of the blade is always calculated as a constant value. Therefore, in a situation where the dehydrated sludge is not being transported, the momentum acquisition unit 213 calculates only the constant momentum of the blade. On the other hand, in a case where the dehydrated sludge is being transported, the momentum acquisition unit 213 calculates the momentum of the dehydrated sludge added to the constant momentum of the blade. In other words, if the momentum of the blade is known in advance, the momentum of the dehydrated sludge may be calculated by excluding the momentum of the blade from the momentum of the entire dehydrated sludge image. Thus, in this embodiment, the momentum acquisition unit 213 does not need to calculate the momentum of the dehydrated sludge and the momentum of the blade separately. Furthermore, since the momentum of the blade is always calculated as a constant value and the momentum of the blade is contained in each of the multiple dehydrated sludge images taken at different times by the image acquisition unit 211, it goes without saying that it is possible to calculate the direction and amount of movement of the feature point by comparing the first dehydrated sludge image with the second dehydrated sludge image acquired before the first dehydrated sludge image, as described above.

[0041] In addition, the momentum acquisition unit 213 may calculate a moving average value of the momentum of the dehydrated sludge obtained for each of multiple dehydrated sludge images acquired during a specified period, and use the calculated moving average value as the momentum to be finally output.

[0042] (2-4) State determination unit 214 The state determination unit 214 has a function of determining the state of the dewatered sludge. For example, the state determination unit 214 determines whether the state of the dewatered sludge satisfies the required conditions based on the transport amount of the dewatered sludge acquired by the transport amount acquisition unit 212 and the momentum of the dewatered sludge acquired by the momentum acquisition unit 213. Then, the state determination unit 214 outputs the determination result information indicating whether the state of the dewatered sludge satisfies the required conditions to the output processing unit 215. In the present embodiment, the state determination unit 214 outputs one of three types of determination result information, for example, OK, GRAY, and NG. OK indicates that the state of the dewatered sludge satisfies the required conditions. GRAY indicates that it is unclear whether the state of the dewatered sludge satisfies the required conditions. NG indicates that the state of the dewatered sludge does not satisfy the required conditions.

[0043] Specifically, the state determination unit 214 determines the state of the dewatered sludge using a machine learning model. The machine learning model is a model that learns the correspondence between the transport amount of the dewatered sludge, the momentum of the dewatered sludge, and the state of the dewatered sludge. When the transport amount of the dewatered sludge acquired by the transport amount acquisition unit 212 and the momentum of the dewatered sludge acquired by the momentum acquisition unit 213 are input, the machine learning model outputs information indicating whether the state of the dewatered sludge satisfies a required condition.

[0044] The machine learning model used by the state determination unit 214 is, for example, a model obtained by machine learning using a support vector machine (SVM) (hereinafter, also referred to as an "SVM model"). The SVM model uses the transport amount and momentum of the dewatered sludge as learning data during learning. When the transport amount and momentum of the dewatered sludge are input as input data, the trained SVM model (hereinafter also referred to as the "trained SVM model") outputs, as output data, a result inferred based on the input transport amount and momentum of the dewatered sludge. The SVM model of this embodiment, for example, learns to use the transport amount and momentum of the dewatered sludge as input data and output information indicating whether the state of the dewatered sludge satisfies the required conditions as output data.

[0045] In generating a trained SVM model, for example, supervised learning is performed. In supervised learning, a learning model is made to learn using a learning dataset. The dataset is a set of input data (explanatory variables) and training data (objective variables) corresponding to the input data. The input data is data that is input during learning. The input data in this embodiment is, for example, the transport amount and momentum of dewatered sludge in a certain dewatered sludge image. The training data is data that indicates the correct answer of the output data that is output based on the input data. The training data in this embodiment is, for example, the result of a human judgment (OK, GRAY, or NG) of the state of the dewatered sludge in the dewatered sludge image that is the subject of calculation of the transport amount and momentum of the dewatered sludge, which is the input data.

[0046] The state determination unit 214 may determine the state of the dewatered sludge based on the moving average value of the transport amount of the dewatered sludge and the moving average value of the momentum of the dewatered sludge in a predetermined period. When a person visually determines the state of the dewatered sludge, the person often observes the manner in which the dewatered sludge is transported for a certain period of time to determine the state. The moving average value of the transport amount of the dewatered sludge and the moving average value of the momentum are calculated based on the dewatered sludge image in a predetermined period of time. That is, by using the moving average value of the transport amount of the dewatered sludge and the moving average value of the momentum, the state determination unit 214 can determine the state of the dewatered sludge in a state closer to the state in which a person visually determines the state of the dewatered sludge. As a result, the state determination unit 214 can improve the accuracy of determining the state of the dewatered sludge by using the moving average value of the transport amount of the dewatered sludge and the moving average value of the momentum.

[0047] (2-5) Output processing unit 215 The output processing unit 215 has a function of outputting output information to the output device 30. For example, the output processing unit 215 causes the output device 30 to output the determination result information output from the state determination unit 214. For example, the output processing unit 215 transmits the determination result information to the output device 30 via the communication unit 200 and causes it to be displayed.

[0048] (3) Storage section 220 The storage unit 220 has a function of storing various information. For example, the storage unit 220 stores various models. Specifically, the storage unit 220 stores a machine learning model.

[0049] The memory unit 220 is configured with a storage medium, such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media.

[0050] <4. Processing flow> The functional configuration of the state determination device 20 according to this embodiment has been described above. Next, the flow of processing in the state determination system 1 according to the embodiment of the present invention will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of processing in the state determination system 1 according to this embodiment.

[0051] As shown in FIG. 5, first, the imaging device 10 captures an image of the dewatered sludge being transported by the screw conveyor 3 (step S101).

[0052] Next, the image acquisition unit 211 of the state determination device 20 acquires the dehydrated sludge image captured by the imaging device 10 from the imaging device 10 via the communication unit 200 (step S102). The image acquisition unit 211 outputs the acquired dehydrated sludge image to the transport amount acquisition unit 212 and the motion amount acquisition unit 213 of the state determination device 20.

[0053] Next, the transport amount acquisition unit 212 acquires the transport amount of the dewatered sludge by inputting the dewatered sludge image to the trained SS model (step S103).

[0054] Furthermore, in parallel with the processing by the transport amount obtaining unit 212, the momentum obtaining unit 213 obtains the momentum of the dewatered sludge by optical flow (step S104).

[0055] Next, the state determination unit 214 determines the state of the dewatered sludge based on the transport amount acquired by the transport amount acquisition unit 212 and the exercise amount acquired by the exercise amount acquisition unit 213 (step S105). After the determination, the state determination unit 214 outputs the determination result information.

[0056] Next, the output processing unit 215 transmits the determination result information output by the state determination unit 214 from the communication unit 200 to the output device 30 (step S106).

[0057] Then, the output device 30 outputs the received determination result information (step S107).

[0058] <5. Examples> The process flow in the state determination system 1 according to the present embodiment has been described above. Next, an example of the state determination system 1 according to the present embodiment will be described with reference to Figs. 6 to 8. Fig. 6 is a diagram showing an F-value of the determination result according to the example of the present invention. Fig. 7 is a diagram showing a confusion matrix of the determination result according to the example of the present invention. Fig. 8 is a diagram showing a graph of the determination result when a moving average value according to the example of the present invention is used.

[0059] 6 shows the result of the state determination unit 214 determining whether the state of the dewatered sludge is OK or other than OK (GRAY or NG), and the result of the state determination unit 214 determining whether the state of the dewatered sludge is other than NG (OK or GRAY) or NG. Furthermore, each result shows the F value (SVM) of the determination result obtained by inputting the transport amount acquired by the transport amount acquisition unit 212 and the motion amount acquired by the motion amount acquisition unit 213 directly into the trained SVM model, and the F value (SVM+MA) of the determination result obtained by inputting the moving average value of the transport amount acquired by the transport amount acquisition unit 212 and the moving average value of the motion amount acquired by the motion amount acquisition unit 213 into the trained SVM model.

[0060] As shown in Figure 6, when determining whether the state of the dewatered sludge was OK or other than OK (GRAY or NG), the F value of SVM was 0.46, and the F value of SVM+MA was 0.88. When determining whether the state of the dewatered sludge was other than NG (OK or GRAY) or NG, the F value of SVM was 0.52, and the F value of SVM+MA was 0.66. From the above, it can be seen that in either case, using the moving average value improves the accuracy of the judgment. Also, when using the moving average value, it can be seen that a judgment result of OK or other than OK (GRAY or NG) is more accurate than a judgment result of other than NG (OK or GRAY) or NG. Therefore, when the priority is to operate the equipment safely, it is considered more practical to judge whether the state of the dewatered sludge is OK or other than OK (GRAY or NG).

[0061] FIG. 7 shows a confusion matrix when the state determination unit 214 performs a determination of OK or non-OK (GRAY or NG) using a moving average value (that is, SVM+MA of OK, GRAY or NG). As shown in FIG. 7, the number of cases where the status determining unit 214 determined the true value to be OK among cases where the true value was OK, that is, the number of cases where the status determining unit 214 determined the true positive, was 1867. The number of results judged by the state judging unit 214 as GRAY or NG among results whose true value was OK, that is, the number of results judged as false negatives, was 217. The number of times that the status determining unit 214 determined that the true value was OK out of GRAY or NG, that is, the number of times that the status determining unit 214 determined that the true value was false positive, was 117. The number of cases in which the state determining unit 214 determined that the true value was GRAY or NG, that is, the number of cases in which the state determining unit 214 determined that the true value was GRAY or NG, was 482. The F-score of 0.88 for SVM+MA when determining whether the state of the dewatered sludge is OK or not OK (GRAY or NG), as shown in Figure 6, is a value calculated based on the confusion matrix in Figure 7.

[0062] FIG. 8 shows graphs illustrating the time series changes in the determination result, the moving average value, and the true value when state determination section 214 makes a determination using the moving average value. The vertical axis in FIG. 8 indicates true values, and the horizontal axis indicates measurement time. A value of 1.0 on the vertical axis indicates that the state of the dewatered sludge is OK, and -1.0 indicates that the state of the dewatered sludge is NG. In FIG. 8, the dashed-dotted line graph indicates the results of an actual judgment made by an operator from the images. The solid line graph indicates the results of judgment made by the state judgment unit 214 for each image. The dashed line graph indicates a moving average value of the judgment results made by the state judgment unit 214. By calculating the moving average, a state judgment similar to the judgment result made by an operator can be made even if there is variation in the transport amount of the transported goods.

[0063] As described above, the condition determining device 20 according to the embodiment of the present invention includes the image acquiring unit 211 , the transport amount acquiring unit 212 , the exercise amount acquiring unit 213 , and the condition determining unit 214 . The image acquisition unit 211 acquires dehydrated sludge images from an imaging device 10 that is configured to capture images of the dehydrated sludge being transported on the screw conveyor 3 that transports the dehydrated sludge from the dehydrator 2 to the dryer 4. The transport amount acquisition unit 212 acquires the transport amount of the dewatered sludge in the image based on the dewatered sludge image acquired by the image acquisition unit 211. The momentum acquisition unit 213 acquires the momentum of the dewatered sludge in the images based on a plurality of dewatered sludge images captured at different times by the image acquisition unit 211 . The state determination unit 214 determines whether the state of the dewatered sludge satisfies the required conditions based on the amount of dewatered sludge transported acquired by the transport amount acquisition unit 212 and the momentum of the dewatered sludge acquired by the momentum acquisition unit 213.

[0064] With this configuration, the state of the dewatered sludge being transported by the screw conveyor 3 is determined by the state determination device 20. This eliminates the need for a person to visually determine the state of the dewatered sludge. In other words, there is no need to prepare personnel to visually determine the state of the dewatered sludge.

[0065] Therefore, the state determination device 20 according to the embodiment of the present invention makes it possible to reduce the burden on the personnel operating the equipment and to save labor.

[0066] <5. Modifications> The above describes the embodiments of the present invention. Next, modified examples of the embodiments of the present invention will be described. Each modified example described below may be applied alone to the embodiments of the present invention, or may be applied in combination to the embodiments of the present invention. Furthermore, each modified example may be applied in place of the configuration described in the embodiment of the present invention, or may be applied in addition to the configuration described in the embodiment of the present invention.

[0067] (1) First Modification In the above embodiment, an example has been described in which the condition determination system 1 is applied to a water treatment plant and the target is dewatered sludge, but the present invention is not limited to such an example. For example, the specific target may be one that corresponds to the process to which the target is transported. The condition determination system 1 may also be applied to, for example, a food plant. In this case, one example of the target is a paste.

[0068] (2) Second modified example In the above embodiment, an example has been described in which the conveying equipment is the screw conveyor 3, but the conveying equipment is not limited to this example. For example, the conveying equipment may be a belt conveyor or the like as long as the equipment has a configuration (e.g., a switching damper) that changes at least the momentum of the object while the object is being conveyed.

[0069] (3) Third modified example In the above embodiment, an example has been described in which the dehydrated sludge images captured by the imaging device 10 are moving images, but the present invention is not limited to such an example. For example, the imaging device 10 may capture still images of the dehydrated sludge being transported on the screw conveyor 3. In this case, the imaging device 10 captures still images at regular time intervals to capture multiple dehydrated sludge images. As a result, the image acquisition unit 211 can acquire multiple dehydrated sludge images captured at regular time intervals over a predetermined period of time, even when the imaging device 10 captures still images.

[0070] (4) Fourth modified example In the above embodiment, the state determination unit 214 uses the SVM model as a machine learning model, but is not limited to this example. For example, the state determination unit 214 may use a model that has been machine-learned using a decision tree.

[0071] (5) Fifth Variation In the above embodiment, an example has been described in which the output processing unit 215 outputs the determination result information to the output device 30 capable of outputting the determination result information, but the present invention is not limited to such an example. For example, the output processing unit 215 may output the determination result information to an external device capable of controlling the operation of the equipment based on the determination result information. This allows the external device to control the equipment based on the determination result information received from the status determination device 20. Furthermore, the output processing unit 215 may generate a control signal according to the equipment to be controlled based on the determination result information and output the generated control signal to an external device capable of controlling the operation of the equipment. This allows the external device to control the equipment based on the control signal received from the status determination device 20. As a result, it is possible to eliminate the need for a person to control the equipment based on the judgment result information, thereby enabling automation of the equipment operation and reducing the number of operators.

[0072] The above describes the modified examples of the present invention. Note that a part or all of the configuration of the state determination device 20 in the above embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. Note that the "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" may include a device that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a device that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. In addition, the above program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0073] Furthermore, the state determination device 20 in the above-described embodiment or a control device that realizes at least the functions of the control unit 210 of the state determination device 20 is provided in a plant or the like in which the industrial machine to be controlled is installed. The location in which the state determination device 20 or the control device is provided is not limited to this example, and the state determination device 20 or the control device may be provided on a cloud. For example, when the state determination device 20 or the control device is a server device, the state determination device 20 or the control device may be a server device provided in a plant, or may be a server device provided on a cloud (i.e., a cloud server). In addition, all of the functions of the units constituting the state determination device 20 or the control device may be realized by devices provided in the plant, or may be realized by devices provided on the cloud. In addition, some of the functions of the units constituting the state determination device 20 or the control device may be realized by devices provided in the plant, and the remaining functions may be realized by devices provided on the cloud.

[0074] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0075] 1. Status Judgment System 2 Dehydrator 3. Screw conveyor 4 Dryer 10. Imaging device 20 Status determination device 30 Output Device 200 Communications Department 210 Control section 211 Image acquisition unit 212 Transport amount acquisition unit 213 Momentum acquisition unit 214 Status determination unit 215 Output Processing Unit 220 Storage section

Claims

1. In a conveying facility that conveys an object from a facility in a previous stage to a facility in a subsequent stage, an image acquiring unit that acquires an image of the object from an imaging device that is provided so as to be able to image a state in which the object is conveyed; a transport amount acquisition unit that acquires an amount of the object in the image based on the image acquired by the image acquisition unit as a transport amount indicating an amount of the object being transported by the transport facility; a motion amount acquisition unit that acquires a value that quantifies the motion of the object between two images based on the plurality of images captured at different times acquired by the image acquisition unit, as a motion amount that indicates the quantified value of the motion of the object being transported by the transport facility; a state determination unit that uses a machine learning model that has learned a correspondence relationship between the transport amount of the object, the motion amount of the object, and the state of the object to determine whether or not the state of the object corresponding to the transport amount of the object acquired by the transport amount acquisition unit and the motion amount of the object acquired by the motion amount acquisition unit is suitable for a subsequent process; A state determination device comprising:

2. the image acquisition unit acquires a plurality of images captured by the imaging device at regular time intervals over a predetermined period of time; the transport amount acquisition unit calculates a moving average value of the transport amount of the object obtained for each of the plurality of images acquired during the predetermined period; the motion amount acquisition unit calculates a moving average value of the motion amount of the object obtained for each of the plurality of images acquired during the predetermined period; the state determination unit determines the state of the object based on a moving average value of a transport amount of the object and a moving average value of a momentum amount of the object during the predetermined period. The state determination device according to claim 1 .

3. The machine learning model used by the state determination unit is a model that has been machine-learned using a support vector machine. The state determination device according to claim 1 .

4. the carry distance acquisition unit classifies an area of ​​the image into elements included in the image, and calculates a carry distance of the object based on a number of pixels included in an area in which the element is the object. The state determination device according to claim 1 or 2.

5. the transport amount acquisition unit calculates the transport amount of the object using a machine learning model that has learned a correspondence relationship between each pixel of the image of the object and the element represented by each pixel; The state determination device according to claim 4.

6. The machine learning model used by the transport amount acquisition unit is a model that has been machine-learned by semantic segmentation. The state determination device according to claim 5 .

7. the motion amount acquisition unit calculates the motion amount of the object based on a moving direction and a moving amount of a feature point when a first image acquired by the image acquisition unit is compared with a second image acquired before the first image; The state determination device according to any one of claims 1 to 6.

8. The motion amount acquisition unit acquires the motion amount of the object by using an optical flow. The state determination device according to claim 7.

9. an output processing unit that outputs a result of the determination by the state determination unit to an external device capable of controlling an operation of the facility based on the result of the determination; The state determination device according to claim 1 , further comprising:

10. a step in which an image acquisition unit acquires an image of an object from an imaging device provided in a conveying facility that conveys the object from a facility in a previous stage to a facility in a subsequent stage, the imaging device being capable of capturing an image of the object being conveyed; a conveyance amount acquisition unit acquiring, based on the image acquired by the image acquisition unit, an amount of the object in the image as a conveyance amount indicating an amount of the object being conveyed by the conveyance facility; a step in which an amount of motion acquiring unit acquires a value obtained by quantifying the movement of the object between two images based on a plurality of images captured at different times by the image acquiring unit, as an amount of motion indicating a value obtained by quantifying the movement of the object being transported by the transport facility; a state determination unit determining whether or not a state of the object corresponding to the transport amount of the object acquired by the transport amount acquisition unit and the momentum of the object acquired by the momentum acquisition unit is suitable for a subsequent process by using a machine learning model that has learned a correspondence relationship between the transport amount of the object, the momentum of the object, and the state of the object; A state determination method comprising:

11. Computer, In a conveying facility that conveys an object from a facility in a previous stage to a facility in a subsequent stage, an image acquiring means that acquires an image of the object from an imaging device that is provided so as to be able to capture an image of the object being conveyed; a conveyance amount acquiring means for acquiring an amount of the object in the image based on the image acquired by the image acquiring means as a conveyance amount indicating an amount of the object being conveyed by the conveyance facility; a motion amount acquiring means for acquiring a value obtained by quantifying the motion of the object between two images based on a plurality of images captured at different times by the image acquiring means as a motion amount indicating the value obtained by quantifying the motion of the object being transported by the transport facility; a state determination means for determining whether or not a state of the object corresponding to the transport amount of the object acquired by the transport amount acquisition means and the momentum of the object acquired by the momentum acquisition means is suitable for a subsequent process, using a machine learning model that has learned a correspondence relationship between the transport amount of the object, the momentum of the object, and the state of the object; A program that functions as a

Images